Administration

Site Admin Interface

Allura has an admin interface at http://MYSITE/nf/admin/ You must be an admin of the /p/allura project on the site to access it. If you want to use another project to control admin access, change the site_admin_project setting in development.ini.

The admin interface allows you to:

• View newly registered projects

• Search for projects

• Delete projects

• View neighborhood total stats

• Search for users, view user details, update user status, email address, and reset their password

• View background task statuses, and submit new background tasks

• Manage “trove” categories (for user skill choices)

• Subscribe a user to an artifact

• Reclone a repository

• Manage site notifications

Customizing appearance

Global navigation

Allura supports adding global navigation links which will be displayed in the header of every page.

To set up this add global_nav option to [app:main] section of your development.ini. It should be a JSON list of dicts like the following:

[app:main]
...
global_nav = [{"title": "Example", "url": "http://example.com"}, {"title": "Another", "url": "http://another.com"}]

Site logo

You can set up logo to be displayed in the top left corner of the site.

Add the following to your development.ini:

[app:main]
...
logo.link = /          ; link to attach to the logo (optional, defaults to "/")
logo.path = sf10a.png  ; fs path to the logo image, relative to Allura/allura/public/nf/images/
logo.width = 78        ; logo width in pixels (optional)
logo.height = 30       ; logo height in pixels (optional)

Commands, Scripts, and Tasks

Overview

Allura has many commands and scripts that can be run from the server commandline to administrate Allura. There are also tasks that can be run through the taskd system in the background. These tasks can be submitted via the web at http://MYSITE/nf/admin/task_manager Some paster scripts have been set up so that they are runnable as tasks too, giving you the convenience of starting them through the web and letting taskd execute them, rather than from a server shell.

Commands can be discovered and run via the paster command when you are in the ‘Allura’ directory that has your .ini file. For example:

 paster help
... all commands listed here ...

 paster create-neighborhood --help
... specific command help ...

 paster create-neighborhood development.ini myneighborhood myuser ...

Scripts are in the scripts/ directory and run slightly differently, via paster script. An extra -- is required to separate script arguments from paster arguments. Example:

 paster script development.ini ../scripts/add_user_to_group.py -- --help
... help output ...

 paster script development.ini ../scripts/add_user_to_group.py -- --nbhd /u/ johndoe Admin

To run these when using docker, prefix with docker compose run taskd and use docker-dev.ini like:

docker compose run --rm taskd paster create-neighborhood docker-dev.ini myneighborhood myuser ...

Or with the docker production setup:

docker compose run --rm oneoff paster create-neighborhood /allura-data/production.ini myneighborhood myuser ...

Tasks can be run via the web interface at http://MYSITE/nf/admin/task_manager You must know the full task name, e.g. allura.tasks.admin_tasks.install_app You can optionally provide a username and project and app which will get set on the current context (c). You should specify what args and kwargs will be passed as parameters to the task. They are specified in JSON format on the form. If you are running a script via this interface, the args/kwargs JSON should be like:

{
    "args": ["--foo --bar baz"],
    "kwargs": {}
}

See the listing of some available tasks.

Available scripts and commands are:

create-neighborhood

Usage: paster create-neighborhood [options] <ini file> <neighborhood_shortname> <admin1>
[<admin2>...]
Create a new neighborhood with the listed admins

Options:
  -h, --help     show this help message and exit
  -v, --verbose

ensure_index

Usage: paster ensure_index [options] [<ini file>]
Create all the Mongo indexes specified by Ming models

Options:
  -h, --help     show this help message and exit
  -v, --verbose  
  --clean        Drop any unneeded indexes

ircbot

Usage: paster ircbot [options] <ini file>
For the ForgeChat tool.  Connect to all configured IRC servers and relay messages

Options:
  -h, --help            show this help message and exit
  -v, --verbose         
  -c CONTEXT, --context=CONTEXT
                        The context of the message (path to the project and/or
                        tool

reindex

Usage: paster reindex [options] <ini file>
Reindex into solr and re-shortlink all artifacts
By default both --refs and --solr are enabled, but if you specify just one, the other will be disabled

Options:
  -h, --help            show this help message and exit
  -v, --verbose         
  -p PROJECT, --project=PROJECT
                        project to reindex
  --project-regex=PROJECT_REGEX
                        Restrict reindex to projects for which the shortname
                        matches the provided regex.
  -n NEIGHBORHOOD, --neighborhood=NEIGHBORHOOD
                        neighborhood to reindex (e.g. p)
  --solr                Solr needs artifact references to already exist.
  --skip-solr-delete    Skip clearing solr index.
  --refs                Update artifact references and shortlinks
  --tasks               Run each individual index operation as a background
                        task.  Note: this is often better, since tasks have
                        "request" objects which are needed for some markdown
                        macros to run properly
  --solr-hosts=SOLR_HOSTS
                        Override the solr host(s) to post to.  Comma-separated
                        list of solr server URLs
  --solr-creds=SOLR_CREDS
                        Creds for the solr host(s).  Comma-separated list of
                        user:pwd strings
  --max-chunk=MAX_CHUNK
                        Max number of artifacts to index in one Solr update
                        command
  --ming-config=MING_CONFIG
                        Path (absolute, or relative to Allura root) to .ini
                        file defining ming configuration.

set-neighborhood-features

Usage: paster set-neighborhood-features [options] <ini file> <neighborhood> <feature> <value>
Change the neighborhood features
	<neighborhood> - the neighborhood name or object id
	<feature> - feature value to change options are max_projects, css, google_analytics,
	or private_projects
	<value> - value to give the feature - see below for descriptions
	    max_projects - maximum projects allowed in neighborhood - specify None for no
	    limit
	    css - type of css customization - use "none", "picker", or "custom".
	    google_analytics - allow the user to use google analytics - True or False
	    private_projects - allow private projects in the neighborhood - True or False

Options:
  -h, --help	 show this help message and exit
  -v, --verbose

set-tool-access

Usage: paster set-tool-access [options] <ini file> <project_shortname> <neighborhood_name>
<access_level>...
Set the tool statuses that are permitted to be installed on a given project

Options:
  -h, --help     show this help message and exit
  -v, --verbose

taskd

Usage: paster taskd [options] [<ini file>]
Task server

Options:
  -h, --help     show this help message and exit
  -v, --verbose  
  --only=ONLY    only handle tasks of the given name(s) (can be comma-
                 separated list)
  --nocapture    Do not capture stdout and redirect it to logging.  Useful for
                 development with pdb.set_trace()

task

Usage: paster task [options] <ini file> [list|count|retry|purge|timeout|commit]

    list: prints tasks matching --state (default: 'ready') and filters
    count: counts tasks matching --state (default: 'ready') and filters
    retry: re-run tasks with 'error' state. --state has no effect
    purge: remove all tasks that match --state ( default: '*') with result_type "forget".
    timeout: retry all tasks with state 'busy' and older than --timeout seconds (does not
    stop existing task). --state has no effect
    commit: run a solr 'commit' as a background task

    All subcommands except 'commit' can use the --filter-... options.

Task command

Options:
  -h, --help            show this help message and exit
  -v, --verbose
  -s STATE, --state=STATE
                        state of processes for "list", "count", or "purge"
                        subcommands.  * means all. (Defaults per command:
                        list="ready", count="ready", purge="*")
  -t TIMEOUT, --timeout=TIMEOUT
                        timeout (in seconds) for busy tasks (only applies to
                        "timeout" command)
  --filter-name-prefix=FILTER_NAME_PREFIX
                        limit to task names starting with this.  Example
                        allura.tasks.index_tasks.
  --filter-result-regex=FILTER_RESULT_REGEX
                        limit to tasks with result matching this regex.
                        Example "pysolr"
  --filter-queued-days-ago=DAYS_AGO
                        limit to tasks queued more than NUM days ago.  Example
                        "180"
  --filter-queued-less-than-days=LESS_THAN_DAYS
                        limit to tasks queued less than NUM days ago.  Example
                        "0.5"

taskd_cleanup

Usage: paster taskd_cleanup [options] <ini file> [-k] <taskd status log file>
Tasks cleanup command.  Determines which taskd processes are handling tasks, and what has
been dropped or got hung.

Options:
  -h, --help            show this help message and exit
  -v, --verbose
  -k, --kill-stuck-taskd
                        automatically kill stuck taskd processes.  Be careful
                        with this, a taskd process may just be very busy on
                        certain operations and not able to respond to our
                        status request
  -n NUM_RETRY, --num-retry-status-check=NUM_RETRY
                        number of retries to read taskd status log after
                        sending USR1 signal (5 by default)

pull-rss-feeds

Blog tools may optionally be configured to fetch external RSS feeds. If that is in place, this command should be used to fetch all those rss feeds and convert new entries into blog posts.

Requires html2text, a GPL library.

cd ../ForgeBlog
paster pull-rss-feeds development.ini --help

disable_users.py

Can be run as a background task using task name: allura.scripts.disable_users.DisableUsers

Disable listed users

usage: paster script development.ini allura/scripts/disable_users.py -- [-h] [--message MESSAGE] --usernames <username> [<username> ...]

Named Arguments

--message, -m

Message to add to the audit log for each user (set to empty string to skip message altogether)

Default: “Account disabled”

--usernames

List of usernames, or “-” to read from stdin

delete_projects.py

Can be run as a background task using task name: allura.scripts.delete_projects.DeleteProjects

More convenient way to delete project is this site admin page. It uses this script under the hood.

Completely delete projects

usage: paster script development.ini allura/scripts/delete_projects.py -- [-h] [-r REASON] [--disable-users] nbhd/project [nbhd/project ...]

Positional Arguments

nbhd/project

List of projects to delete in a form nbhd_prefix/shortname

Named Arguments

-r, --reason

Reason why these projects are being deleted

--disable-users

Disable all users belonging to groups Admin and Developer in these projects.

Default: False

refreshrepo.py

Can be run as a background task using task name: allura.scripts.refreshrepo.RefreshRepo

Scan repos on filesytem and update repo metadata in MongoDB. Run for all repos (no args), or restrict by neighborhood, project, or code tool mount point.

usage: paster script development.ini allura/scripts/refreshrepo.py -- [-h] [--nbhd NBHD] [--project PROJECT] [--project-regex PROJECT_REGEX] [--repo-types REPO_TYPES] [--mount-point MOUNT_POINT] [--clean] [--clean-after DATETIME] [--all] [--notify] [--commits-are-new true/false] [--dry-run] [--profile]

Named Arguments

--nbhd

Restrict update to a particular neighborhood, e.g. /p/.

Default: “”

--project

Restrict update to a particular project. To specify a subproject, use a slash: project/subproject.

Default: “”

--project-regex

Restrict update to projects for which the shortname matches the provided regex.

Default: “”

--repo-types

Only refresh repos of the given type(s). Defaults to: svn,git,hg. Example: –repo-types=git,hg

Default: [‘svn’, ‘git’, ‘hg’]

--mount-point

Restrict update to repos at the given tool mount point.

Default: “”

--clean

Remove repo-related mongo docs (for project(s) being refreshed only) before doing the refresh.

Default: False

--clean-after

Like –clean but only docs for commits after date (%Y-%m-%dT%H:%M:%S format)

--all

Refresh all commits (not just the ones that are new).

Default: False

--notify

Send email notifications of new commits.

Default: False

--commits-are-new

Specify true/false to override smart default. Controls creating activity entries, stats, sending webhook etc.

--dry-run

Log names of projects that would have their repos refreshed, but do not perform the actual refresh.

Default: False

--profile

Enable the profiler (slow). Will log profiling output to ./refresh.profile

Default: False

reindex_projects.py

Can be run as a background task using task name: allura.scripts.reindex_projects.ReindexProjects

Reindex all project records into Solr (for searching)

usage: paster script development.ini allura/scripts/reindex_projects.py -- [-h] [-n NBHD] [-p PROJECT] [--project-regex PROJECT_REGEX] [--dry-run] [--tasks] [--max-chunk MAX_CHUNK]

Named Arguments

-n, --nbhd

Restrict reindex to a particular neighborhood, e.g. /p/.

Default: “”

-p, --project

Restrict update to a particular project. To specify a subproject, use a slash: project/subproject.

Default: “”

--project-regex

Restrict update to projects for which the shortname matches the provided regex.

Default: “”

--dry-run

Log names of projects that would be reindexed, but do not perform the actual reindex.

Default: False

--tasks

Run each individual index operation as a background task.

Default: False

--max-chunk

Max number of artifacts to index in one Solr update command

Default: 100000

reindex_users.py

Can be run as a background task using task name: allura.scripts.reindex_users.ReindexUsers

Reindex all users into Solr (for searching)

usage: paster script development.ini allura/scripts/reindex_users.py -- [-h] [--dry-run] [--tasks] [--max-chunk MAX_CHUNK]

Named Arguments

--dry-run

Log names of projects that would be reindexed, but do not perform the actual reindex.

Default: False

--tasks

Run each individual index operation as a background task.

Default: False

--max-chunk

Max number of artifacts to index in one Solr update command

Default: 100000

create_sitemap_files.py

Can be run as a background task using task name: allura.scripts.create_sitemap_files.CreateSitemapFiles

Generate Allura sitemap xml files. You will need to configure your webserver to serve the files.

This takes a while to run on a prod-sized data set. There are a couple of things that would make it faster, if we need/want to.

1. Monkeypatch forgetracker.model.ticket.Globals.bin_count to skip the refresh (Solr search) and just return zero for everything, since we don’t need bin counts for the sitemap.

2. Use multiprocessing to distribute the offsets to n subprocesses.

usage: paster script development.ini allura/scripts/create_sitemap_files.py -- [-h] [-o OUTPUT_DIR] [-u URLS_PER_FILE] [--exclude-neighborhood [N ...]] [--exclude-tools [TOOL ...]] [--url-dir URL_DIR]

Named Arguments

-o, --output-dir

Output directory (absolute path).[default: “/tmp/allura_sitemap”]

Default: “/tmp/allura_sitemap”

-u, --urls-per-file

Number of URLs per sitemap file. [default: 10000, max: 50000]

Default: 10000

--exclude-neighborhood, -n, --neighborhood

URL prefix of excluded neighborhood(s) Example: u

--exclude-tools

URL prefix of excluded neighborhood(s)

Default: [‘link’, ‘git’, ‘hg’, ‘svn’]

--url-dir

URL directory in which the files will be served from

Default: “/allura_sitemap”

clear_old_notifications.py

Can be run as a background task using task name: allura.scripts.clear_old_notifications.ClearOldNotifications

Remove old temporary notifications

usage: paster script development.ini allura/scripts/clear_old_notifications.py -- [-h] [--back-days BACK_DAYS]

Named Arguments

--back-days

How many days back to clear from (keeps newer notifications)

Default: 60

publicize-neighborhood.py

Cannot currently be run as a background task.

Make all projects in a neighborhood public.

usage: paster script development.ini ../scripts/publicize-neighborhood.py -- [-h] [--test] [--log LOG_LEVEL] NEIGHBORHOOD

Positional Arguments

NEIGHBORHOOD

Neighborhood name.

Named Arguments

--test

Run in test mode (no updates will be applied).

Default: False

--log

Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL).

Default: “INFO”

scrub-allura-data.py

Cannot currently be run as a background task.

Removes private data from the Allura MongoDB. DO NOT RUN THIS on your main database. This is intended to be used on a copy of your database, to prepare it for sharing with others.

usage: paster script development.ini ../scripts/scrub-allura-data.py -- [-h] [--dry-run] [--log LOG_LEVEL]

Named Arguments

--dry-run

Run in test mode (no updates will be applied).

Default: False

--log

Log level (DEBUG, INFO, WARNING, ERROR, CRITICAL).

Default: “INFO”

Site Notifications

Allura has support for site-wide notifications that appear below the site header. UI for managing them can be found under “Site Notifications” in the left sidebar on the site admin interface.

For example, setting available options to:

Active:      ✓
Impressions: 10
Content:     You can now reimport exported project data.
User Role:   Developer
Page Regex:  (Home|browse_pages)
Page Type:   wiki

will create a notification that will be shown for 10 page views or until the user closes it manually. The notification will be shown only for users which have role ‘Developer’ or higher in one of their projects. And if url of the current page is matching regex (Home|browse_pages) and app tool type is wiki. An “Impressions” value of 0 will show the notification indefinitely (until closed). The notification content can contain HTML. The most recent notification that is active and matches for the visitor will be shown. “User Role”, “Page Regex” and “Page Type” are optional.

Deleting projects

Site administrators can delete projects using web interface. This is running delete_projects.py script under the hood. You can access it choosing “Delete projects” from the left sidebar on the site admin interface.

Be careful, projects and all related data are actually deleted from the database!

Just copy and paste URLs of the project you want to delete into “Projects” field, separated by newlines. You can also use nbhd_prefix/project_shortname or just project_shortname format, e.g.

http://MYSITE/p/test3/wiki/
p/test2
test

will delete projects test3, test2 and test.

NOTE: if you omit neighborhood prefix project will be matched only if project with such short name are unique across all neighborhoods, i.e. if you have project with short name test in p2 neighborhood and project with the same short name in p neighborhood project will not be deleted. In this case you should specify neighborhood explicitly to disambiguate it.

The “Reason” field allows you to specify a reason for deletion, which will be logged to disk.

“Disable all project members” checkbox disables all users belonging to groups “Admin” and “Developer” in these projects. The reason will be also recorded in the users’ audit logs if this option is checked.

After clicking “Delete” you will see a confirmation page. It shows which projects are going to be deleted and which are failed to parse, so you can go back and edit your input.

Using Projects and Tools

We currently don’t have any further documentation for basic operations of managing users, projects, and tools on Allura. However, SourceForge has help docs that cover these functions https://sourceforge.net/p/forge/documentation/Docs%20Home/ Note that this documentation also covers some SourceForge features that are not part of Allura.

Public API Documentation

All url endpoints are prefixed with /rest/ and the path to the project and tool.

For example, in order to access a wiki installed in the ‘test’ project with the mount point ‘docs’ the API endpoint would be /rest/p/test/docs.

Explore Allura’s REST API documentation here.

Client Scripts

Allura includes some client scripts that demonstrate use of the Allura REST API and do not have to be run from an Allura environment. They do require some python packages to be installed, though.

wiki-copy.py

Usage: python scripts/wiki-copy.py [options]

Reads the wiki pages from one Allura wiki instance and uploads them to another
Allura wiki instance.

Options:
  -h, --help            show this help message and exit
  -f FROM_WIKI, --from-wiki=FROM_WIKI
                        URL of wiki API to copy from like
                        http://fromserver.com/rest/p/test/wiki/
  -t TO_WIKI, --to-wiki=TO_WIKI
                        URL of wiki API to copy to like
                        http://toserver.com/rest/p/test/wiki/
  -D, --debug           
  -O OAUTH_VERSION, --oauth=OAUTH_VERSION
                        OAuth version to use for authentication. Defaults to
                        OAuth v1.

new_ticket.py

Illustrates creating a new ticket, using the simple OAuth Bearer token.

Post a new ticket using the API

usage: python scripts/new_ticket.py [-h] [-H HOST] project mount_point

Positional Arguments

project

Project shortname

mount_point

Tracker mount point

Named Arguments

-H, --host

Base domain

Default: “sourceforge.net”