Installation

Step-by-Step Installation

For a simpler setup using Docker images, see Using Docker instead.

In these instructions, we’ll use VirtualBox and Ubuntu 16.04 (15.04 works too) to create a disposable sandbox for Allura development/testing. Allura should work on other Linux systems (including OSX), but setting up all the dependencies will be different.

  • Download and install VirtualBox for your platform.
  • Download a minimal Ubuntu 16.04 64-bit ISO.
  • Create a new virtual machine in Virtual Box, selecting Ubuntu (64 bit) as the OS type. The rest of the wizards’ defaults are fine.
  • When you launch the virtual machine for the first time, you will be prompted to attach your installation media. Browse to the mini.iso that you downloaded earlier.
  • After a text-only installation, you may end up with a blank screen and blinking cursor. Press Alt-F1 to switch to the first console.
  • Consult available documentation for help installing Ubuntu.

System Packages

Before we begin, you’ll need to install some system packages.

~$ sudo apt-get install git-core python2.7-dev libssl-dev libldap2-dev libsasl2-dev libjpeg8-dev zlib1g-dev libffi-dev

To install MongoDB, follow the instructions here.

Optional, for SVN support:

~$ sudo apt-get install subversion python-svn

Setting up a virtual python environment

The first step to installing the Allura platform is installing a virtual environment via virtualenv. This helps keep our distribution python installation clean.

~$ sudo apt-get install python-pip
~$ sudo pip install virtualenv

Once you have virtualenv installed, you need to create a virtual environment. We’ll call our Allura environment ‘env-allura’.

~$ virtualenv env-allura

This gives us a nice, clean environment into which we can install all the allura dependencies. In order to use the virtual environment, you’ll need to activate it:

~$ . env-allura/bin/activate

You’ll need to do this whenever you’re working on the Allura codebase so you may want to consider adding it to your ~/.bashrc file.

Creating the log directory

(env-allura)~$ sudo mkdir -p /var/log/allura
(env-allura)~$ sudo chown $(whoami) /var/log/allura

Installing the Allura code and dependencies

Now we can get down to actually getting the Allura code and dependencies downloaded and ready to go. If you don’t have the source code yet, run:

(env-allura)~$ mkdir src
(env-allura)~$ cd src
(env-allura)~/src$ git clone https://git-wip-us.apache.org/repos/asf/allura.git allura

If you already reading this file from an Allura release or checkout, you’re ready to continue.

Although the application setup.py files define a number of dependencies, the requirements.txt files are currently the authoritative source, so we’ll use those with pip to make sure the correct versions are installed.

(env-allura)~/src$ cd allura
(env-allura)~/src/allura$ pip install -r requirements.txt

This will take a while. If you get an error from pip, it is typically a temporary download error. Just run the command again and it will quickly pass through the packages it already downloaded and then continue.

Optional, for SVN support: symlink the system pysvn package into our virtual environment

(env-allura)~/src/allura$ ln -s /usr/lib/python2.7/dist-packages/pysvn ~/env-allura/lib/python2.7/site-packages/

Next, run this to set up all the Allura tools:

(env-allura)~/src/allura$ ./rebuild-all.bash

Note

If you only want to use a few tools, run this instead:

(env-allura)~/src/allura$ cd Allura
(env-allura)~/src/allura/Allura$ python setup.py develop
(env-allura)~/src/allura/Allura$ cd ../ForgeWiki   # required tool
(env-allura)~/src/allura/ForgeWiki$ python setup.py develop
# repeat for any other tools you want to use

Initializing the environment

The Allura forge consists of several components, all of which need to be running to have full functionality.

SOLR search and indexing server

We have a custom config ready for use.

(env-allura)~$ cd tmp
(env-allura)/tmp$ sudo apt-get install default-jre-headless unzip
(env-allura)/tmp$ wget -nv http://archive.apache.org/dist/lucene/solr/5.3.1/solr-5.3.1.tgz
(env-allura)/tmp$ tar xvf solr-5.3.1.tgz solr-5.3.1/bin/install_solr_service.sh --strip-components=2
(env-allura)/tmp$ sudo ./install_solr_service.sh solr-5.3.1.tgz

(env-allura)/tmp$ cd ~/src/allura
(env-allura)~/src/allura$ sudo -H -u solr bash -c 'cp -R solr_config/allura/ /var/solr/data/'
(env-allura)~/src/allura$ sudo service solr start

Create code repo directories

The default configuration stores repos in /srv, so we need to create those directories:

~$ sudo mkdir /srv/{git,svn,hg}
~$ sudo chown $USER /srv/{git,svn,hg}
~$ sudo chmod 775 /srv/{git,svn,hg}

If you don’t have sudo permission or just want to store them somewhere else, change the /srv paths in development.ini

If you want to set up remote access to the repositories, see SCM Host Setup

Allura task processing

Allura uses a background task service called “taskd” to do async tasks like sending emails, and indexing data into solr, etc. Let’s get it running

(env-allura)~$ cd ~/src/allura/Allura
(env-allura)~/src/allura/Allura$ nohup paster taskd development.ini > /var/log/allura/taskd.log 2>&1 &

A few more steps, if using git

If you’re using a released version of Allura, these are already done for you. This transpiles JS into a version all browsers support. For non-Ubuntu installations see https://nodejs.org/en/download/package-manager/ for other options to replace the first line here:

(env-allura)~$ curl --silent --location https://deb.nodesource.com/setup_4.x | sudo bash -
(env-allura)~$ sudo apt-get install nodejs
(env-allura)~$ cd ~/src/allura
(env-allura)~$ npm install
(env-allura)~$ npm run build

The application server

In order to initialize the Allura database, you’ll need to run the following:

For development setup:

(env-allura)~/src/allura/Allura$ paster setup-app development.ini

For production setup:

(env-allura)~/src/allura/Allura$ ALLURA_TEST_DATA=False paster setup-app development.ini

This shouldn’t take too long, but it will start the taskd server doing tons of stuff in the background. Once this is done, you can start the application server:

(env-allura)~/src/allura/Allura$ gunicorn --reload --paste development.ini  # add --daemon to run in the background

Next Steps

Go to the Allura webapp running on your local machine port 8080. (If you’re running this inside a VM, you’ll probably have to configure the port forwarding settings)

Using Docker

First run

Download the latest release of Allura, or clone from git for the bleeding edge.

Install Docker and Docker Compose. On Linux, you may need to create a docker group. On Mac, make sure you’re in a directory that Virtual Box shares through to the VM (by default, anywhere in your home directory works).

Run the following commands in your allura directory:

Build/fetch all required images:

export COMPOSE_PROJECT_NAME=allura
docker-compose build

Python and JS package setup (and first containers started):

docker-compose run web scripts/init-docker-dev.sh

Restart SOLR container, so it will see changes from the command above and create index:

docker-compose restart solr

Initialize database with test data:

docker-compose run taskd paster setup-app docker-dev.ini

Note

If you want to skip test data creation you can instead run: docker-compose run -e ALLURA_TEST_DATA=False taskd paster setup-app docker-dev.ini

Start containers in the background:

docker-compose up -d

You’re up and running! Visit localhost:8080, or on a Mac or Windows whatever IP address Docker Toolbox is using. Then see our Post-setup instructions and read more below about the Docker environment for Allura.

Containers

Allura runs on the following docker containers:

  • web
  • mongo
  • taskd
  • solr
  • inmail
  • outmail

Host-mounted volumes

These are created on first run.

Current directory mounted as /allura inside containers. This means your current source code in your host environment is shared with the containers. You can edit Allura code directly, and the containers will reflect your changes.

Python environment:

  • /allura-data/env-docker/python
  • /allura-data/env-docker/bin

Services data:

  • /allura-data/mongo - mongo data
  • /allura-data/solr - SOLR index
  • /allura-data/scm/{git,hg,svn} - code repositories
  • /allura-data/scm/snapshots - generated code snapshots

Ports, exposed to host system

Useful commands

Restarting all containers:

docker-compose up -d

View logs from all services:

docker-compose logs -f

You can specify one or more services to view logs only from them, e.g. to see outgoing mail:

docker-compose logs -f outmail

Update requirements and reinstall apps:

docker-compose run web pip install -r requirements.txt
docker-compose run web ./rebuild-all.bash

You may want to restart at least “taskd” container after that in order for it to pick up changes. Run docker-compose restart taskd

Running all tests:

docker-compose run web ./run_tests

Running subset of tests:

docker-compose run web bash -c 'cd ForgeGit && nosetests forgegit.tests.functional.test_controllers:TestFork'

Connecting to mongo using a container:

docker-compose run mongo mongo --host mongo

Post-setup instructions

Logging in and sample projects

You can log in with username admin1, test-user or root. They all have password “foo”. (For more details on the default data, see bootstrap.py)

There are a few default projects (like “test”) and neighborhoods. Feel free to experiment with them. If you want to register a new project in your own forge, visit /p/add_project.

Create project for site-admin

First of all you need to create a project, which will serve as a container for keeping site administrators (users who will have access to the admin interface).

In order to do that:

  • open main page of the site in your browser
  • go to “Projects” neighborhood (What are neighborhoods?)
  • click “Register a new project” link

By default all admins of “allura” project in “Projects” neighborhood are treated as site admins. If you want to use different project for that, change site_admins_project in development.ini.

Change default configuration

The development.ini file is geared towards development, so you will want to review carefully and make changes for production use.

Change [handler_console] section, so that logs go to a file and will include background tasks info.

class = allura.lib.utils.CustomWatchedFileHandler
args = ('/path/to/allura.log', 'a')

Add write permissions to the /path/to/allura.log for the user you use to run allura proccess.

Change “secrets”.

beaker.session.secret = <your-secret-key>
beaker.session.validate_key = <yet-another-secret-key>

The first one is used for simple cookies, the latter is used for encrypted cookies.

You can use the following command to generate a good key:

~$ python -c 'import os; l = 20; print "%.2x" * l % tuple(map(ord, os.urandom(l)))'

Production-quality web server

If you are running on a public facing server, you should check out some of the additional gunicorn configuration options at http://gunicorn.org/. For example, you’ll want multiple worker processes to handle simultaneous requests, proxy behind nginx for added protection, etc.

If you’d like to use another webserver, here are a few options:

uWSGI

~$ pip install uwsgi  # or install via system packages
~$ uwsgi --ini-paste-logged development.ini --virtualenv /PATH/TO/VIRTUALENV --http 0.0.0.0:8080

mod_wsgi-express

~$ pip install mod_wsgi  # requires httpd2 devel libraries installed in the system
~$ mod_wsgi-express start-server development.ini --application-type paste --user allura --group allura --port 8080  --python-path /PATH/TO/VIRTUALENV/lib/python2.7/site-packages/

For any other wsgi server (e.g. mod_wsgi with Apache, or waitress) you will need a wsgi callable set up like this:

from paste.deploy import loadapp
from paste.script.util.logging_config import fileConfig

config_file = '/PATH/TO/Allura/development.ini'
fileConfig(config_file)
application = loadapp('config:%s' % config_file)

Configuring Optional Features

The development.ini file has many options you can explore and configure.

To run SVN and Git services, see the SCM Host Setup page.

Some features may be added as separate Allura extensions

Enabling inbound email

Allura can listen for email messages and update tools and artifacts. For example, every ticket has an email address, and emails sent to that address will be added as comments on the ticket. To set up the SMTP listener, run:

nohup paster smtp_server development.ini > /var/log/allura/smtp.log &

By default this uses port 8825. Depending on your mail routing, you may need to change that port number. And if the port is in use, this command will fail. You can check the log file for any errors. To change the port number, edit development.ini and change forgemail.port to the appropriate port number for your environment.

SMTP in development

The following command can be used for quick and easy monitoring of smtp during development. Just be sure the port matches the smtp_port from your development.ini (8826 by default).

python -u -m smtpd -n -c DebuggingServer localhost:8826

This will create a new debugging server that discards messages and prints them to stdout.

Using LDAP

Allura has a pluggable authentication system, and can use an existing LDAP system. In your config file (e.g. development.ini), there are several “ldap” settings to set:

  • Change auth.method to: auth.method = ldap
  • Set all the auth.ldap.* settings to match your LDAP server configuration. (auth.ldap.schroot_name won’t be used, don’t worry about it.)
  • Keep auth.ldap.autoregister = true This means Allura will use existing users from your LDAP server.
  • Set auth.allow_user_registration = false since your users already are present in LDAP.
  • Change user_prefs_storage.method to user_prefs_storage.method = ldap
  • Change user_prefs_storage.ldap.fields.display_name if needed (e.g. if display names are stored in a different LDAP attribute).

Restart Allura and you should be all set. Now users can log in with their LDAP credentials and their Allura records will be automatically created the first time they log in.

Note: if you want users to register new accounts into your LDAP system via Allura, you should turn off autoregister and turn on allow_user_registration