Apache Allura™

Forge software for hosting software projects

Brought to you by: alexluberg, brondsem, ccruz, deshani, and 11 others

forge-allura hosting setup

This describes how forge-allura.apache.org itself is hosted. This is only relevant for the Apache Allura developers, and not for the general public.

Infrastructure
Directories & Files
Installed packages
Docker and initial app setup
Cron jobs
Mail routing
Migrating to a new host
Maintenance & General Administration
- If python version is changed

Infrastructure

https://infra.apache.org/vm-management.html

ASF infrastructure team is responsible for:

providing allura-ec2-va host (currently Ubuntu 24.04)
controlling which committers can SSH in
- for sudo setup see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=122916896
ports 22, 25, 80, 443 open only
DNS records:
- A & MX record for forge-allura.apache.org
- MX records for **.forge-allura.apache.org (any subdomain, recursively)
- reverse DNS
routing inbound mail from apache mx server to our host
outgoing mail routing
backups

Directories & Files

/allura-data holds configs, cron scripts, virtualenv and repo data

/var/local/allura has the app and is the docker root directory. Its a git checkout with some customizations (committed on top of normal master). Things like:

.env has LOCAL_SHARED_DATA_ROOT=/allura-data to make that dir at the root of the host OS
copying docker-compose-prod.yml to docker-compose.yml and further:
- use /allura-data/forge-allura.ini for all command: lines
- make sure all right ports are exposed or not exposed
- letsencrypt-nginx-proxy-companion
in scm_config/git-http/git-http.conf
- some bot/spider blocks
- serve url /docs/ from /allura-data/www-docs/ similarly rest-api-docs
- serve /allura-data/www-misc/ files

Code repos in /allura-data/scm/git/p/allura are for browsing, merge-requests etc. (separate from /var/local/allura used to run the site). Initially set up with:

git clone --mirror https://forge-allura.apache.org/git/p/allura/AlluraSite
git clone --mirror https://gitbox.apache.org/repos/asf/allura-site website.git
git clone --mirror https://gitbox.apache.org/repos/asf/allura git.git

And update-repos.sh keeps our mirrors up to date

Installed packages

docker-compose-v2 (and thus docker.io) installed with apt

https://docs.docker.com/engine/install/ubuntu/ has longer install options, but apt install seems to get new enough version.

Docker and initial app setup

Enabled at startup with: systemctl enable docker

/etc/docker/daemon.json is set to

{
  "log-driver": "syslog",
  "live-restore": true
}

Docker was set up based around normal instructions https://forge-allura.apache.org/docs/getting_started/installation.html#using-docker:

scripts/init-docker-dev.sh (might clobber www-misc/robots.txt, can run steps by hand)
docker compose run --rm oneoff bash then:
- cd ..; pip install -r requirements-dev.txt for sphinx for docs building
- pip install ForgePastebin
- pip install git+file:/allura-data/scm/git/p/allura/AlluraSite.git#egg=AlluraSite custom theme
- pip uninstall ForgeSVN
No setup-app cmd

Cron jobs

The root cron file is currently:

MAILTO=""
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SHELL=/bin/bash

# every few days restart of web services due to memory bloat, actually more often since something gets janky after a while
50 */3 * * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart web git-http
# also restart solr, so that things get persisted.  Gotta be a better way
5 4 * * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart solr


# check for new commits in repos
* * * * * /allura-data/update-repos.sh >>/tmp/cron-update-repos.log 2>&1

# TODO: /allura-data/update-allura.sh in cron

# every few days restart docker, seems necessary to help letsencrypt-nginx-proxy-companion connect to docker sometimes?
0 3 * * */5 service docker reload

# every month restart to make sure SSL certs renew ok
0 4 10 * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart letsencrypt-nginx-proxy-companion
5 4 10 * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart nginx-proxy

Mail routing

Inbound email is received by postfix service, we need it routed the inmail docker container. Outbound mail goes from within docker (taskd) to postfix on main host. Edit /etc/postfix/main.cf:

append , forge-allura.apache.org, tickets.allura.p.forge-allura.apache.org, git.allura.p.forge-allura.apache.org, wiki.allura.p.forge-allura.apache.org to the mydestination line. Could do wildcards with https://serverfault.com/questions/133190/host-wildcard-subdomains-using-postfix
add local_recipient_maps = (blank) so that postfix doesn't try to validate 1234@ (ticket num) as a local user mailbox etc.
ensure inet_interfaces = all (so docker can reach out to it)
# 172.16+ private/internal network needed for docker hosts with addr like 172.18.0.9
append 172.16.0.0/12 to mynetworks = line
add transport_maps = hash:/etc/postfix/transport
Create /etc/postfix/transport as:

# send all email to our domain to the python handler
forge-allura.apache.org smtp:127.0.0.1:8825
.forge-allura.apache.org smtp:127.0.0.1:8825
*.forge-allura.apache.org smtp:127.0.0.1:8825

Run postmap /etc/postfix/transport and service postfix reload
Will infra's "puppet" config overwrite those? https://github.com/apache/infrastructure-p6/blob/production/data/nodes/allura-vm.apache.org.yaml has some of the customizations we need
/allura-data/forge-allura.ini has smtp_server set to 172.17.0.1 because that's result of ip addr show docker0 and will let a docker container reach the outer host

Testing outgoing mail:

apt install swaks
swaks --protocol ESMTPS --to myself@wherever.com --from noreply@forge-allura.apache.org --body "test message"

And then also from within docker / the app with --server 172.17.0.1:25 appended

Test incoming mail:

Run telnet localhost 25 and then see [Notes] for send mail, making sure to use TO: with some forge-allura.apache.org addr. Check that inmail and taskd containers got it.

Migrating to a new host

From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup. Here's notes:

install docker per above
setup too?
setup mail routing per above

Copy data from one host (in this example vm2 is the old host). Can't ssh to old host as root (unless you add pub file to /etc/ssh/ssh_keys/ and change PermitRootLogin) but can old access files as regular user.

sudo -E bash  # so ssh agent works

cd /var/local/
git clone https://gitbox.apache.org/repos/asf/allura.git
cd allura
git remote add oldvm ssh://brondsem@allura-vm2.apache.org/var/local/allura
git cherry-pick oldvm/master  # to get custom changes.  Assumes they are rebased into one commit

mkdir /allura-data
cd /allura-data
scp -r brondsem@allura-vm2.apache.org:/allura-data/solr .  # and other files similarly

rm /allura-data/solr/index/write.lock
docker compose -f /var/local/allura/docker-compose.yml run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'

cd /allura-data/scm/git/
ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -

# on old host, as root (maybe skip task.monq_task?):
       docker compose run --volume /allura-data:/allura-data --rm mongo mongodump --host mongo --gzip --archive=/allura-data/mongo/mongoarchive.gz
# then back on new host:
scp brondsem@allura-vm2.apache.org:/allura-data/mongo/mongoarchive.gz .
docker compose up -d mongo
docker compose run --volume /allura-data:/allura-data --rm mongo mongorestore --host mongo -v --gzip --archive=/allura-data/mongoarchive.gz --drop

And Docker and initial app setup section above

When testing URL other than forge-allura.apache.org you'll have to edit VIRTUAL_HOST and LETSENCRYPT_HOST in docker-compose.yml and restart the nginx containers. Remember to set them back when all ready to go live under forge-allura DNS.

The direct hostname (allura-vm) is reachable at port 80 & 443 which isn't ideal (everyone should use forge-allura.apache.org). But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.

Remember to test that local repos (user forks mostly) serve ok. They use ApacheAccessHandler etc.

Maintenance & General Administration

Periodically we should run:

apt upgrade to keep system packages up to date. Reboot as needed.

/allura-data/update-allura.sh to update latest code & python packages. Occasionally manual steps need too. If docker-compose.yml for examples is changed, then local changes to that file will cause a conflict, and will have to be resolved. If Dockerfile is changed you'll have to run docker compose build and stop/start services for it to take effect.

If python version is changed

This is controlled by PY_VERSION in Dockerfile

docker compose stop
docker compose build
mv /allura-data/virtualenv/ /allura-data/virtualenv-old
docker compose run --rm -w /allura oneoff bash -c '$PYTHON_EXE -m venv /allura-data/virtualenv && /allura-data/virtualenv/bin/pip install -U pip wheel'
use "oneoff" commands to install requirements.txt AND extra packages, see Initial Setup section above with its pip install several commands
docker compose up -d

Here's how to run some commands with docker:

cd /var/local/allura
docker compose run --rm oneoff paster ensure_index /allura-data/forge-allura.ini
docker compose run --rm oneoff paster script /allura-data/forge-allura.ini allura/scripts/whatever.py -- --options here
docker compose logs -f --tail 10

Tickets: ~~#8366~~
Wiki: Home
Wiki: Notes