Recent changes to forge-allura hosting setup

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Tue, 13 Jan 2026 20:43:57 -0000

--- v16
+++ v17
@@ -14,10 +14,9 @@
 * ports 22, 25, 80, 443 open only
 * DNS records:
     * A & MX record for forge-allura.apache.org
-    * MX records for `**.forge-allura.apache.org` (any subdomain, recursively)
-    * reverse DNS
-* routing inbound mail from apache mx server to our host
-* outgoing mail routing
+    * MX records for `**.forge-allura.apache.org` (any subdomain, recursively)
+* routing inbound mail from apache mx server to our host
+* access to use `mail-relay.apache.org` for outgoing mail
 * backups

 ## Directories & Files
@@ -136,6 +135,7 @@
 From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's notes:

 - install docker per above
+- setup too?
 - setup mail routing per above

 Copy data from one host (in this example vm2 is the _old_ host).  Can't ssh to old host as root (unless you add pub file to `/etc/ssh/ssh_keys/` and change `PermitRootLogin`) but can old access files as regular user.
@@ -155,7 +155,6 @@
 rm /allura-data/solr/index/write.lock
 docker compose -f /var/local/allura/docker-compose.yml run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'

-# git repos.  On previous host, run `git gc` on any big repos.  And remove any really-old unneeded repos
 cd /allura-data/scm/git/
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -
@@ -168,15 +167,11 @@
 docker compose run --volume /allura-data:/allura-data --rm mongo mongorestore --host mongo -v --gzip --archive=/allura-data/mongoarchive.gz --drop
 ```

-And `Docker and initial app setup` section above, but instead of `scripts/init-docker-dev.sh` run the steps manually in a container.
-
-Check that `/allura-data/update-repos.sh` and `/allura-data/update-allura.sh` work (run step by step)
-
-Copy crons (see section above)
+And `Docker and initial app setup` section above

 When testing URL other than `forge-allura.apache.org` you'll have to edit `VIRTUAL_HOST` and `LETSENCRYPT_HOST` in `docker-compose.yml` and restart the nginx containers.  Remember to set them back when all ready to go live under forge-allura DNS.

-The direct hostname is reachable at port 80 & 443 which isn't ideal (everyone should use `forge-allura.apache.org`).  But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.
+The direct hostname (`allura-vm`) is reachable at port 80 & 443 which isn't ideal (everyone should use `forge-allura.apache.org`).  But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.

 Remember to test that local repos (user forks mostly) serve ok.  They use ApacheAccessHandler etc.

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Tue, 04 Nov 2025 21:08:55 -0000

--- v15
+++ v16
@@ -155,6 +155,7 @@
 rm /allura-data/solr/index/write.lock
 docker compose -f /var/local/allura/docker-compose.yml run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'

+# git repos.  On previous host, run `git gc` on any big repos.  And remove any really-old unneeded repos
 cd /allura-data/scm/git/
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Tue, 04 Nov 2025 20:57:34 -0000

--- v14
+++ v15
@@ -136,7 +136,6 @@
 From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's notes:

 - install docker per above
-- setup too?
 - setup mail routing per above

 Copy data from one host (in this example vm2 is the _old_ host).  Can't ssh to old host as root (unless you add pub file to `/etc/ssh/ssh_keys/` and change `PermitRootLogin`) but can old access files as regular user.
@@ -168,11 +167,15 @@
 docker compose run --volume /allura-data:/allura-data --rm mongo mongorestore --host mongo -v --gzip --archive=/allura-data/mongoarchive.gz --drop
 ```

-And `Docker and initial app setup` section above
+And `Docker and initial app setup` section above, but instead of `scripts/init-docker-dev.sh` run the steps manually in a container.
+
+Check that `/allura-data/update-repos.sh` and `/allura-data/update-allura.sh` work (run step by step)
+
+Copy crons (see section above)

 When testing URL other than `forge-allura.apache.org` you'll have to edit `VIRTUAL_HOST` and `LETSENCRYPT_HOST` in `docker-compose.yml` and restart the nginx containers.  Remember to set them back when all ready to go live under forge-allura DNS.

-The direct hostname (`allura-vm`) is reachable at port 80 & 443 which isn't ideal (everyone should use `forge-allura.apache.org`).  But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.
+The direct hostname is reachable at port 80 & 443 which isn't ideal (everyone should use `forge-allura.apache.org`).  But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.

 Remember to test that local repos (user forks mostly) serve ok.  They use ApacheAccessHandler etc.

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Tue, 04 Nov 2025 20:11:37 -0000

--- v13
+++ v14
@@ -133,17 +133,28 @@
 Run `telnet localhost 25` and then see [Notes] for send mail, making sure to use `TO:` with some forge-allura.apache.org addr.  Check that `inmail` and `taskd` containers got it.

 ## Migrating to a new host
-From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's some info specific to migration (in addition to the general setup noted above).
+From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's notes:
+
+- install docker per above
+- setup too?
+- setup mail routing per above

 Copy data from one host (in this example vm2 is the _old_ host).  Can't ssh to old host as root (unless you add pub file to `/etc/ssh/ssh_keys/` and change `PermitRootLogin`) but can old access files as regular user.
 ```
 sudo -E bash  # so ssh agent works

+cd /var/local/
+git clone https://gitbox.apache.org/repos/asf/allura.git
+cd allura
+git remote add oldvm ssh://brondsem@allura-vm2.apache.org/var/local/allura
+git cherry-pick oldvm/master  # to get custom changes.  Assumes they are rebased into one commit
+
+mkdir /allura-data
 cd /allura-data
 scp -r brondsem@allura-vm2.apache.org:/allura-data/solr .  # and other files similarly

 rm /allura-data/solr/index/write.lock
-docker compose run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'
+docker compose -f /var/local/allura/docker-compose.yml run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'

 cd /allura-data/scm/git/
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
@@ -157,6 +168,8 @@
 docker compose run --volume /allura-data:/allura-data --rm mongo mongorestore --host mongo -v --gzip --archive=/allura-data/mongoarchive.gz --drop
 ```

+And `Docker and initial app setup` section above
+
 When testing URL other than `forge-allura.apache.org` you'll have to edit `VIRTUAL_HOST` and `LETSENCRYPT_HOST` in `docker-compose.yml` and restart the nginx containers.  Remember to set them back when all ready to go live under forge-allura DNS.

 The direct hostname (`allura-vm`) is reachable at port 80 & 443 which isn't ideal (everyone should use `forge-allura.apache.org`).  But the SSL cert shouldn't be valid and port 80 errors, so nobody will really use that even if they happen across it.

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Wed, 29 Oct 2025 15:19:24 -0000

--- v12
+++ v13
@@ -8,7 +8,7 @@

 ASF infrastructure team is responsible for:

-* providing `allura-pnap` host (currently Ubuntu 24.04)
+* providing `allura-ec2-va` host (currently Ubuntu 24.04)
 * controlling which committers can SSH in
     * for sudo setup see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=122916896
 * ports 22, 25, 80, 443 open only

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Thu, 04 Sep 2025 17:29:00 -0000

--- v11
+++ v12
@@ -28,11 +28,13 @@

 * `.env` has `LOCAL_SHARED_DATA_ROOT=/allura-data` to make that dir at the root of the host OS
 * copying `docker-compose-prod.yml` to `docker-compose.yml` and further:
-* use `/allura-data/forge-allura.ini` for all `command:` lines
-* make sure all right ports are exposed or not exposed
-* some bot/spider blocks in `scm_config/git-http/git-http.conf`
-* serve url `/docs/` from `/allura-data/www-docs/`
-* serve `/allura-data/www-misc/` files 
+    * use `/allura-data/forge-allura.ini` for all `command:` lines
+    * make sure all right ports are exposed or not exposed
+    * letsencrypt-nginx-proxy-companion 
+* in `scm_config/git-http/git-http.conf`
+    * some bot/spider blocks
+    * serve url `/docs/` from `/allura-data/www-docs/` similarly rest-api-docs
+    * serve `/allura-data/www-misc/` files 

 Code repos in `/allura-data/scm/git/p/allura` are for browsing, merge-requests etc. (separate from `/var/local/allura` used to run the site). Initially set up with:
 ```
@@ -122,13 +124,13 @@
 Testing outgoing mail:
 ```
 apt install swaks
-swaks --protocol ESMTPS --to myself@wherever.com --from noreply@forge-allura.apache.org --body "test message2" --server 172.17.0.1:25
+swaks --protocol ESMTPS --to myself@wherever.com --from noreply@forge-allura.apache.org --body "test message"
 ```
-And then also from within docker / the app.
+And then also from within docker / the app with `--server 172.17.0.1:25` appended

 Test incoming mail:

-Run `telnet localhost 25` and then see [Notes] for send mail.  Check that `outmail` and `taskd` containers got it.
+Run `telnet localhost 25` and then see [Notes] for send mail, making sure to use `TO:` with some forge-allura.apache.org addr.  Check that `inmail` and `taskd` containers got it.

 ## Migrating to a new host
 From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's some info specific to migration (in addition to the general setup noted above).

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Thu, 04 Sep 2025 15:47:26 -0000

--- v10
+++ v11
@@ -140,6 +140,9 @@
 cd /allura-data
 scp -r brondsem@allura-vm2.apache.org:/allura-data/solr .  # and other files similarly

+rm /allura-data/solr/index/write.lock
+docker compose run --rm -u root solr bash -c 'chown -R solr:solr /opt/solr/server/solr/allura/data/'
+
 cd /allura-data/scm/git/
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Thu, 04 Sep 2025 15:38:29 -0000

--- v9
+++ v10
@@ -26,10 +26,10 @@

 `/var/local/allura` has the app and is the docker root directory.  Its a git checkout with some **customizations (committed on top of normal `master`)**.  Things like:

+* `.env` has `LOCAL_SHARED_DATA_ROOT=/allura-data` to make that dir at the root of the host OS
 * copying `docker-compose-prod.yml` to `docker-compose.yml` and further:
 * use `/allura-data/forge-allura.ini` for all `command:` lines
 * make sure all right ports are exposed or not exposed
-* `allura_web` image names instead of `allura-web` until newer Docker is used
 * some bot/spider blocks in `scm_config/git-http/git-http.conf`
 * serve url `/docs/` from `/allura-data/www-docs/`
 * serve `/allura-data/www-misc/` files 
@@ -44,7 +44,7 @@

 ## Installed packages

-`docker-compose-v2` (and thus `docker.io`) and `mongodb-clients` installed with `apt`
+`docker-compose-v2` (and thus `docker.io`) installed with `apt`

 https://docs.docker.com/engine/install/ubuntu/ has longer install options, but `apt install` seems to get new enough version.

@@ -64,7 +64,7 @@

 * `scripts/init-docker-dev.sh` (might clobber www-misc/robots.txt, can run steps by hand)
 * `docker compose run --rm oneoff bash` then:
-    * `pip install -r requirements-dev.txt` for sphinx for docs building
+    * `cd ..; pip install -r requirements-dev.txt` for sphinx for docs building
     * `pip install ForgePastebin`
     * `pip install git+file:/allura-data/scm/git/p/allura/AlluraSite.git#egg=AlluraSite` custom theme
     * `pip uninstall ForgeSVN`
@@ -78,8 +78,11 @@
 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
 SHELL=/bin/bash

-# every few days restart of web services due to memory bloat
-50 0 * * */2 /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart web git-http
+# every few days restart of web services due to memory bloat, actually more often since something gets janky after a while
+50 */3 * * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart web git-http
+# also restart solr, so that things get persisted.  Gotta be a better way
+5 4 * * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart solr
+

 # check for new commits in repos
 * * * * * /allura-data/update-repos.sh >>/tmp/cron-update-repos.log 2>&1
@@ -88,6 +91,10 @@

 # every few days restart docker, seems necessary to help letsencrypt-nginx-proxy-companion connect to docker sometimes?
 0 3 * * */5 service docker reload
+
+# every month restart to make sure SSL certs renew ok
+0 4 10 * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart letsencrypt-nginx-proxy-companion
+5 4 10 * * /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart nginx-proxy
 ```


@@ -129,14 +136,20 @@
 Copy data from one host (in this example vm2 is the _old_ host).  Can't ssh to old host as root (unless you add pub file to `/etc/ssh/ssh_keys/` and change `PermitRootLogin`) but can old access files as regular user.
 ```
 sudo -E bash  # so ssh agent works
+
 cd /allura-data
-scp -r allura-vm2.apache.org:/allura-data/solr .  # and other files similarly
+scp -r brondsem@allura-vm2.apache.org:/allura-data/solr .  # and other files similarly
+
 cd /allura-data/scm/git/
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
 ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -
-ssh brondsem@allura-vm2.apache.org 'cd /var/local/allura && docker compose run --rm mongo mongodump --host mongo --gzip --archive' > /allura-data/mongoarchive.gz
+
+# on old host, as root (maybe skip task.monq_task?):
+       docker compose run --volume /allura-data:/allura-data --rm mongo mongodump --host mongo --gzip --archive=/allura-data/mongo/mongoarchive.gz
+# then back on new host:
+scp brondsem@allura-vm2.apache.org:/allura-data/mongo/mongoarchive.gz .
 docker compose up -d mongo
-mongorestore -v --gzip --archive=/allura-data/mongoarchive.gz --drop
+docker compose run --volume /allura-data:/allura-data --rm mongo mongorestore --host mongo -v --gzip --archive=/allura-data/mongoarchive.gz --drop
 ```

 When testing URL other than `forge-allura.apache.org` you'll have to edit `VIRTUAL_HOST` and `LETSENCRYPT_HOST` in `docker-compose.yml` and restart the nginx containers.  Remember to set them back when all ready to go live under forge-allura DNS.

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Wed, 03 Sep 2025 20:20:27 -0000

--- v8
+++ v9
@@ -8,14 +8,15 @@

 ASF infrastructure team is responsible for:

-* providing `allura-vm` host (currently Ubuntu 18.04)
+* providing `allura-pnap` host (currently Ubuntu 24.04)
 * controlling which committers can SSH in
     * for sudo setup see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=122916896
+* ports 22, 25, 80, 443 open only
 * DNS records:
     * A & MX record for forge-allura.apache.org
     * MX records for `**.forge-allura.apache.org` (any subdomain, recursively)
     * reverse DNS
-* routing inboud mail from apache mx server to allura-vm
+* routing inbound mail from apache mx server to our host
 * outgoing mail routing
 * backups

@@ -43,7 +44,7 @@

 ## Installed packages

-`docker-compose` (and thus `docker.io`) and `mongodb-clients` installed with `apt`
+`docker-compose-v2` (and thus `docker.io`) and `mongodb-clients` installed with `apt`

 https://docs.docker.com/engine/install/ubuntu/ has longer install options, but `apt install` seems to get new enough version.

@@ -62,12 +63,12 @@
 Docker was set up based around normal instructions https://forge-allura.apache.org/docs/getting_started/installation.html#using-docker:

 * `scripts/init-docker-dev.sh` (might clobber www-misc/robots.txt, can run steps by hand)
-* `docker-compose run --rm oneoff bash` then:
+* `docker compose run --rm oneoff bash` then:
     * `pip install -r requirements-dev.txt` for sphinx for docs building
     * `pip install ForgePastebin`
     * `pip install git+file:/allura-data/scm/git/p/allura/AlluraSite.git#egg=AlluraSite` custom theme
     * `pip uninstall ForgeSVN`
-    * No `setup-app` cmd
+* No `setup-app` cmd

 ## Cron jobs

@@ -78,7 +79,7 @@
 SHELL=/bin/bash

 # every few days restart of web services due to memory bloat
-50 0 * * */2 /usr/bin/docker-compose -f /var/local/allura/docker-compose.yml restart web git-http
+50 0 * * */2 /usr/bin/docker compose -f /var/local/allura/docker-compose.yml restart web git-http

 # check for new commits in repos
 * * * * * /allura-data/update-repos.sh >>/tmp/cron-update-repos.log 2>&1
@@ -125,12 +126,16 @@
 ## Migrating to a new host
 From time to time, ASF infra will work with us to upgrade to newer hardware and/or newer OS setup.  Here's some info specific to migration (in addition to the general setup noted above).

-Copy data from one host (in this example vm2 is the _old_ host).  Probably need to tweak ssh config on old host so you can ssh into it as root (e.g. add pub file to `/etc/ssh/ssh_keys/`)
+Copy data from one host (in this example vm2 is the _old_ host).  Can't ssh to old host as root (unless you add pub file to `/etc/ssh/ssh_keys/` and change `PermitRootLogin`) but can old access files as regular user.
 ```
-scp -r allura-vm2.apache.org:/allura-data/solr .
-ssh allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -
-ssh allura-vm2.apache.org 'cd /var/local/allura && docker-compose run --rm mongo mongodump --host mongo --gzip --archive' > /allura-data/mongoarchive.gz
-docker-compose up -d mongo
+sudo -E bash  # so ssh agent works
+cd /allura-data
+scp -r allura-vm2.apache.org:/allura-data/solr .  # and other files similarly
+cd /allura-data/scm/git/
+ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - p' | tar zxf -
+ssh brondsem@allura-vm2.apache.org 'cd /allura-data/scm/git/ && tar zcf - u' | tar zxf -
+ssh brondsem@allura-vm2.apache.org 'cd /var/local/allura && docker compose run --rm mongo mongodump --host mongo --gzip --archive' > /allura-data/mongoarchive.gz
+docker compose up -d mongo
 mongorestore -v --gzip --archive=/allura-data/mongoarchive.gz --drop
 ```

@@ -146,25 +151,25 @@

 `apt upgrade` to keep system packages up to date.  Reboot as needed.

-`/allura-data/update-allura.sh` to update latest code & python packages.  Occasionally manual steps need too.  If `docker-compose.yml` for examples is changed, then local changes to that file will cause a conflict, and will have to be resolved.  If `Dockerfile` is changed you'll have to run `docker-compose build` and stop/start services for it to take effect.
+`/allura-data/update-allura.sh` to update latest code & python packages.  Occasionally manual steps need too.  If `docker-compose.yml` for examples is changed, then local changes to that file will cause a conflict, and will have to be resolved.  If `Dockerfile` is changed you'll have to run `docker compose build` and stop/start services for it to take effect.

 ### If python version is changed

 This is controlled by `PY_VERSION` in `Dockerfile`

-- `docker-compose stop`
-- `docker-compose build`
+- `docker compose stop`
+- `docker compose build`
 - `mv /allura-data/virtualenv/ /allura-data/virtualenv-old`
-- `docker-compose run --rm -w /allura oneoff bash -c '$PYTHON_EXE -m venv /allura-data/virtualenv && /allura-data/virtualenv/bin/pip install -U pip wheel'`
+- `docker compose run --rm -w /allura oneoff bash -c '$PYTHON_EXE -m venv /allura-data/virtualenv && /allura-data/virtualenv/bin/pip install -U pip wheel'`
 - use "oneoff" commands to install requirements.txt AND extra packages, see Initial Setup section above with its `pip install` several commands
-- `docker-compose up -d`
+- `docker compose up -d`


 Here's how to run some commands with docker:
 ```
 cd /var/local/allura
-docker-compose run --rm oneoff paster ensure_index /allura-data/forge-allura.ini
-docker-compose run --rm oneoff paster script /allura-data/forge-allura.ini allura/scripts/whatever.py -- --options here
-docker-compose logs -f --tail 10
+docker compose run --rm oneoff paster ensure_index /allura-data/forge-allura.ini
+docker compose run --rm oneoff paster script /allura-data/forge-allura.ini allura/scripts/whatever.py -- --options here
+docker compose logs -f --tail 10
 ```

forge-allura hosting setup modified by Dave Brondsema

Dave Brondsema — Mon, 30 Sep 2024 20:07:42 -0000

--- v7
+++ v8
@@ -4,11 +4,13 @@

 ## Infrastructure

+https://infra.apache.org/vm-management.html
+
 ASF infrastructure team is responsible for:

 * providing `allura-vm` host (currently Ubuntu 18.04)
 * controlling which committers can SSH in
-    * for sudo setup see https://cwiki.apache.org/confluence/display/INFRA/OPIE
+    * for sudo setup see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=122916896
 * DNS records:
     * A & MX record for forge-allura.apache.org
     * MX records for `**.forge-allura.apache.org` (any subdomain, recursively)