#8406 Installing on "Ubuntu 20.04" not possible

unreleased
closed
nobody
install (2)
General
nobody
2022-02-15
2021-12-24
No

Installing on Ubuntu 20.04 according to the manual giving in the command:

sudo docker-compose run taskd paster setup-app docker-dev.ini

results with:

Creating network "allura-git_default" with the default driver
Building web
Step 1/9 : FROM ubuntu:18.04
---> 5a214d77f5d7
Step 2/9 : RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git-core python3-dev gcc libmagic1 libssl-dev libldap2-dev libsasl2-dev libjpeg8-dev zlib1g-dev python3-venv zip subversion python3-svn curl locales sudo && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 9e4c012987ac
Step 3/9 : RUN curl --silent --location https://deb.nodesource.com/setup_10.x | sudo bash - && DEBIAN_FRONTEND=noninteractive apt-get install --yes --no-install-recommends nodejs
---> Using cache
---> 03e2dc378636
Step 4/9 : RUN locale-gen en_US.UTF-8
---> Using cache
---> 754eb37004ea
Step 5/9 : ENV LANG en_US.UTF-8
---> Using cache
---> 80fe4470c51b
Step 6/9 : ENV USER root
---> Using cache
---> a9a65fbce4c4
Step 7/9 : WORKDIR /allura
---> Using cache
---> f3205a90cce3
Step 8/9 : ENV PYTHONUNBUFFERED 1
---> Using cache
---> 336704d9b3c3
Step 9/9 : CMD gunicorn --paste Allura/docker-dev.ini -b :8088 --reload
---> Using cache
---> 6eb3703bbbf8
Successfully built 6eb3703bbbf8
Successfully tagged allura-git_web:latest
WARNING: Image for service web was built because it did not already exist. To rebuild this image you must use docker-compose build or docker-compose up --build.
Creating allura-git_solr_1 ... done
Creating allura-git_mongo_1 ... done
Creating allura-git_web_1 ... error

ERROR: for allura-git_web_1 Cannot create container for service web: create .: volume name is too short, names should be at least two alphanumeric characters

ERROR: for web Cannot create container for service web: create .: volume name is too short, names should be at least two alphanumeric characters
ERROR: Encountered errors while bringing up the project.

Discussion

  • Dave Brondsema

    Dave Brondsema - 2021-12-27

    The volume names in the docker-compose.yml file are:

          - .:/allura  # Allura source code from local host
          - ${LOCAL_SHARED_DATA_ROOT}:/allura-data  # for virtualenv, scm repos, etc
          - ${LOCAL_SHARED_DATA_ROOT}/root/home:/root  # for persistent home dir, mainly for pudb
    

    So my guess is that the LOCAL_SHARED_DATA_ROOT variable isn't getting set. The top of the CHANGES file explains a bit about it:

    [#8399] In an effort to update the Docker startup steps to make them as
    easy as possible and compatible across as many OS's as possible, the default
    allura-data location has been moved from /allura-data to ./allura-data.
    This will likely break existing Docker deployments. To fix your deployment,
    Either set the LOCAL_SHARED_DATA_ROOT env variable to /allura-data or change
    the value in the local .env file (or move your /allura-data to
    ./allura-data).

    So I guess you can make sure the .env file is there and correct? Or you can set export LOCAL_SHARED_DATA_ROOT=./allura-data (or some other value) in your shell terminal before running the docker-compose commands and see if that helps?

     

    Related

    Tickets: #8399

  • peacecop kalmer:

    kalmer@test:/projektid/allura-git/Allura$ export LOCAL_SHARED_DATA_ROOT=./allura-data
    kalmer@test:/projektid/allura-git/Allura$ sudo docker-compose run taskd paster setup-app docker-dev.ini
    [sudo] password for kalmer: 
    

    WARNING: The LOCAL_SHARED_DATA_ROOT variable is not set. Defaulting to a blank string.
    Starting allura-git_solr_1 ... done
    Starting allura-git_mongo_1 ... done
    Creating allura-git_web_1 ... error

    ERROR: for allura-git_web_1 Cannot create container for service web: create .: volume name is too short, names should be at least two alphanumeric characters

    ERROR: for web Cannot create container for service web: create .: volume name is too short, names should be at least two alphanumeric characters
    ERROR: Encountered errors while bringing up the project.

     
  • Dave Brondsema

    Dave Brondsema - 2021-12-28

    What version of docker-compose do you have? It'd be nice for us to figure out if older versions don't automatically use the .env file or if there's something else going on.

    In the commands you shared above, I think sudo is making the exported environment variable not pass through to the docker-compose command. Here's a few possible ways to make it work: https://askubuntu.com/questions/57915/environment-variables-when-run-with-sudo#57938 The second answer with sudo -E ... seems pretty convenient if it works for you.

     
    • peacecop kalmer:

      docker-compose version 1.26.2, build eefe0d31

      sudo -E sh -c 'docker-compose run taskd paster setup-app docker-dev.ini'
      

      Starting allura-git_mongo_1 ... done
      Creating allura-git_web_1 ... done
      Creating allura-git_outmail_1 ... done
      Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"paster\": executable file not found in $PATH": unknown

       
  • Dave Brondsema

    Dave Brondsema - 2021-12-29

    Thanks for the version number - I think us developers can make some improvements (from https://docs.docker.com/compose/environment-variables/ it seems like the .env file location changed on 1.28 which is why it wasn't found on yours. Maybe we could use inline defaults)

    Anyway back to your latest error. The "paster" executable should be available (inside the docker container) if all the previous commands ran correctly. My guess is that because of the sudo & environment problems earlier, the earlier commands didn't all work right. Could you try starting the instructions over from the beginning? Using your new sudo stuff of course.

    You can also check out these docker instructions if you want to try running without sudo: https://docs.docker.com/engine/install/linux-postinstall/

     
    • peacecop kalmer:

      I added my user to docker's group and after that, only docker-compose build required sudo.

      This time, I reached further. After:

      docker-compose up -d
      

      I got:

      allura_mongo_1 is up-to-date
      allura_solr_1 is up-to-date
      allura_web_1 is up-to-date
      allura_outmail_1 is up-to-date
      Creating allura_inmail_1 ... done
      Creating allura_http_1 ... done
      Creating allura_taskd_1 ... done

      In the manual, I read:

      Visit localhost:8080, or on a Mac or Windows whatever IP address Docker Toolbox is using.

      So I went and entered:

      http://192.168.1.173:8080/
      

      into the address bar of my web browser. That IP address is the one of my server machine running docker. The address was automatically turned into:

      http://localhost:8080/neighborhood
      

      The resulting web page was:

      This site can’t be reachedlocalhost refused to connect.
      Try:

      Checking the connection
      Checking the proxy and the firewall
      ERR_CONNECTION_REFUSED

       
  • Dave Brondsema

    Dave Brondsema - 2021-12-30

    Ok good progress :)

    See note 2 at https://forge-allura.apache.org/docs/getting_started/installation.html#first-run That should help with different IP addresses

     
    • peacecop kalmer:

      I added these two lines suggested in the note 2. Thereafter, I did:

      docker-compose restart web
      

      as suggested and what I did get was:

      Restarting allura-git_web_1 ... error

      ERROR: for allura-git_web_1 b'Cannot restart container c12c96ff2d17ed189dc507ad278202c6c9678d4ac3bc85f7088e20cb8519c7ef: driver failed programming external connectivity on endpoint allura-git_web_1 (3163d2222fca567087b3c9db1758af36d0f3d6aaf54649de0230d83b5b6ad117): Bind for 0.0.0.0:8088 failed: port is already allocated'

      I wanted to see what's listening at 8088 and got this answer:

      tcp6 0 0 :::8088 :::* LISTEN 2275581/docker-prox

       
      • Dave Brondsema

        Dave Brondsema - 2021-12-30

        I haven't seen that error before, I think the 'web' container should be the only thing using that port. Maybe try docker-compose down and then docker-compose up -d to start the containers again? Otherwise there are several other possible fixes at https://stackoverflow.com/questions/46176584/docker-bind-for-0-0-0-04000-failed-port-is-already-allocated

         
        • peacecop kalmer:

          This time, I got allura up:

          CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
          94a4b1fae96d allura_web "paster taskd docker…" 10 seconds ago Up 5 seconds allura-git_taskd_1
          b94e5553b729 allura-git_http "/usr/sbin/apache2 -…" 15 seconds ago Up 8 seconds 0.0.0.0:8080->80/tcp allura-git_http_1
          dbd2fe548629 allura_web "python -u -m smtpd …" 16 seconds ago Up 10 seconds 8826/tcp allura-git_outmail_1
          efc2af7d4421 allura-git_web "/bin/sh -c 'gunicor…" 21 seconds ago Up 15 seconds 0.0.0.0:8088->8088/tcp allura-git_web_1
          3d833a327fcf allura_web "paster smtp_server …" 22 seconds ago Up 15 seconds 0.0.0.0:8825->8825/tcp allura-git_inmail_1
          d31f59227236 solr:6-alpine "docker-entrypoint.s…" 27 seconds ago Up 21 seconds 0.0.0.0:8983->8983/tcp allura-git_solr_1
          edd8ffd136d4 mongo:4.2 "docker-entrypoint.s…" 27 seconds ago Up 22 seconds 0.0.0.0:27017->27017/tcp allura-git_mongo_1

          I logged in as admin1, created a new project and imported it successfully from GitHub. However, I can't clone it. Initially, the address for cloning was localhost:8080... and I changed it in docker-dev.ini to allura.tennis24.ee.... This is my domain. I configured nginx to catch all the web requests to allura.tennis24.ee and forward them to localhost:8088. I'm not sure whether I should also forward something to 8080. Anyway, the address suggestion git clone http://allura.tennis24.ee/git/p/poster/puud.kalmer poster-puud.kalmer doesn't work. I also tried it with 8080 and no success. If I go to the address http://allura.tennis24.ee/p/poster/puud.kalmer/ci/master/tree/, I see the code in Allura. If I try to clone the code, I get:

          Cloning into 'poster-puud.kalmer'...
          fatal: repository 'http://allura.tennis24.ee/git/p/poster/puud.kalmer/' not found

           
          • Dave Brondsema

            Dave Brondsema - 2022-01-04

            That URL looks right to me. I think you want to forward everything to port 8080 though. That port is the docker http server which handles git request as well as forwarding internally to 8088 for regular webpages.

            If that doesn't work still, I'd recommend looking at the docker-compose logs, specifically for the 'http' container (maybe 'web' too) and see if it has any clues.

             
            • peacecop kalmer:

              In order to make it more comfortable for you to advice, I post some information.

              I did docker-compose logs and found these things for web_1 and http_1:

              web_1 | [2021-12-30 19:55:55 +0000] [9] [INFO] Starting gunicorn 20.1.0
              web_1 | [2021-12-30 19:55:55 +0000] [9] [INFO] Listening at: http://0.0.0.0:8088 (9)
              web_1 | [2021-12-30 19:55:55 +0000] [9] [INFO] Using worker: sync
              web_1 | [2021-12-30 19:55:55 +0000] [12] [INFO] Booting worker with pid: 12
              web_1 | 19:55:59,271 ERROR [tg.configuration.app_config] Failed to initialize mako template engine, removing it...
              web_1 | 19:56:15,659 WARNI [allura.lib.app_globals] Image path not set for nav_logo
              web_1 | 19:56:36,555 WARNI [forgeimporters.github] github_importer. not set up in .ini file; cannot use OAuth for GitHub
              web_1 | 19:56:52,614 WARNI [forgeimporters.github] github_importer.
              not set up in .ini file; cannot use OAuth for GitHub
              web_1 | 19:57:06,479 WARNI [forgeimporters.github] github_importer. not set up in .ini file; cannot use OAuth for GitHub
              web_1 | 19:57:24,707 WARNI [forgeimporters.github] github_importer.
              not set up in .ini file; cannot use OAuth for GitHub
              web_1 | 19:58:07,140 ERROR [forgegit.model.git_repo] Error with rev_parse(master^0) at http://allura.tennis24.ee/p/poster/code/ref/master/
              web_1 | Traceback (most recent call last):
              web_1 | File "/allura/ForgeGit/forgegit/model/git_repo.py", line 240, in commit
              web_1 | impl = self._git.rev_parse(str(rev) + '^0')
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 120, in wrapper
              web_1 | return self.run_and_log(func, inst, args, kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 152, in run_and_log
              web_1 | retval = func(*args,
              kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 225, in rev_parse
              web_1 | obj = name_to_object(repo, rev[:start])
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 159, in name_to_object
              web_1 | raise BadName(name)
              web_1 | gitdb.exc.BadName: Ref 'master' did not resolve to an object
              web_1 | 18:11:53,079 ERROR [forgegit.model.git_repo] Error with rev_parse(master^0) at http://allura.tennis24.ee/p/poster/code/ref/master/
              web_1 | Traceback (most recent call last):
              web_1 | File "/allura/ForgeGit/forgegit/model/git_repo.py", line 240, in commit
              web_1 | impl = self._git.rev_parse(str(rev) + '^0')
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 120, in wrapper
              web_1 | return self.run_and_log(func, inst,
              args, kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 152, in run_and_log
              web_1 | retval = func(*args,
              kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 225, in rev_parse
              web_1 | obj = name_to_object(repo, rev[:start])
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 159, in name_to_object
              web_1 | raise BadName(name)
              web_1 | gitdb.exc.BadName: Ref 'master' did not resolve to an object
              web_1 | 20:53:57,126 ERROR [forgegit.model.git_repo] Error with rev_parse(master^0) at http://allura.tennis24.ee/p/poster/code/ref/master/
              web_1 | Traceback (most recent call last):
              web_1 | File "/allura/ForgeGit/forgegit/model/git_repo.py", line 240, in commit
              web_1 | impl = self._git.rev_parse(str(rev) + '^0')
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 120, in wrapper
              web_1 | return self.run_and_log(func, inst, *args, kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/timermiddleware/init.py", line 152, in run_and_log
              web_1 | retval = func(*args,
              kwargs)
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 225, in rev_parse
              web_1 | obj = name_to_object(repo, rev[:start])
              web_1 | File "/allura-data/virtualenv/lib/python3.6/site-packages/git/repo/fun.py", line 159, in name_to_object
              web_1 | raise BadName(name)
              web_1 | gitdb.exc.BadName: Ref 'master' did not resolve to an object

              http_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.25.0.7. Set the 'ServerName' directive globally to suppress this message
              inmail_1 | 19:55:59,301 INFO [allura.command] Loaded tools
              http_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.25.0.7. Set the 'ServerName' directive globally to suppress this message
              http_1 | [Thu Dec 30 19:56:00.737742 2021] [:error] [pid 1:tid 140034370296768] python_init: Python version mismatch, expected '2.7.6', found '2.7.17'.
              http_1 | [Thu Dec 30 19:56:00.739671 2021] [:error] [pid 1:tid 140034370296768] python_init: Python executable found '/usr/bin/python'.
              http_1 | [Thu Dec 30 19:56:00.739688 2021] [:error] [pid 1:tid 140034370296768] python_init: Python path being used '/usr/lib/python2.7:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
              http_1 | [Thu Dec 30 19:56:00.739714 2021] [:notice] [pid 1:tid 140034370296768] mod_python: Creating 8 session mutexes based on 6 max processes and 25 max threads.
              http_1 | [Thu Dec 30 19:56:00.739721 2021] [:notice] [pid 1:tid 140034370296768] mod_python: using mutex_directory /tmp
              http_1 | [Thu Dec 30 19:56:00.761391 2021] [mpm_event:notice] [pid 1:tid 140034370296768] AH00489: Apache/2.4.29 (Ubuntu) mod_python/3.3.1 Python/2.7.17 configured -- resuming normal operations
              http_1 | [Thu Dec 30 19:56:00.761631 2021] [core:notice] [pid 1:tid 140034370296768] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'

              Regarding to configuration, I have these rows regarding to web addresses and ports in docker-dev.ini:

              domain = allura.tennis24.ee
              base_url = http://allura.tennis24.ee

              scm.repos.refresh_base_url = http://web:8080

              scm.host.http.git = http://allura.tennis24.ee/git$path
              scm.host.file.git = /allura-data/scm/git$path

              scm.repos.tarball.url_prefix = http://localhost:8088/

              Regarding to the same things, I have the following configuration rows in production.ini:

              use = config:/allura/Allura/docker-dev.ini#main

              base_url = http://allura.tennis24.ee
              domain = allura.tennis24.ee

              scm.repos.refresh_base_url = http://web:8080

              scm.host.https.git = https://allura.tennis24.ee/git$path

              nginx configuration:

              server {
              server_name allura.tennis24.ee;
              location / {
              proxy_http_version 1.1;
              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection "Upgrade";
              proxy_pass http://localhost:8088;
              }

              }

              If I change 8088 into 8080 in that file then for the address http://allura.tennis24.ee/git/p/poster/puud.kalmer, I get:

              This allura.tennis24.ee page can’t be foundNo webpage was found for the web address: http://allura.tennis24.ee/git/p/poster/puud.kalmer
              HTTP ERROR 404

               
  • Dave Brondsema

    Dave Brondsema - 2022-01-05

    Port 8080 is what you want everything to go through. When you have it set to 8080 and try to access the git repository, what logs show up in the 'http' container log?

    Also when you are trying the git URL, I wouldn't expect it to work in a browser. It provides git data, so it may only work with the git tool. Try git ls-remote http://allura.tennis24.ee/git/p/poster/puud.kalmer that actually works for me currently so you probably have it working right now!

     
    • peacecop kalmer:

      Really, using git, the address worked, I could push and the data got pushed.

      After that, I rebooted the server machine and had to perform the following tasks in order to get allura running again:

      history
      

      2002 cd /projektid/allura-git/Allura
      ...
      2014 export LOCAL_SHARED_DATA_ROOT=./allura-data
      2015 sudo -E sh -c 'docker-compose run taskd paster setup-app docker-dev.ini'
      ...
      2022 docker-compose down
      2023 docker-compose up -d

      The system is up again. However, there's another problem now: the pushed code is gone. I don't see the repo poster anymore. Only the preinstalled four repos for admin1 are left.

       
      • Dave Brondsema

        Dave Brondsema - 2022-01-10

        The ... setup-app ... command clears mongodb and sets up initial data, so you don't want to run that each time. I think you can just run docker-compose up -d after you set LOCAL_SHARED_DATA_ROOT.

        The ./allura-data/ directories should still contain the git repositories that you committed & pushed to, in case you need to recover that. But probably best to remove those directories and re-push any new commits (assuming your local checkout has the new commits still available to push again).

         
  • Dave Brondsema

    Dave Brondsema - 2022-02-15
    • status: open --> closed
    • Milestone: v1.13.0 --> unreleased
     
  • Dave Brondsema

    Dave Brondsema - 2022-02-15

    Some of the issues reported here have been addressed with updates via [#8411]

     

    Related

    Tickets: #8411


Log in to post a comment.