#4914 Add graceful restart command to taskd

v1.0.0
closed
General
Cory Johns
2015-08-20
2012-09-12
No

Add a graceful restart to command taskd, so that new code can take effect for new tasks, but existing long-running tasks can keep running.

  1. send the command - a separate paster command that sends a OS signal? need to make sure it goes to the right proc when multiple taskd processes are being used
  2. taskd receives the orders and should continue its current task but then stop when done with it
  3. when done, it should restart itself, e.g. http://www.daniweb.com/software-development/python/code/260268/restart-your-python-program
  4. add appropriate logging at every step, so it is clear what happens

Discussion

  • Dave Brondsema

    Dave Brondsema - 2012-09-14
    • status: open --> in-progress
    • assigned_to: Dave Brondsema
    • milestone: forge-oct-05 --> forge-sep-21
     
  • Dave Brondsema

    Dave Brondsema - 2012-09-14

    allura:db/4914

    For easier testing on our sandboxes:

    • /etc/init.d/mod_wsgi_autoreload stop
    • set NUM_TASKS=1 in /etc/init.d/reactor

    Test:

    • tail allura.log
    • killall -s USR1 paster
    • killall -s USR2 paster
    • killall -s TRAP paster
    • test when idle and with a long-running task

    Here's an example of a task good for testing

    import logging
    import time
    log = logging.getLogger(__name__)
    
    @task
    def count():
        for i in range(1,5):
            log.info('count %s' % i)
            time.sleep(1)
    

    Create a lot of them with:

    allurapaste shell /var/local/config/production.ini
    > from allura.tasks.event_tasks import count
    > count.post()
    > count.post()
    > ...
    

    Then you can change the logging within the event, and do a graceful restart to confirm that the new code is used.

     
  • Dave Brondsema

    Dave Brondsema - 2012-09-14
    • status: in-progress --> code-review
    • qa: Cory Johns
    • size: --> 1
     
  • Cory Johns

    Cory Johns - 2012-09-14
    • status: code-review --> closed
     
  • Dave Brondsema

    Dave Brondsema - 2012-09-14
    • labels: --> stability
     

Log in to post a comment.