#5105 Move monq_task collection to separate db NEEDS ENSURE_INDEX, MONGO CMD

v1.0.0
closed
Dave Brondsema
performance (126) sf-1 (616)
General
Cory Johns
2015-08-20
2012-10-10
Dave Brondsema
No

The monq_task collection is quite busy; more write % than any other collection. With Mongo 2.2, moving it to a separate db will give it a separate write lock, which should help overall mongo performance.

Discussion

  • Dave Brondsema

    Dave Brondsema - 2012-10-11
    • status: open --> in-progress
    • assigned_to: Dave Brondsema
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11

    Deployment plan:

    • fix incorrect 'busy' tasks
    • push code changes live
      • ensure_index
      • including graceful restart of taskd procs
    • mongo command to move ready/complete/error task records onto new db
    • after the busy taskd procs finish up their tasks, move those completed task records over into the new db
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11
    • summary: Move monq_task collection to separate db --> Move monq_task collection to separate db NEEDS ENSURE_INDEX, MONGO CMD
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11 
    db.getSiblingDB('pyforge').monq_task.find({state:{$in:['complete','error','ready']}}).forEach(function(t){
    db.monq_task.insert(t);
    err = db.getLastError();
    if (err)
        print(err);
});
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11

    forge:db/5105 Note the development.ini changes and make corresponding changes in your production.ini Then ensure that everything works for new events, and look for potential issues with tasks that are in-process during the deployment.

     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11
    • status: in-progress --> code-review
    • qa: Cory Johns
     

  • Cory Johns - 2012-10-11
    • status: code-review --> validation
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-11
    • status: validation --> closed
     

Log in to post a comment.