#3270 Add needed mongodb indexes - NEEDS ENSURE INDEX
There's a lot of slow queries logged by mongo. Ones that scan lots of rows and should have indexes. Here are a few frequent ones, there are probably more. Add the indexes on the models, and make sure they get created in prod.
Wed Nov 16 22:22:15 [conn182000] query project-data.artifact_feed ntoreturn:10 reslen:5503 nscanned:642058 { $query: { ref_id: "forgediscussion/model/forum/ForumThread#33d035fb" }, $orderby: { pubdate: -1 } } nreturned:5 1080ms
Wed Nov 16 21:52:17 [conn182274] query project-data.config ntoreturn:1 reslen:94 nscanned:700927 { options.import_id: "tracker/538807" } nreturned:1 1202ms
Wed Nov 16 22:17:02 [conn182004] query pyforge.email_address reslen:209 nscanned:338173 { $query: { claimed_by_user_id: ObjectId('4e4c72801be1ce413e0002c6') } } nreturned:2 339ms
Wed Nov 16 22:30:32 [conn182013] query pyforge.mailbox reslen:251 nscanned:37888 { $query: { is_flash: true, user_id: null } } nreturned:1 133ms
Wed Nov 16 21:43:31 [conn182090] query pyforge.mailbox reslen:1150 nscanned:341460 { $query: { queue: { $ne: {} }, type: "direct" } } nreturned:3 1120ms
Wed Nov 16 21:43:31 [conn182086] query pyforge.mailbox reslen:36 nscanned:341460 { $query: { type: { $in: [ "digest", "summary" ] }, next_scheduled: { $lt: new Date(1321479809393) } } } nreturned:0 1136ms
forge:tv/3270
To test, code review + make sure email subscriptions still work.
I tested each of the slow queries above by running an explain() in mongo shell, adding an index, then rerunning explain() to confirm index was used and query was faster. Then I moved the index into the appropriate python model.
The 5th query I improved with code instead, since the low selectivity on "type" + $ne operator don't work well with indexes.
There is definitely more work to do with indexes. Monitoring logs and updating indexes is something we should do regularly.
Merged to dev.