#2530 Add new mongo indexes - NEEDS ENSURE INDEX

v1.0.0
closed
Tim Van Steenburgh
performance (126) sf-1 (616)
General
nobody
2015-08-20
2011-07-28
Tim Van Steenburgh
No

Based on analysis of https://control.siteops.geek.net/sog/trac/ticket/18779

  1. AppConfig needs a compound index on (options.mount_point, project_id)
  2. AppConfig needs an index on project_id
  3. Project needs a compound index on (deleted, shortname, neighborhood_id)

Discussion

  • Tim Van Steenburgh - 2011-07-28
    • summary: Add index to AppConfig --> Add new mongo indexes
    • Description has changed:

    Diff:

    --- old 
+++ new 
@@ -1,8 +1,5 @@
-We need a compound index on (options.mount_point, project_id) as evidenced by this log snippet from ceverest:
+Based on analysis of https://control.siteops.geek.net/sog/trac/ticket/18779

-Wed Jul 27 23:59:42 [conn169403] query project-data.config reslen:230 nscanned:441800 { $query: { options.mount_point: "profile", project_id: ObjectId('4d57a1bfbfc09e144e0a6bbf') } }  nreturned:1 616ms
-Wed Jul 27 23:59:51 [conn169383] query project-data.config reslen:718 nscanned:441800 { $query: { options.mount_point: "blog", project_id: ObjectId('4e2760640594ca21d0000d0c') } }  nreturned:1 542ms
-Wed Jul 27 23:59:52 [conn169356] query project-data.config reslen:1372 nscanned:441800 { $query: { options.mount_point: "svn", project_id: ObjectId('4c2a2a791be1ce3efe00000f') } }  nreturned:1 542ms
-Wed Jul 27 23:59:53 [conn169380] query project-data.config reslen:230 nscanned:441800 { $query: { options.mount_point: "profile", project_id: ObjectId('4d5913f90594ca5071000592') } }  nreturned:1 611ms
-Wed Jul 27 23:59:54 [conn169388] query project-data.config reslen:230 nscanned:441800 { $query: { options.mount_point: "profile", project_id: ObjectId('4d5913f90594ca5071000592') } }  nreturned:1 602ms
-Wed Jul 27 23:59:57 [conn169372] query project-data.config reslen:1669 nscanned:441800 { $query: { options.mount_point: "tickets", project_id: ObjectId('4c3487c61be1ce11da00003e') } }  nreturned:1 540ms
+1. AppConfig needs a compound index on (options.mount_point, project_id)
+2. AppConfig needs an index on project_id
+3. Project needs a compound index on (deleted, shortname, neighborhood_id)
     

  • Tim Van Steenburgh - 2011-07-28
    • status: open --> in-progress
    • assigned_to: Tim Van Steenburgh
    • size: --> 1
    • milestone: backlog --> jul-29
     

  • Tim Van Steenburgh - 2011-07-28

    forge:tv/2530

    Timothy Van Steenburgh: can i get a quick code review on this? https://engr.geek.net/git/?p=forge.git;a=commitdiff;h=fa7a9b825d8794fdf972c087093554fe23254299

    David Brondsema: do you know what query uses 'deleted', 'shortname', 'neighborhood_id' ?

    Timothy Van Steenburgh: 1s

    Timothy Van Steenburgh: dave: the projects macro looks like the most likely suspect

    Timothy Van Steenburgh: although it's not doing an explicit count, so i'm not sure if that jives with the log output: Wed Jul 27 00:55:10 [conn164814] query pyforge.$cmd ntoreturn:1 command: { count: "project", query: { deleted: false, shortname: { $ne: "--init--" }, neighborhood_id: ObjectId('4be2faf8898e33156f00003e') }, fields: null } reslen:64 1069ms

    David Brondsema: tvansteenburgh: looks like there are a few places similar to that

    David Brondsema: Allura/allura/controllers/root.py +82

    David Brondsema: Allura/allura/controllers/project.py +101

    David Brondsema: all listing projects

    David Brondsema: anyway, commit looks great

    • summary: Add new mongo indexes --> Add new mongo indexes - NEEDS ENSURE INDEX
    • status: in-progress --> closed
     

  • Tim Van Steenburgh - 2011-07-28

    Merged to dev.

     

Log in to post a comment.