#8272 Really big artifact_feed queries
closed
General
nobody
2019-06-17
2019-03-13
No
Some discussions with lots of threads can cause artifact_feed queries that are huge, like this:
2019-03-12T23:38:23.950+0000 I COMMAND [conn6349421] warning: log line attempted (3226kB) over max size (10kB), printing beginning and end ... query project-data.artifact_feed query: { $query: { ref_id: { $in: [ "forgediscussion/model/forum/ForumThread#00841475", "forgediscussion/model/forum/ForumThread#00c62d67", "forgediscussion/model/forum/ForumThread#007b66b8", "forgediscussion/model/forum/ForumThread#00c67d7c", "forgediscussion/model/forum/ForumThread#00c84065", "forgediscussion/model/forum/ForumThread#004372fb", "forgediscussion/.......... /ForumThread#2f8fe61bcb", "forgediscussion/model/forum/ForumThread#e255b209ec", "forgediscussion/model/forum/ForumThread#cf784149a4", "forgediscussion/model/forum/ForumThread#94d83cb848", "forgediscussion/model/forum/ForumThread#eb607a976b", "forgediscussion/model/forum/ForumThread#48a736ec01", "forgediscussion/model/forum/ForumThread#706e08a2b4", "forgediscussion/model/forum/ForumThread#a8b9ad8eaf", "forgediscussion/model/forum/ForumThread#8eaf24bc77", "forgediscussion/model/forum/ForumThread#32d92622f7" ] } }, $orderby: { pubdate: -1 } } planSummary: IXSCAN { ref_id: 1, pubdate: -1 }, IXSCAN { ref_id: 1, pubdate: -1 } cursorid:421910004086 ntoreturn:10 ntoskip:0 keysExamined:52445 docsExamined:8252 hasSortStage:1 keyUpdates:0 writeConflicts:0 numYields:411 nreturned:10 reslen:9110 locks:{ Global: { acquireCount: { r: 824 } }, Database: { acquireCount: { r: 412 } }, Collection: { acquireCount: { r: 412 } } } 1758ms
db/8272
QA: test that URL like /p/test/discussion/general/feed.rss produces the same results as before. Can compare it to overall discussion feed /p/test/discussion/feed.rss too
May need an index on num_replies, flags, last_post_date at some point, to facilitate this query & sorting (sorting without an index has a max size limit). Same query as in ForumController already, and that seems to be not a problem.