#6203 Bulk edit times out

v1.0.0
closed
nobody
42cc (432)
General
Cory Johns
2015-08-20
2013-05-03
Dave Brondsema
No

When saving a bulk edit of allura tickets, the response times out. The changes do take effect properly, it just takes too long. This is new behavior, since [#2502]

Related

Tickets: #2502
Tickets: #6203

Discussion

  • Dave Brondsema

    Dave Brondsema - 2013-05-03

    It works fine on project with a small number of total tickets, so it is an issue with the total number of tickets in the project -- not the number of tickets being changed.

     

  • Dave Brondsema

    Dave Brondsema - 2013-05-03
    • Size: --> 2
     

  • Dave Brondsema

    Dave Brondsema - 2013-05-03
    • Milestone: forge-may-17 --> forge-may-31
     

  • Dave Brondsema

    Dave Brondsema - 2013-05-17
    • Milestone: forge-may-31 --> forge-jun-06
     

  • Dave Brondsema

    Dave Brondsema - 2013-05-22
    • labels: --> 42cc
    • Size: 2 --> 0
    • Milestone: forge-jun-06 --> forge-backlog
     

  • Dave Brondsema

    Dave Brondsema - 2013-05-22

    Make performance improvements, or if no easy wins, change to a background task.

     

  • Igor Bondarenko - 2013-05-23
    • status: open --> in-progress
     

  • Igor Bondarenko - 2013-05-23

    Created #362: [#6203] Bulk edit times out (3cp)

     

    Related

    Tickets: #6203

  • Igor Bondarenko - 2013-06-07
    • status: in-progress --> code-review
     

  • Igor Bondarenko - 2013-06-07

    Closed #362. je/42cc_6203

     

  • Cory Johns - 2013-06-17
    • QA: Cory Johns
     

  • Cory Johns - 2013-06-17

    The change to background task seems fine, and I'm merging it, but we do need some more info on why it's taking so long. I found this stats record in the logs, showing it take over 4 minutes and make over 185k mongo calls, which is way over the top.

    What research did you do to determine what's causing these updates to taking so long? Can you please post the results of your research here. Can you please see if there's a way to reduce the number of mongo calls made?

    At a quick glance, I expect Globals.filtered_by_subscription could be refactored to a single mongo query instead of more than one per user per ticket by replacing the classmethod subscribed call with an instance method that checks the Mailbox's properties instead of re-querying mongo needlessly. I'm not certain if that's the main source of queries, but it's definitely one that could be improved.

     

  • Cory Johns - 2013-06-17
    • status: code-review --> open
     

  • Igor Bondarenko - 2013-06-18
    • status: open --> in-progress
     

  • Igor Bondarenko - 2013-06-18

    Yes, I think main source of problems is in Globals.filtered_by_subscription. We'll investigate it further.

    Created #377 [#6203] Optimize bulk edit (3cp)

     

    Related

    Tickets: #6203

  • Igor Bondarenko - 2013-06-26
    • status: in-progress --> code-review
     

  • Igor Bondarenko - 2013-06-26

    Closed #377. je/42cc_6203a

    We've refactored filtered_by_subscription to a single query.

    Here's some timings:

    Before:

    0.234148979187
0.1740899086
0.166922807693
0.267369031906

    After:

    0.00868606567383
0.0089738368988
0.00773596763611
0.00699090957642

    Also, you can see profiler logs here: before, after.

    As you can see, there are still a lot of mongo calls on discussion's post, approve and ticket's commit, but I think they are not relevant to performance degradation, because they was present before [#2502]. So, I think, main source of troubles was in filtered_by_subscription (especially, when there are many users subscribed to many different tickets).

     

    Related

    Tickets: #2502

  • Cory Johns - 2013-06-26
    • status: code-review --> closed
    • Milestone: forge-backlog --> forge-jun-28
     

Log in to post a comment.