#5240 Improve performance of security functions

v1.0.0
closed
nobody
42cc (433)
General
2015-08-20
2012-11-05
No

http://screencast.com/t/GNgzT2OAbXj is a profile of sampled requests from a live server. Note the pairs of arrows which show essentially the same call stack, just starting from a different point. And the first stack has some details expanded which the second stack has collapsed.

Looks like lots of opportunities for improvement here. Ming odm at the root of it seems like the slowest part, perhaps optimize that. Avoiding expensive calls may be even easier. E.g. don't validate the ACL info when reading it. Or cache security results better (but safely)

Related

Tickets: #5240

Discussion

  • Dave Brondsema

    Dave Brondsema - 2012-11-12
    • labels: performance --> performance, 42cc
     
  • Igor Bondarenko

    Igor Bondarenko - 2012-11-13

    Created #213: [#5240] Improve performance of security functions (4cp)

     

    Related

    Tickets: #5240

  • Igor Bondarenko

    Igor Bondarenko - 2012-11-13
    • status: open --> in-progress
     
  • Igor Bondarenko

    Igor Bondarenko - 2012-12-03
    • status: in-progress --> code-review
     
  • Igor Bondarenko

    Igor Bondarenko - 2012-12-03

    Closed #213. Branch 42cc_5240.

    We've rewritten queries inside Credentials using pure pymongo to avoid overhead of Ming ODM validation and object creation (we haven't found another way to skip Ming validation).

    Minor changes were introduced to tests and few methods (without changing the functionality, just to adapt the code that are working with Credentials and RoleCache.

    Also, cache invalidation after project creation has changed a bit. See class ProjectRegistrationProvider._create_project and Credentials.clear_user for details.

    It works faster on dev sandbox (by ~30%). Though, I guess, our benchmarks was pretty inaccurate, so it requires testing in environment with real requests by multiple users.

     
  • Dave Brondsema

    Dave Brondsema - 2012-12-03
    • qa: Tim Van Steenburgh
     
  • Tim Van Steenburgh

    • status: code-review --> validation
    • milestone: forge-backlog --> forge-dec-14
     
  • Tim Van Steenburgh

    Looks good, although it's hard to get a good performance comparison on a sandbox, even with profiling numbers. Let's recheck the production profile once this is pushed to see if we need any more improvement.

     
  • Dave Brondsema

    Dave Brondsema - 2012-12-06

    http://screencast.com/t/uNdWCNduhzC is a new profile in production, after this code was live. Earlier about 29% of time was spent on security predicates, and now it's only 14%. We may be able to get some more out of it, but that seems like a good improvement.

    And here's a graph of page rendering times, with the code change marked http://screencast.com/t/drFN8sSv6Rh

     
  • Dave Brondsema

    Dave Brondsema - 2012-12-06
    • status: validation --> closed
     
  • Dave Brondsema

    Dave Brondsema - 2012-12-14
    • labels: --> 42cc
     

Log in to post a comment.