#5240 Improve performance of security functions
http://screencast.com/t/GNgzT2OAbXj is a profile of sampled requests from a live server. Note the pairs of arrows which show essentially the same call stack, just starting from a different point. And the first stack has some details expanded which the second stack has collapsed.
Looks like lots of opportunities for improvement here. Ming odm at the root of it seems like the slowest part, perhaps optimize that. Avoiding expensive calls may be even easier. E.g. don't validate the ACL info when reading it. Or cache security results better (but safely)
Created #213: [#5240] Improve performance of security functions (4cp)
Related
Tickets:
#5240Closed #213. Branch
42cc_5240.We've rewritten queries inside
Credentialsusing pure pymongo to avoid overhead of Ming ODM validation and object creation (we haven't found another way to skip Ming validation).Minor changes were introduced to tests and few methods (without changing the functionality, just to adapt the code that are working with
CredentialsandRoleCache.Also, cache invalidation after project creation has changed a bit. See
class ProjectRegistrationProvider._create_projectandCredentials.clear_userfor details.It works faster on dev sandbox (by ~30%). Though, I guess, our benchmarks was pretty inaccurate, so it requires testing in environment with real requests by multiple users.
Looks good, although it's hard to get a good performance comparison on a sandbox, even with profiling numbers. Let's recheck the production profile once this is pushed to see if we need any more improvement.
http://screencast.com/t/uNdWCNduhzC is a new profile in production, after this code was live. Earlier about 29% of time was spent on security predicates, and now it's only 14%. We may be able to get some more out of it, but that seems like a good improvement.
And here's a graph of page rendering times, with the code change marked http://screencast.com/t/drFN8sSv6Rh