Admins are needed as it allows correlation of username to project name, and grouping of projects by username.

On Mon, Sep 22, 2014 at 5:23 AM, Igor Bondarenko <jetmind@users.sf.net> wrote:
  • status: in-progress --> code-review
  • Comment:

I've generated ~2200 projects with scripts/perf/generate-projects.py to test this.

Closed #658. je/42cc_7644

The bottleneck is call to get project admins in the template, which generates a lot of queries to lookup ProjectRoles. Roles are cached, but for every call to admins() there's actually a cache miss :)

There are two reasons for that:

  1. we ask for roles for each project exactly once within every request, and role cache is not preserved between different requests
  2. users_with_named_role() (which Project.admins() uses) does not actually use a cache and always generates a new query

I've added cache pre-population before rendering template, which generates only one query to get all roles for all given projects and fixed reason (2) above.

On my test data response time improved from ~12-14sec to ~9-11sec.

That's better, but not as much as I want it to be :)

Removing calls to project.admins() entirely gives response time of ~4-5sec for the same data. So another option could be to initially render page without displaying admins. Instead we can add a button "Show admins" for each project which will make an ajax call to get admins or even just a link to a page, which already contains a list of admins (don't sure such page exists, though).

Any thoughts?


[tickets:#7644] Make /nf/admin/new_projects faster

Status: code-review
Milestone: forge-sep-19
Labels: performance 42cc
Created: Thu Aug 28, 2014 02:24 PM UTC by Dave Brondsema
Last Updated: Fri Sep 19, 2014 06:04 AM UTC
Owner: Igor Bondarenko

/nf/admin/new_projects can take a very long time when showing hundreds of projects. I don't have timermiddleware stats, that'll probably have to be reproduced on a sandbox for detailed analysis of what is slow. I do have data from New Relic which indicates hundreds of mongo queries are being run and it looks like they are invoked from within the template. Perhaps we are doing separate queries for each project and can do a batch query instead?


Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/allura/tickets/7644/

To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/