Apache Allura™

• Size: --> 4

• labels: performance --> performance, 42cc
• status: open --> in-progress
• assigned_to: Igor Bondarenko

• status: in-progress --> code-review

I've generated ~2200 projects with scripts/perf/generate-projects.py to test this.

Closed #658. je/42cc_7644

The bottleneck is call to get project admins in the template, which generates a lot of queries to lookup ProjectRoles. Roles are cached, but for every call to admins() there's actually a cache miss :)

There are two reasons for that:

1. we ask for roles for each project exactly once within every request, and role cache is not preserved between different requests
2. users_with_named_role() (which Project.admins() uses) does not actually use a cache and always generates a new query

I've added cache pre-population before rendering template, which generates only one query to get all roles for all given projects and fixed reason (2) above.

On my test data response time improved from ~12-14sec to ~9-11sec.

That's better, but not as much as I want it to be :)

Removing calls to project.admins() entirely gives response time of ~4-5sec for the same data. So another option could be to initially render page without displaying admins. Instead we can add a button "Show admins" for each project which will make an ajax call to get admins or even just a link to a page, which already contains a list of admins (don't sure such page exists, though).

Any thoughts?

Originally by: moorman

Admins are needed as it allows correlation of username to project name, and
grouping of projects by username.

On Mon, Sep 22, 2014 at 5:23 AM, Igor Bondarenko jetmind@users.sf.net
wrote:

• status: in-progress --> code-review
• Comment:

I've generated ~2200 projects with scripts/perf/generate-projects.py to
test this.

Closed #658. je/42cc_7644

The bottleneck is call to get project admins in the template, which
generates a lot of queries to lookup ProjectRoles. Roles are cached, but
for every call to admins() there's actually a cache miss :)

There are two reasons for that:

1. we ask for roles for each project exactly once within every
   request, and role cache is not preserved between different requests
2. users_with_named_role() (which Project.admins() uses) does not
   actually use a cache and always generates a new query

I've added cache pre-population before rendering template, which generates
only one query to get all roles for all given projects and fixed reason (2)
above.

On my test data response time improved from ~12-14sec to ~9-11sec.

That's better, but not as much as I want it to be :)

Removing calls to project.admins() entirely gives response time of
~4-5sec for the same data. So another option could be to initially render
page without displaying admins. Instead we can add a button "Show admins"
for each project which will make an ajax call to get admins or even just a
link to a page, which already contains a list of admins (don't sure such
page exists, though).

Any thoughts?

• [tickets:#7644] http://sourceforge.net/p/allura/tickets/7644 Make
  /nf/admin/new_projects faster*

Status: code-review
Milestone: forge-sep-19
Labels: performance 42cc
Created: Thu Aug 28, 2014 02:24 PM UTC by Dave Brondsema
Last Updated: Fri Sep 19, 2014 06:04 AM UTC
Owner: Igor Bondarenko

/nf/admin/new_projects can take a very long time when showing hundreds of
projects. I don't have timermiddleware stats, that'll probably have to be
reproduced on a sandbox for detailed analysis of what is slow. I do have
data from New Relic which indicates hundreds of mongo queries are being run
and it looks like they are invoked from within the template. Perhaps we are
doing separate queries for each project and can do a batch query instead?

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/allura/tickets/7644/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

• status: code-review --> closed
• QA: Dave Brondsema

• Milestone: unreleased --> asf_release_1.2.0