#8019 Too many cached_branches/tags

unreleased
open
nobody
General
nobody
2016-02-24
2015-11-09
No

Some repos can get a large number of cached branches or tags. The large lists cause a ming performance penalty: e.g. one request timing had 296ms in mongo, 2304ms in ming.

This is particular true for the sidebar when it hits repo.forks property (twice), which loads many repo objects into memory. We could look into ways to not load those fields when not needed, or skip ming validation on them (see validate_ranges option of ming Array). Or just querying for a count and not getting the objects at all. Or cap the size of the lists.

From forge-allura's own mongo, look at the size of these cached branches lists:

> db['git-repository'].find({}).forEach(function(u) { print(u.cached_branches && u.cached_branches.length, u.url_path) });
undefined /p/test/
undefined /u/masterbunnyfu/
undefined /u/shujisado/
undefined /p/allura/
0 /u/swelltt/
undefined /u/mtem/
undefined /u/barki2015/
0 /p/allura/
0 /u/dnayak/
24 /u/brondsem/
776 /p/allura/
547 /u/saintaardvark/
725 /u/lxj/
550 /u/testmr/
550 /u/vansteenburgh/
583 /u/alexluberg/
543 /u/masterbunnyfu/
643 /u/jetmind/
649 /u/heiths/
577 /u/shujisado/
641 /u/mishravikas/
695 /u/yesidid180/
758 /u/jredd/
776 /u/pranav/

Discussion

  • Pranav Sharma

    Pranav Sharma - 2016-02-23

    @Dave, How to proceed with this one?

     
  • Dave Brondsema

    Dave Brondsema - 2016-02-24

    I think the biggest benefit will be making the sidebar fast (not necessarily making all fork-related lists faster). On Allura itself the sidebar shows "Forks 16" and that's the part that can be made much simpler by executing a count() query instead of getting the full list. So I'd recommend looking into that.

     

    Last edit: Dave Brondsema 2016-02-24

Log in to post a comment.