#8332 Fix slowness on some large files in code repos

v1.12.0
closed
General
nobody
2019-10-04
2019-09-03
No

Huge files (like 8MB) in code repos can generate very slow timings like this (from timermiddleware stats.log):

{
    "level": "INFO ",
    "message": {
        "call_counts": {
            "git_tool.__init__": 1,
            "git_tool._get_refs": 2,
            "git_tool._object": 2,
            "git_tool.blob_size": 1,
            "git_tool.commit": 1,
            "git_tool.is_empty": 3,
            "git_tool.open_blob": 1,
            "git_tool.shorthand_for_commit": 2,
            "git_tool.url_for_commit": 23,
            "jinja": 1,
            "ming": 222,
            "mongo": 170,
            "repo.Blob.__init__": 2,
            "repo.Blob.open": 1,
            "repo.Blob.path": 3,
            "repo.Commit.__repr__": 3,
            "repo.Commit.get_tree": 1,
            "repo.Commit.set_context": 1,
            "repo.Commit.shorthand_id": 2,
            "repo.Commit.url": 2,
            "repo.Tree.__getitem__": 2,
            "repo.Tree.__repr__": 3,
            "repo.Tree.get_blob": 1,
            "repo.Tree.path": 3,
            "repo.Tree.set_context": 2,
            "sidebar": 1,
            "sqlalchemy": 2,
            "total": 1
        },
        "request_category": "git",
        "timings": {
            "git_tool.__init__": 0,
            "git_tool._get_refs": 0,
            "git_tool._object": 1105,
            "git_tool.blob_size": 485,
            "git_tool.commit": 116,
            "git_tool.is_empty": 1191,
            "git_tool.open_blob": 1959,
            "git_tool.shorthand_for_commit": 0,
            "git_tool.url_for_commit": 1,
            "jinja": 524566,
            "ming": 2929,
            "mongo": 3029,
            "repo.Blob.__init__": 0,
            "repo.Blob.open": 1959,
            "repo.Blob.path": 0,
            "repo.Commit.__repr__": 0,
            "repo.Commit.get_tree": 191,
            "repo.Commit.set_context": 0,
            "repo.Commit.shorthand_id": 0,
            "repo.Commit.url": 0,
            "repo.Tree.__getitem__": 4,
            "repo.Tree.__repr__": 0,
            "repo.Tree.get_blob": 0,
            "repo.Tree.path": 0,
            "repo.Tree.set_context": 0,
            "sidebar": 3559,
            "sqlalchemy": 238,
            "total": 528318
        },
        "uptime": 3592,
        "url": "/p/mingw-w64/mingw-w64/ci/8331eb05125cb7fd26462b9be0a772f8d29ccf13/tree/mingw-w64-crt/Makefile.in"
    },
    "name": "stats",
    "time": "2019-08-23 12:45:01,270"
}

Related

Tickets: #8341
Tickets: #8381

Discussion

  • Dave Brondsema

    Dave Brondsema - 2019-09-03
    • status: in-progress --> review
     
  • Dave Brondsema

    Dave Brondsema - 2019-09-03

    db/8332

     
  • Kenton Taylor

    Kenton Taylor - 2019-09-05
    • status: review --> closed
     
  • Dave Brondsema

    Dave Brondsema - 2019-10-04
    • Milestone: unreleased --> v1.12.0
     

Log in to post a comment.