Apache Allura™ / Tickets / #4964 Git file handles not cleaned up in taskd procs

#4964 Git file handles not cleaned up in taskd procs

Milestone: v1.0.0

Status: closed

Owner: Cory Johns

Labels: migration (63) stability (19) sf-1 (616)

Component: General

Reviewer: Dave Brondsema

Updated: 2015-08-20

Created: 2012-09-18

Creator: Dave Brondsema

Private: No

See [#3312] for previous work on this issue; perhaps it just need to be applied to taskd.

Our long-running taskd processes are accumulating git filehandles. Proof:

$ sudo -u allura /usr/sbin/lsof -p 6406 | grep git | tail
paster  6406 allura  336r  unknown                                       /p/odcleanstore/code.git/objects/pack/pack-a4022ff36c0239c4040456144c6ddfdf6e04e6fc.idx (stat: No such file or directory)
paster  6406 allura  337r  unknown                                       /p/odcleanstore/code.git/objects/pack/pack-4afcdc5c1fa600738e95b3a33cf9cb646bae47b0.idx (stat: No such file or directory)
paster  6406 allura  338r  unknown                                       /p/ismrmrd/code.git/objects/pack/pack-ec1295a6a871c039c33d1538d691b5de5325c7bf.pack (stat: No such file or directory)
paster  6406 allura  339r  unknown                                       /p/ismrmrd/code.git/objects/pack/pack-ec1295a6a871c039c33d1538d691b5de5325c7bf.idx (stat: No such file or directory)
paster  6406 allura  340r  unknown                                       /p/ufoai/code.git/objects/pack/pack-8e9dd5fb5ca71e1658e3f666d74f4127d55ed5fb.pack (stat: No such file or directory)
...

Discussion

Cory Johns - 2012-09-18

We appear to be running afoul of this as-of-yet unresolved bug in GitPython.

The only suggested work-around is to switch to the GitCmdObjectDb backend, but according to the documentation, this should actually be more efficient except for memory use when reading large files. It is unclear how much that will affect us, but we don't keep repo objects around for extended periods, so it will possibly be a non-issue.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Cory Johns - 2012-09-18

status: open --> code-review

assigned_to: Cory Johns

qa: Dave Brondsema

size: --> 1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2012-09-18

status: code-review --> validation

milestone: forge-oct-05 --> forge-sep-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2012-09-19

Seems to be working well

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Dave Brondsema - 2012-09-19

status: validation --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Apache Allura™

Forge software for hosting software projects

Milestone

Searches

Help

#4964 Git file handles not cleaned up in taskd procs

Discussion