#4258 Capture stdout/stderr from svn import failures
This error isn't very useful on its own. This is from the "SourceForge Repo Clone Failure" email, which occurs after using the SVN admin option for importing a repo.
Traceback (most recent call last): File "/var/local/allura/Allura/allura/tasks/repo_tasks.py", line 31, in clone cloned_from_url) File "/var/local/allura/Allura/allura/model/repository.py", line 227, in init_as_clone self._impl.clone_from(source_url) File "/var/local/allura/ForgeSVN/forgesvn/model/svn.py", line 176, in clone_from subprocess.check_call(['svnsync', 'init', self._url, source_url]) File "/usr/lib64/python2.7/subprocess.py", line 511, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['svnsync', 'init', u'file:///svn/p/testupgradetool/code', u'http://testupgradetool.svn.sourceforge.net/svnroot/testupgradetool']' returned non-zero exit status 1
labels: --> 42cc
Description has changed:
Diff:
created #75: [#4258] Capture stdout/stderr from svn import failures (2cp)
Related
Tickets:
#4258closed #75, branch - 42cc_4258
This could suffer from the issue mentioned in wait, namely that the OS pipe buffer could fill up and block the call.
It should either use communicate or check_output.
created #84: [#4258] Use communicate or check_output instead of wait
Related
Tickets:
#4258Originally by: tramadolmen
You are wrong :)
What do you mean? Please explain more.
Originally by: tramadolmen
Without waiting for command complete some tests are failed. If i right remember problem was in commits save. My first version was without wait() :). After broken test - i read the source of subprocess.py file and find that:
def call(*popenargs, **kwargs): return Popen(*popenargs, **kwargs).wait() def check_call(*popenargs, **kwargs): retcode = call(*popenargs, **kwargs) cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] if retcode: raise CalledProcessError(retcode, cmd) return retcodeYou see in call function called wait command. So in my code i simply copy paste original check_call function code and add stdout, stderr read
The problem is not with the fact that it
waits, but that if youwaitwithout reading any of the input from a pipe, the OS pipe buffer could fill up and the OS will block the process until some input is read from it, resulting in a deadlock. This is mentioned in the warning for wait as well as the note for check_call. It won't always, or even usually, of course, but it's a definite possibility, especially with large repos.The solution is to read the output to prevent blocking, which can be done using either
communicateorcheck_output(communicate allows you to separate thestdoutandstderrbut you have to do more boiler-plate, whilecheck_outputwill only returnstdoutso you have to pipestderrintostdoutif you want to retrieve it, which we do).Originally by: tramadolmen
I checked what you say about process blocking. Yes you are right
Updated the branch 42cc_4258 and closed #84
Perfect