I didn't know about datadiff, thanks! I just needed a quick way to see the difference, 'cause nose.tools.assert_equal strips long output and does not provide anything useful. datadiff makes a much better job!
Allura only has "compare" link for revisions of individual files, but not for arbitrary trees, so we can't include that. I think we need a ticket to support compare between any two revisions :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm pretty sure those VMs have a lot of cores so tests will run highly parallelized. I'm thinking @with_git should be changed to @with_git2 in test_webhooks.py so that it doesn't compete with git repos that other test suites use. Thoughts?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
To avoid conflicts with ForgeGit tests, they install repo to abode-1 project, which is not useed anywhere, so I guess conflicts are within test_webhooks.py itself. Don't sure about changing with_git to with_git2, since all of suites in test_webhooks.py use with_git also (inheriting TestWebhookBase).
Maybe we can try to set_multiprocess_shared_ = True for each of test classes in test_webhooks.py?
Last edit: Igor Bondarenko 2015-02-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yeah I think you're right. Running just nosetests allura.tests.test_webhooks --processes=4 -v got several failures.
I tried setting _multiprocess_shared_ = True at each class or for the whole package and that seemed to reduce the errors but there still were a couple (hard to tell since its different each time). Maybe we need that plus use a different repo in each class?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have tried that (see ib/4542-tests) and it still fails. I think that's why git and svn tests marked as NOT_MULTIPROC_SAFE in ./run_tests :) Don't want to mark Allura as such, though...
Maybe we can change ./run_tests so that we can mark individual files as NOT_MULTIPROC_SAFE? I.e. using --exclude option for them, and then running them separately in one thread.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What about mocking the c.app.repo.init() call? As far as I can tell none of these tests should need the git repository to actually be there.
If that doesn't work out, using an --exclude from ./run_tests sounds okay.
FYI, I also tried running a single class in parallel and still got failures. Both with and without _multiprocess_shared_ = True set. So if we really wanted to isolate the tests, we'd have to do it for every single test separately.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Closed #728. Force-pushed
ib/4542(rebase)I didn't know about
datadiff, thanks! I just needed a quick way to see the difference, 'causenose.tools.assert_equalstrips long output and does not provide anything useful.datadiffmakes a much better job!Allura only has "compare" link for revisions of individual files, but not for arbitrary trees, so we can't include that. I think we need a ticket to support compare between any two revisions :)
We got some test failures on Jenkins: https://builds.apache.org/job/allura/767/testReport/
I'm pretty sure those VMs have a lot of cores so tests will run highly parallelized. I'm thinking
@with_gitshould be changed to@with_git2intest_webhooks.pyso that it doesn't compete with git repos that other test suites use. Thoughts?To avoid conflicts with
ForgeGittests, they install repo toabode-1project, which is not useed anywhere, so I guess conflicts are withintest_webhooks.pyitself. Don't sure about changingwith_gittowith_git2, since all of suites intest_webhooks.pyusewith_gitalso (inheritingTestWebhookBase).Maybe we can try to set
_multiprocess_shared_ = Truefor each of test classes intest_webhooks.py?Last edit: Igor Bondarenko 2015-02-17
Yeah I think you're right. Running just
nosetests allura.tests.test_webhooks --processes=4 -vgot several failures.I tried setting
_multiprocess_shared_ = Trueat each class or for the whole package and that seemed to reduce the errors but there still were a couple (hard to tell since its different each time). Maybe we need that plus use a different repo in each class?I have tried that (see
ib/4542-tests) and it still fails. I think that's why git and svn tests marked asNOT_MULTIPROC_SAFEin./run_tests:) Don't want to markAlluraas such, though...Maybe we can change
./run_testsso that we can mark individual files asNOT_MULTIPROC_SAFE? I.e. using--excludeoption for them, and then running them separately in one thread.What about mocking the
c.app.repo.init()call? As far as I can tell none of these tests should need the git repository to actually be there.If that doesn't work out, using an
--excludefrom./run_testssounds okay.FYI, I also tried running a single class in parallel and still got failures. Both with and without
_multiprocess_shared_ = Trueset. So if we really wanted to isolate the tests, we'd have to do it for every single test separately.Good observation! It seems to work for me. Please, review
ib/4542-tests(force-pushed).Great! I've merged it. It runs faster too :)