allura/tests/test_globals.py:test_filtering
fails sometimes. It randomly picks a trove category to use for the test, and some trove categories are in the test data twice, so the lookup fails. It seems that in allura/command/create_trove_categories.py
the m__add_agpl_and_lppl
method creates TroveCategories that already exist in the main command
method.
Yes, it seems like
m__add_agpl_and_lppl
duplicates categories that already created in main method. I remember that in [#6848] I've compared dumps of all categories generated with old command + migration and generated with new command and they was identical. As you can see from the commits below, there were already duplicates in old command + migration scripts (014-add-trove-category-agpl.py
,016-add-trove-category-lppl.py
)git show 350faaae13433e34f3b161aa9beacdf7ee933f0a
git show 77fb5d8c3788ee3d7fa5133354ca38213c8cedb5
It seems to me that
m__add_agpl_and_lppl
can be removed to avoid duplicates (but probably need a script to remove duplicates from mongo instances where categories were already created)Related
Tickets:
#6848Closed #646.
je/42cc_7628
To reproduce test failure on master you can change it like this (it still might not fail, but after 5-6 runs usually does):
Make sure that it is not failing on a branch (with given change).
Also, created a script to remove duplicates if they present in db. It's a bit tricky, so make sure you run it with
--dry-run
first, to understand what will be done. Unfortunately, re-runningcreate_trove_categories
to re-generate categories will remove categories from all existing projects (since all categories would get new ids), so we need that script.Actually, I don't sure why project -> category relationship depends on
ObjectId
s, when categories have a natural unique id (cat_id + parent_id). Maybe we need to refactor it someday. :)Command to run to clean up duplicate trove records:
paster script development.ini allura/scripts/remove_duplicate_troves.py -- --dry-run
and watch log file for results. Then run without--dry-run