#7628 test_filtering fails occasionally NEEDS CMD

v1.2.0
closed
sf-2 (994)
General
2015-08-20
2014-08-19
No

allura/tests/test_globals.py:test_filtering fails sometimes. It randomly picks a trove category to use for the test, and some trove categories are in the test data twice, so the lookup fails. It seems that in allura/command/create_trove_categories.py the m__add_agpl_and_lppl method creates TroveCategories that already exist in the main command method.

Discussion

  • Igor Bondarenko - 2014-08-20

    Yes, it seems like m__add_agpl_and_lppl duplicates categories that already created in main method. I remember that in [#6848] I've compared dumps of all categories generated with old command + migration and generated with new command and they was identical. As you can see from the commits below, there were already duplicates in old command + migration scripts (014-add-trove-category-agpl.py, 016-add-trove-category-lppl.py)

    git show 350faaae13433e34f3b161aa9beacdf7ee933f0a
    git show 77fb5d8c3788ee3d7fa5133354ca38213c8cedb5

    It seems to me that m__add_agpl_and_lppl can be removed to avoid duplicates (but probably need a script to remove duplicates from mongo instances where categories were already created)

     

    Related

    Tickets: #6848

  • Dave Brondsema

    Dave Brondsema - 2014-08-22
    • Milestone: forge-aug-22 --> forge-sep-5
     
  • Dave Brondsema

    Dave Brondsema - 2014-08-22
    • status: in-progress --> open
    • assigned_to: Dave Brondsema --> nobody
    • Milestone: forge-sep-5 --> forge-aug-22
     
  • Dave Brondsema

    Dave Brondsema - 2014-08-25
    • Milestone: forge-aug-22 --> forge-sep-5
     
  • Igor Bondarenko - 2014-08-29
    • status: open --> in-progress
    • assigned_to: Igor Bondarenko
     
  • Igor Bondarenko - 2014-09-03

    Closed #646. je/42cc_7628

    To reproduce test failure on master you can change it like this (it still might not fail, but after 5-6 runs usually does):

         # set up for test
         from random import choice
         setup_trove_categories()
    -    random_trove = choice(M.TroveCategory.query.find().all())
    +    random_trove = choice(M.TroveCategory.query.find({'shortname': 'lppl'}).all())
    

    Make sure that it is not failing on a branch (with given change).

    Also, created a script to remove duplicates if they present in db. It's a bit tricky, so make sure you run it with --dry-run first, to understand what will be done. Unfortunately, re-running create_trove_categories to re-generate categories will remove categories from all existing projects (since all categories would get new ids), so we need that script.

    Actually, I don't sure why project -> category relationship depends on ObjectIds, when categories have a natural unique id (cat_id + parent_id). Maybe we need to refactor it someday. :)

     
  • Igor Bondarenko - 2014-09-03
    • status: in-progress --> code-review
     
  • Dave Brondsema

    Dave Brondsema - 2014-09-09
    • QA: Dave Brondsema
     
  • Dave Brondsema

    Dave Brondsema - 2014-09-09
    • summary: test_filtering fails occasionally --> test_filtering fails occasionally NEEDS CMD
    • status: code-review --> closed
     
  • Dave Brondsema

    Dave Brondsema - 2014-09-09

    Command to run to clean up duplicate trove records:

    paster script development.ini allura/scripts/remove_duplicate_troves.py -- --dry-run and watch log file for results. Then run without --dry-run

     
  • Dave Brondsema

    Dave Brondsema - 2015-01-05
    • Milestone: unreleased --> asf_release_1.2.0
     

Log in to post a comment.