Apache Allura™

It looks like colander is only used in project-import.py and that doesn't have any test coverage, so we'll have to do manual code checking and testing for this. Have you looked at the colandar changelog? Any significant changes in what's being upgraded?

Next to project-import.py is a sample project-import.json data file, you could try running it with that and see if it works.

Acording to the changelog, colander adds support to Python 3.4 and 3.5 starting from version 1.1 and support to Python 3.6 starting from version 1.3.2.

I ran that test, with colander==0.9.3, but got the following error:

$ docker-compose run web python scripts/project-import.py scripts/project-import.json Projects
Starting allura_mongo_1 ... done
Starting allura_solr_1 ... done
Traceback (most recent call last):
File "scripts/project-import.py", line 379, in <module>
sys.exit(main(parse_options()))
File "scripts/project-import.py", line 307, in main
nbhd = M.Neighborhood.query.get(name=options.neighborhood)
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/odm/mapper.py", line 343, in get
return self.find(kwargs).first()
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/odm/mapper.py", line 323, in inner
return method(self.mapped_class, args, kwargs)
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/odm/odmsession.py", line 178, in find
ming_cursor = self.impl.find(m.collection, args, **kwargs)
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/session.py", line 75, in find
collection = self._impl(cls)
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/session.py", line 52, in _impl
return self.db[cls.m.collection_name]
File "/allura-data/virtualenv/local/lib/python2.7/site-packages/ming/session.py", line 59, in db
raise exc.MongoGone('No MongoDB connection for "%s"' % getattr(self, '_name', 'unknown connection'))
ming.exc.MongoGone: No MongoDB connection for "unknown connection"</module>

I also see some incompatibilities mentioned on https://docs.pylonsproject.org/projects/colander/en/latest/changes.html too. They seem pretty minor/obscure though.

Ah yea you have to run those commands with a helper script to get all the database connections and other context set up. See https://forge-allura.apache.org/docs/getting_started/administration.html#commands-scripts-and-tasks

Yes. There are some incompatibilities but they don't impact the functionality of project-import.

Now I ran the test for project-import.py with the helper script and there were no errors.

Actually, there was an error, but it's not related to Colander. It is caused by "awards": [What?] in the project-import.json:

InvalidId: u'What?' is not a valid ObjectId, it must be a 12-byte input of type 'str' or a 24-character hex string

I tried to set a valid ObjectId, but got another error:

colander.Invalid: {'awards.0': u'Invalid award "0123456789ab0123456789ab".'}

Then, I set "awards": [] and there were no errors.

What is that "awards" supposed to be?

Yea it looks like the sample file has a bogus entry there :( Awards need to be mongo ObjectIds, but those will be context dependent, so setting it to [] like you've done would be best. Can you commit that? This is looking good besides that.

Sure. I forgot to mention that I also removed "GNU General Public License (GPL)" from "trove_licenses", because it produced the following error:

colander.Invalid: {'trove_licenses.1': u'"GNU General Public License (GPL)" is not a valid trove category.'}

Is that another bogus entry?

Ok. I will update it.