#7878 Try unicode_literals


Try from __future__ import unicode_literals and possibly absolute_import, division, print_function too, in our existing Allura .py files. See what sort of issues may arise.

Related email thread starts at http://mail-archives.apache.org/mod_mbox/allura-dev/201504.mbox/%3C553955A5.6020006%40brondsema.net%3E


  • Dave Brondsema

    Dave Brondsema - 2015-05-18
    • labels: sf-current --> sf-current, sf-2
  • Igor Bondarenko

    Igor Bondarenko - 2015-05-19
    • labels: sf-current, sf-2 --> sf-current, sf-2, 42cc
    • status: open --> in-progress
    • assigned_to: Igor Bondarenko
  • Igor Bondarenko

    Igor Bondarenko - 2015-05-20

    Closed #771. ib/7878

    Ok, it was worse than I expected :) I've managed to fix some of the issues. A lot of tests still failing, though, and some pages raise unicode errors (e.g. git repo main page) and it need a lot of manual checking.

    What I did:

    • run futurize --stage1 --all-imports --unicode-literals -w **/*.py
    • changed name and polymorphic_identity to string in all ming models/collections
    • changed package_data dictionary keys to string for setup.py
    • converted arguments to sting for the test app (our version of WebTest does not handle unicode well, newer versions do)
    • Fixed AntiSpam decorator. It is quick fix in order to get at least some of the pages to render. We might want to look at it closer and test it more. Don't sure it will work for all the cases. I didn't dig deep in the code to save time to investigate further.
    • Changed some of the headers/environ set from our internal middlewares.


    The last point was hardest to spot and debug. It is reproducible only if you run wsgi app. If you working locally with paster serve this does not arise. Adding LoggingMiddleware to allura.wsgi from here helped a lot. It prints request/response environ to apache error log.

    See also commit comments for some details.

    Further work that need to be done:

    • Look for places where we set response headers, convert keys and vals to plain strings
    • Check as many as possible pages by hand to discover and fix unicode errors
    • Check stuff that interacts with FS (import/export, repo tarballs, etc)
    • ForgeHg and other packages we maintain will probably need steps mentioned above too
    • Fix tests
    • Similar fixes for internal SourceForge stuff?


    Maybe we should tackle these incrementally one by one if we wanna proceed further. They might be intertwined, though.


    Last edit: Igor Bondarenko 2015-05-20
  • Igor Bondarenko

    Igor Bondarenko - 2015-05-20
    • status: in-progress --> review
  • Dave Brondsema

    Dave Brondsema - 2015-05-28

    Sounds like you made good headway, but as you said, worse than expected. I think some of the fixes (e.g. ming) we might want to try to fix upstream (seems like ming should handle unicode names).

    Since there's quite a bit of additional work and testing, with minimal benefit (ready for py3 (which far from anyway) and save some unicode interpolation errors)... I'm ok with letting this sit. Unless someone really wants to make it their personal mission :D

  • Dave Brondsema

    Dave Brondsema - 2015-06-01
    • labels: sf-current, sf-2, 42cc --> sf-2, 42cc
  • Dave Brondsema

    Dave Brondsema - 2020-01-21
    • labels: sf-2, 42cc -->
    • assigned_to: Igor Bondarenko --> Dave Brondsema
  • Dave Brondsema

    Dave Brondsema - 2020-01-21

    Done in db/7878

  • Kenton Taylor

    Kenton Taylor - 2020-01-24
    • status: review --> closed
  • Dave Brondsema

    Dave Brondsema - 2021-05-17
    • Milestone: unreleased --> v1.13.0

Log in to post a comment.