Apache Allura™

created #145: [#4660] Mediawiki import improvements (3cp)

• status: open --> in-progress

I've got an error when trying to load mediawiki dump:

12:44:55,905 INFO  [allura.model.discuss] Auto-approving message from *anonymous
12:44:56,080 INFO  [allura.model.artifact] Snapshot version 1 of <class 'allura.model.discuss.Post'>
Traceback (most recent call last):
  File "/home/vagrant/allura.env/bin/paster", line 8, in <module>
    load_entry_point('PasteScript==1.7.4.2', 'console_scripts', 'paster')()
  File "/home/vagrant/allura.env/local/lib/python2.7/site-packages/paste/script/command.py", line 104, in run
    invoke(command, command_name, options, args[1:])
  File "/home/vagrant/allura.env/local/lib/python2.7/site-packages/paste/script/command.py", line 143, in invoke
    exit_code = runner.run(args)
  File "/home/vagrant/allura.env/local/lib/python2.7/site-packages/paste/script/command.py", line 238, in run
    result = self.command()
  File "/vagrant/sourceforge-allura/ForgeWiki/forgewiki/command/wiki2markdown/__init__.py", line 50, in command
    self.loader.load()
  File "/vagrant/sourceforge-allura/ForgeWiki/forgewiki/command/wiki2markdown/loaders.py", line 41, in load
    self.load_pages()
  File "/vagrant/sourceforge-allura/ForgeWiki/forgewiki/command/wiki2markdown/loaders.py", line 122, in load_pages
    self.load_talk(page_dir, page['title'])
  File "/vagrant/sourceforge-allura/ForgeWiki/forgewiki/command/wiki2markdown/loaders.py", line 150, in load_talk
    ignore_security=True)
  File "/vagrant/sourceforge-allura/Allura/allura/model/discuss.py", line 183, in add_post
    p = self.post(**kw)
  File "/vagrant/sourceforge-allura/Allura/allura/model/discuss.py", line 217, in post
    post.approve(file_info)
  File "/vagrant/sourceforge-allura/Allura/allura/model/discuss.py", line 521, in approve
    g.director.create_activity(author, 'posted', self, target=artifact,
AttributeError: 'NoneType' object has no attribute 'create_activity'

Seems like it is related to some recent changes, but i'm not sure what g.director should be and how to set it during command execution.

g comes from app_globals.py, so you can see it's using our new activitystream package. We'll work on a fix for this shortly.

• can permissions be converted? not sure if that is stored in the mediawiki db or not

Mediawiki db only stores mapping user -> group. Rights specified only in LocalSettings.php (proof) so we can't import that.

• the Hailo page has &nbsp_place_holder; on the last line

• I think this is fixed in https://github.com/aaronsw/html2text/commit/a5c0f3317edd1c9b61f89539b6a6e8363cf99225 which isn't released yet. Please check.

Installed html2text like so:

$ pip install git+git://github.com/aaronsw/html2text.git

and that fixes the isssue.

I can update the requirements files if that fixes it.

Please do.

This is fixed on dev now

Thanks for confirming. I've updated them on dev.

closed #145, branch - 42cc_4660

• status: in-progress --> code-review

• status: code-review --> open

• SF usernames are all lowercase, but Mediawiki has them with the first character uppercase. They should be converted to lowercase so they match.
• I didn't see anything about table of contents. See bullet #2 above

SF usernames are all lowercase, but Mediawiki has them with the first character uppercase. They should be converted to lowercase so they match.

Created #148: [#4660] Convert user names to lowercase (1cp)

I didn't see anything about table of contents. See bullet #2 above

I'll check this

• status: open --> in-progress

Closed #148. Branch - 42cc_4660.

Regarding the table of contents - when converting html to markdown all html attributes are lost (id, name, etc.), thus links with #target do not make any sense and html2markdown replaces them with a simple text. We'll think how to convert them properly. I'll create a new ticket for this soon.

created #154: [#4660] Convert table of contents (2cp)

Since markdown has a table of contents macro, isn't it "just" a matter of removing the original table of contents (which I don't even see in the mediawiki source, maybe the parser adds it) and adding [TOC] to the markdown?

Closed #154

• status: in-progress --> code-review

• status: code-review --> closed
• milestone: forge-backlog --> forge-sep-07

Originally by: gerddie

That looks really nice, but how can I use this to move my sourceforge MediaWiki data base to the Allura Wiki?

many thanks,
GW

Originally by: jayjaybillings

I'm with Gert.

We have a pretty decent-size project wiki going, but I'd like to take advantage of Allura. How can we access this conversion capability?

Thanks,
Jay