#6139 Trac wiki import

v1.0.0
closed
nobody
General
2015-08-20
2013-04-18
No

See [#4213] for trac ticket import.

This can be a standalone script like the trac ticket import, and use the existing wiki APIs. That won't allow for history or attachment import, but that's not too important yet.

Content conversion can be via regexes, e.g. https://gist.github.com/619537 and https://gist.github.com/1286682 or pull down the HTML and convert it to markdown using html2text (GPL, already an optional dep). We'll have to see what works best.

SF should have an internal task to run this for projects easily.

Related

Tickets: #4213
Tickets: #6139
Tickets: #6140
Tickets: #6480

Discussion

  • Dave Brondsema

    Dave Brondsema - 2013-04-19
    • Milestone: forge-backlog --> forge-may-17
     
  • Dave Brondsema

    Dave Brondsema - 2013-04-22
    • labels: import --> import, 42cc
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-04-23

    If I understood correctly we need to:
    1. Create script that imports trac wiki into json file (as trac_export does for tickets).
    2. Add ability to import from above to the allura wiki (using allura_import.py script, I guess)

    Existing wiki API that you talking about is Allura/allura/lib/import_api.py?

    Am I on the right track?

     
  • Dave Brondsema

    Dave Brondsema - 2013-04-23

    #1 yep, exactly. And also convert the formatting to Markdown, so it's ready to import in #2

    For step #2 there already is an API for creating wiki pages. It's not an import API, just a regular REST API. https://sourceforge.net/p/forge/documentation/Allura%20API/#wiki has a few details about it. So the import script can use that to create the pages.

     
  • Igor Bondarenko

    Igor Bondarenko - 2013-04-24
    • status: open --> in-progress
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-04-24

    Created #326: [#6139] Trac wiki import (3cp)

     

    Related

    Tickets: #6139

  • Dave Brondsema

    Dave Brondsema - 2013-05-01
    • Milestone: forge-may-17 --> forge-backlog
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-06-20
    • status: in-progress --> code-review
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-06-20

    Closed #326. je/42cc_6139

    Run export

    `./scripts/wiki-export.py "https://sourceforge.net/apps/trac/paintown/" -v -o wiki.json`
    

    Also there are the --converter option that allows to specify how to convert wiki text. Options are:
    1. 'html2text' (fetch page as html, feed it to html2text with some post-processing of links)
    2. 'regex' (fetch page as plain-text (trac-mardown-formatted) and convert it using regexps)

    The (1) is default and works much better.

    Run import:

    python ./scripts/allura_import.py -a "bfb31674-356a-4daa-af18-f52d2c0945d3" -s "70da41ed922758594e6de3653fdafd3df8b2c370cf6838135808519d0c24ea295527e8a31f00a50f" -p test -w wiki --base-url "http://127.0.0.1:8080/" -v wiki.json
    

    I've tested with these tracs:

    There are no new tests, 'cause export script is standalone script and is hard to test and import just creating pages through the wiki rest api, which is already tested.

     
  • Tim Van Steenburgh

    • QA: Tim Van Steenburgh
    • Milestone: forge-backlog --> forge-jul-12
     
  • Tim Van Steenburgh

    • status: code-review --> in-progress
     
  • Tim Van Steenburgh

    The import looks great, but there are a couple of things to fix:

    1. Copy the contents of the trac wiki home page to the Home page in the Allura wiki.
    2. Replace the wiki-toc div from trac with a Markdown [TOC] tag. Right now there's a numbered list TOC being generated at the top of each Allura wiki page, but the items aren't hyperlinked.
    3. For bonus points, create a ScriptTask that can do the export/import in one step.
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-07-09

    Created #399: [#6139] Trac wiki import followup (2cp)

     

    Related

    Tickets: #6139

  • Dave Brondsema

    Dave Brondsema - 2013-07-10

    I haven't looked into this code, but I expect the "WikiStart" page may preserve that name (similar to our mediawiki import that has "Main_Page"). We can keep that content in the "WikiStart" page instead of renaming it to "Home". And then just set the wiki config to use "WikiStart" as the default page instead of "Home". That'll keep names consistent and make redirects work seamlessly.

     
  • Dave Brondsema

    Dave Brondsema - 2013-07-15
    • Milestone: forge-jul-12 --> forge-jul-26
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-07-26
    • status: in-progress --> code-review
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-07-26

    Closed #399. je/42cc_6139 (forced update)

    You can run ScriptTask as follows:

    $ allurapaste script /var/local/config/production.ini /var/local/allura/ForgeWiki/forgewiki/scripts/wiki_from_trac/wiki_from_trac.py -- https://sourceforge.net/apps/trac/sourceforge/ -a tck8fb1f4bbf74cedfdbe0b -s 906239d69338e9b303b7fc250513b3f5b616773c9a9ea90f7968a44c6d2029d3e996c325f7d8b9f6 -p test2 -w wiki2 --base-url https://sf-fortytwo-1030.sb.sf.net/ -v
    
     
  • Dave Brondsema

    Dave Brondsema - 2013-07-26
    • Milestone: forge-jul-26 --> forge-aug-09
     
  • Tim Van Steenburgh

    • status: code-review --> closed
     

Log in to post a comment.