#6484 Move ForgeWiki mediawiki importer (GPL dep) into standalone importer - NEEDS CONFIGTREE

v1.1.0
closed
nobody
General
Cory Johns
2015-08-20
2013-07-23
No
  • Move forgewiki.command.wiki2markdown:Wiki2MarkDownCommand and everything that goes with it to a separate repo (e.g. a new SourceForge project for any non-Apache License importers)
  • adapt it to use the ToolImporter framework
  • remove items from requirements-optional.txt
  • release it on pypi
  • link to it from https://sourceforge.net/p/allura/wiki/Extensions/

Discussion

  • Dave Brondsema

    Dave Brondsema - 2013-08-07

    We also need to move the forgewiki.scripts.wiki_from_trac code into the tracwikiimporter repo and do similar steps. Can also make its "import html2text" non-optional then.

     
  • Dave Brondsema

    Dave Brondsema - 2013-08-07

    Allura/allura/scripts/trac_export.py has a small dependency on GPL'd html2text also, and would be a good fit in the tracwikiimporter repo.

     
  • Dave Brondsema

    Dave Brondsema - 2013-11-06
    • Labels: import --> import, 42cc
    • Status: open --> in-progress
     
  • Igor Bondarenko - 2013-11-07

    FYI forgewiki.command.wiki2markdown:Wiki2MarkDownCommand was moved to forgewiki.scripts.wiki2markdown:Wiki2Markdown script in [adaf63]

    adapt it to use the ToolImporter framework

    Now script reads directly from mediawiki database (MySQL) and imports that into allura. However ToolImporters are designed to fetch data through some kind of API (or parsing web pages). Do you want Wiki2Markdown fetch data from API/parsing page instead of direct connection to database?

    UPD:

    Figured out we can ask user for a connection string to mysql, instead of project name on this particular importer's page, but don't sure if that's what you intended.

    Also, do you need a 'script' ability of Wiki2Markdown (running it from paster script)? Or we have a way to run ToolImporter as script anyway?

     
  • Dave Brondsema

    Dave Brondsema - 2013-11-08

    This importer should still use all the existing mechanisms of the script (connecting to a local database). It'll be a little different from most other importers, but that's ok. Since we're moving it to a separate repo, it won't be in Allura instances by default and won't confuse people then :)

    I'd like to have all the connection details (database, attachment dir, etc) be hidden from the user. So the user would just enter the project name that they want to import from. This would mean some custom code to put together all the options since they aren't all static. (e.g. we currently run the script with these options --db_config_prefix=hostedapps.db. -a /nfs/sfa-mediawiki-content/project/f/fa/fancypants/ --db_name p_fancypants_mediawiki -n Projects -p fancypants)

    We don't need to be able to run this as a script at this point. But if it makes more sense for the importer to call existing scripttask, that might be nice.

    Perhaps the best thing to do at this point is create a ToolImporter that is SourceForge internal (in our forge-classic repo) that calls the existing scripttask with the proper parameters constructed. And not worry about moving the wiki2markdown files out of the allura repo at this point. I haven't looked at the code recently, so I might be forgetting something though.

     
  • Igor Bondarenko - 2014-01-02
    • status: in-progress --> code-review
     
  • Igor Bondarenko - 2014-01-02

    Closed #492.

    Allura: je/42cc_6484

    Created project for mediawiki importer. Granted admin permissions to Dave, Tim and Cory. https://sourceforge.net/p/mediawikiimporter/git/ci/master/tree/

    New options for Allura's ini for mediawiki importer:

    mediawikiimporter.db_config_prefix = hostedapps.db.
    mediawikiimporter.attachments_dir_prefix = /nfs/sfa-mediawiki-content/project/
    mediawikiimporter.db_name_template = p_%s_mediawiki
    hostedapps.db.host = localhost
    hostedapps.db.port = 3306
    hostedapps.db.user = user
    hostedapps.db.password = password
    

    For trac importer changes see merge request https://sourceforge.net/p/tracwikiimporter/git/merge-requests/1/

     
  • Cory Johns - 2014-01-10
    • QA: Cory Johns
     
  • Cory Johns - 2014-01-13
    • status: code-review --> validation
     
  • Cory Johns - 2014-01-13
    • summary: Move ForgeWiki mediawiki importer (GPL dep) into standalone importer --> Move ForgeWiki mediawiki importer (GPL dep) into standalone importer - NEEDS CONFIGTREE
     
  • Dave Brondsema

    Dave Brondsema - 2014-01-15
    • status: validation --> closed
     

Log in to post a comment.