#6456 Create project importer for Google Code NEEDS CONFIGTREE

v1.0.0
closed
Cory Johns
import (116) google-code (27) sf-4 (350)
General
Dave Brondsema
2015-08-20
2013-07-12
Cory Johns
No

Create a controller that provides a landing page to import a project from Google Code hosting. The landing page should ask for a project name and provide a list of all the tools (wiki, downloads, tracker, etc) supported for import, with a checkbox for each. When submitted, it should pull the basic project info (description, icon, license) from the summary page (e.g., https://code.google.com/p/modwsgi/) and create a stub project with that data.

The description and icon values have itemprop attributes of "description" and "image", respectively, but the license will take a little bit of processing to extract.

The importers for the tools will be done in separate tickets. The list of supported tools should be discovered by iterating the entry-points in the group allura.importers looking for importers with a source attribute equal to "Google Code". Use the importer's target_app attribute to get the tool_label (and possibly icon?) for display. See the mailing list for more information.

The project importer should register itself as an entry point under the allura.project_importers group, and should expose a source attribute with the value of "Google Code". I'm thinking the controller should be under an app-less tool called ForgeImporters, e.g. ForgeImporters/forgeimporters/google/project.py or such.

Related

Tickets: #6458
Tickets: #6461
Tickets: #6464

Discussion

  • Dave Brondsema

    Dave Brondsema - 2013-07-15
    • Milestone: forge-backlog --> forge-jul-26
     

  • Dave Brondsema

    Dave Brondsema - 2013-07-15
    • Size: --> 4
     

  • Cory Johns - 2013-07-15

    Some example code parsing out project information is available here.

    We'll also need to surface this new project import view via a parameter to the add_project page, i.e., /p/add_project?import=Google+Code. The NeighborhoodController will need to do discovery of available project importers in __init__() and add_project() will need to redispatch based on the discovery mapping and parameter.

     

  • Cory Johns - 2013-07-15
    • status: open --> in-progress
    • assigned_to: Cory Johns
     

  • Cory Johns - 2013-07-16

    The base framework is ready for review in allura:cj/6456. We could merge that one commit so that the other tickets could be started in parallel.

     

  • Cory Johns - 2013-07-17

    The commit was amended with some improvements, and tests were added.

     

  • Cory Johns - 2013-07-19
    • status: in-progress --> code-review
     

  • Cory Johns - 2013-07-19

    allura:cj/6456
    sftheme:cj/6456

     

  • Dave Brondsema

    Dave Brondsema - 2013-07-22
    • QA: Dave Brondsema
     

  • Dave Brondsema

    Dave Brondsema - 2013-07-22
    • status: code-review --> in-progress
     

  • Dave Brondsema

    Dave Brondsema - 2013-07-22

    The project_unixname variable name isn't a great name. Unix name is a SF-specific thing (and includes the nbhd suffix). Better to call it shortname

    GoogleCodeProjectImporter only works for /p/ nbhd. Shouldn't it be able to work with any neighborhood?

    "SourceForge" is used in a flash message, which isn't appropriate for other sites using Allura. I know we have the same text during regular project registration (site note: are we copying too much from that logic? will we have to copy it all again for other importers?) Lets not further the SourceForge hard-coding here. Perhaps we need a .ini setting for site name?

    !important in css isn't a good thing.

    Validation of the google code project name has an error message that says "Usernames must include only..."

    Functionality works well.

     

  • Dave Brondsema

    Dave Brondsema - 2013-07-22

    Also, login should be required for the import page. And

    File '/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/google/project.py', line 60 in process
  except exceptions.ProjectOverlimitError:
NameError: global name 'exceptions' is not defined
     

  • Cory Johns - 2013-07-22

    I used project_unixname to match the add_project form. I can change it, though; I do like shortname better. I think was originally going to use the same form and check_names AJAX call, but didn't anyway.

    Right about the neighborhood, and there's no reason for it to have been hard-coded. I'll fix.

    +1 to .ini setting for site name.

    Copy/paste error. I really wanted to re-use add_project.css but could figure out how to reference the file since I'm not using EW. Help is appreciated there.

    That's a standard FormEncode validation message. What should it say instead?

    Upon further discussion with Tim, I also need to change how the tool importers are dispatched to not assume that import_tool will manage tasking.

     

  • Cory Johns - 2013-07-22

    Oh, and regarding re-copying logic for other project importers, I expect most of the logic in GoogleCodeProjectImporter and google/templates/project.html could be moved up to base.ProjectImporter and templates/project_base.html but I was deferring that until we have another project importer to actually see how much overlap there is.

     

  • Dave Brondsema

    Dave Brondsema - 2013-07-22

    It should say "Google Code project names only" instead of "usernames"

     

  • Cory Johns - 2013-07-22

    Oh, I missed the "usernames" bit.

     

  • Cory Johns - 2013-07-22
    • status: in-progress --> code-review
     

  • Cory Johns - 2013-07-22

    Force-pushed to:
    allura:cj/6456
    sftheme:cj/6456

     

  • Dave Brondsema

    Dave Brondsema - 2013-07-23
    • summary: Create project importer for Google Code --> Create project importer for Google Code NEEDS CONFIGTREE
    • status: code-review --> in-progress
     

  • Dave Brondsema

    Dave Brondsema - 2013-07-23

    Merged because the code looks all good, but not closing the ticket because I think it's important to document the importer classes. I see some good docstrings already, so let's expose those in the API docs (perhaps we'll want/need to move the base importer into Allura? and leave ForgeImporters for the specific instances). A bit of narrative high-level explanation might be good too.

     

  • Cory Johns - 2013-07-23
    • status: in-progress --> code-review
     

  • Cory Johns - 2013-07-23

    Refactored the dispatcher to remove the dependency on forgeimporters from allura and added docs for the ForgeImporter package. It's not currently integrated with the docs in Allura because I'm still not sure how to organize the docs directory structure nor how to tie the different packages' docs together.

    Force-pushed to:
    allura:cj/6456

     

  • Cory Johns - 2013-07-25

    Force-pushed with project_updated events for project and code importers, a test fix, and updates from master:
    allura:cj/6456

     

  • Dave Brondsema

    Dave Brondsema - 2013-07-31
    • status: code-review --> closed
     

Log in to post a comment.