#6535 Ticket import for github

v1.0.1
closed
nobody
General
2015-08-20
2013-08-07
No

Use the github api for issues http://developer.github.com/v3/issues/#list-issues-for-a-repository to import tickets into allura. You can use ForgeImporter's GoogleCodeTrackerImporter (at time of writing, cj/6464 on the apache repo has the latest code for it) and TracTicketImporter as reference.

Import all the fields & values possible, including comments, events (e.g. status changes) and attachments. See the Google Code and Trac importers for examples of setting up the custom fields as needed, and referencing the remote users (keep the tickets & comments as anonymous and add identification with link into the body).

Related

Tickets: #6535

Discussion

  • Anton Kasyanov

    Anton Kasyanov - 2013-08-13

    Created #417 [#6535] Ticket import for github (4cp)

     

    Related

    Tickets: #6535

  • Anton Kasyanov

    Anton Kasyanov - 2013-08-13
    • status: open --> in-progress
     
  • Anton Kasyanov

    Anton Kasyanov - 2013-08-29

    Closed #417 [#6535] Ticket import for github, branch 42cc_6535

     

    Related

    Tickets: #6535

  • Anton Kasyanov

    Anton Kasyanov - 2013-08-29
    • status: in-progress --> code-review
     
  • Dave Brondsema

    Dave Brondsema - 2013-08-30
    • QA: Dave Brondsema
     
  • Dave Brondsema

    Dave Brondsema - 2013-08-30
    • status: code-review --> in-progress
     
  • Dave Brondsema

    Dave Brondsema - 2013-08-30

    Good start. Some improvements and fixes to do:

    For the project importer form: underscores and periods are allowed in github repo names, so the validator needs to be updated for that.

    The values for open_status_names and closed_status_names look like they were copied from the Google Code importer. Update them to match github value(s). EnableVoting was set for Google Code since there were 'star' values that were converted to votes. I don't think Github has anything similar, right? So let's not turn voting on the tracker then.

    On comments, "originally posted by" should be at the top of the comment, rather than the bottom. And all the "originally" lines should have the username be a link to the user's page on github. (Google Code import does this, including a check in case the user doesn't exist any more).

    Got the following error when importing "rauhryan/huboard". Besides fixing this particular issue, I wonder if we want a try/catch around the attachment handling if we think that it potentially could have errors on other input that we don't expect yet.

      File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 50, in import_tool
        self.process_fields(ticket, issue)
      File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 74, in process_fields
        body, attachments = self._get_attachments(issue['body'])
      File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 132, in _get_attachments
        found_matches = re.finditer(REGEXP, body, re.IGNORECASE)
      File "/var/local/env-allura/lib64/python2.7/re.py", line 186, in finditer
        return _compile(pattern, flags).finditer(string)
    TypeError: expected string or buffer
    

    We should pull the events_url API and put all the status changes into comments (formatted just like regular Allura ticket status change comments).

    I'm pretty sure all the URLs we fetch from Github need to handle pagination, so that all the tickets/comments/events are fetched. http://developer.github.com/v3/#pagination

    Github markdown has a few minor differences that we should try to convert. I've created a new ticket [#6622] for it. We can tackle that later, once this is ticket is done.

    Like, [#6533] this should have a controller for importing onto existing projects (it'll show up on the Import admin menu).

    It looks like this branch includes commits from [#6533]. I want to make sure this is easy to rebase & merge into master when we're done. Perhaps you want to have just one branch for the github tickets you're working on? Not sure what works best for your workflow. At any rate, if you can rebase when ever you merge branches together, that'll help keep the commit chain simple and ready for master.

     

    Related

    Tickets: #6533
    Tickets: #6622

  • Dave Brondsema

    Dave Brondsema - 2013-08-30

    For the events feed, there are several types of events that can occur. http://developer.github.com/v3/issues/events/#events I think we should handle "closed" "reopened" and "assigned" and ignore the others.

     
  • Igor Bondarenko

    Igor Bondarenko - 2013-09-04

    Created:

    • #423 [#6535] Ticket import for github fixes (2cp)
    • #424 [#6535] Ticket import for github: pull events and handle pagination (4cp)
    • #425 [#6535] Ticket import for github: import into existing project (2cp)
     

    Related

    Tickets: #6535

  • Igor Bondarenko

    Igor Bondarenko - 2013-09-13

    Closed #423. Pushed code to je/42cc_6535. Cleaned up branch a bit and rebased on master.

     
  • Igor Bondarenko

    Igor Bondarenko - 2013-09-13

    During testing #423, I've noticed that github has a request rate limit for the API. Unauthorized user can only hit 60 requests/hour, and user with Basic HTTP Auth or OAuth can hit 5000 requests/hour. AFAIS, exporter uses unauthenticated calls for now. Maybe we should consider the posibility to add the authentication for API calls in the future? I think 60 requests/hour won't be enough for production use.

    See http://developer.github.com/v3/#rate-limiting for details.

     
  • Dave Brondsema

    Dave Brondsema - 2013-09-13

    Yes, I agree. We'll make a separate ticket with that, since I think it'll be fairly involved.

     
  • Igor Bondarenko

    Igor Bondarenko - 2013-09-16

    Closed #425. Force-pushed je/42cc_6535

     
  • Igor Bondarenko

    Igor Bondarenko - 2013-09-17
    • status: in-progress --> code-review
     
  • Igor Bondarenko

    Igor Bondarenko - 2013-09-17

    Closed #424. Force-pushed je/42cc_6535

     
  • Dave Brondsema

    Dave Brondsema - 2013-09-19
    • status: code-review --> closed
    • Milestone: forge-backlog --> forge-sep-20
     

Log in to post a comment.