Apache Allura™

Created #417 [#6535] Ticket import for github (4cp)

• status: open --> in-progress

Closed #417 [#6535] Ticket import for github, branch 42cc_6535

• status: in-progress --> code-review

• QA: Dave Brondsema

• status: code-review --> in-progress

Good start. Some improvements and fixes to do:

For the project importer form: underscores and periods are allowed in github repo names, so the validator needs to be updated for that.

The values for open_status_names and closed_status_names look like they were copied from the Google Code importer. Update them to match github value(s). EnableVoting was set for Google Code since there were 'star' values that were converted to votes. I don't think Github has anything similar, right? So let's not turn voting on the tracker then.

On comments, "originally posted by" should be at the top of the comment, rather than the bottom. And all the "originally" lines should have the username be a link to the user's page on github. (Google Code import does this, including a check in case the user doesn't exist any more).

Got the following error when importing "rauhryan/huboard". Besides fixing this particular issue, I wonder if we want a try/catch around the attachment handling if we think that it potentially could have errors on other input that we don't expect yet.

  File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 50, in import_tool
    self.process_fields(ticket, issue)
  File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 74, in process_fields
    body, attachments = self._get_attachments(issue['body'])
  File "/home/dbrondsema/dbrondsema-1019/forge/ForgeImporters/forgeimporters/github/tracker.py", line 132, in _get_attachments
    found_matches = re.finditer(REGEXP, body, re.IGNORECASE)
  File "/var/local/env-allura/lib64/python2.7/re.py", line 186, in finditer
    return _compile(pattern, flags).finditer(string)
TypeError: expected string or buffer

We should pull the events_url API and put all the status changes into comments (formatted just like regular Allura ticket status change comments).

I'm pretty sure all the URLs we fetch from Github need to handle pagination, so that all the tickets/comments/events are fetched. http://developer.github.com/v3/#pagination

Github markdown has a few minor differences that we should try to convert. I've created a new ticket [#6622] for it. We can tackle that later, once this is ticket is done.

Like, [#6533] this should have a controller for importing onto existing projects (it'll show up on the Import admin menu).

It looks like this branch includes commits from [#6533]. I want to make sure this is easy to rebase & merge into master when we're done. Perhaps you want to have just one branch for the github tickets you're working on? Not sure what works best for your workflow. At any rate, if you can rebase when ever you merge branches together, that'll help keep the commit chain simple and ready for master.

For the events feed, there are several types of events that can occur. http://developer.github.com/v3/issues/events/#events I think we should handle "closed" "reopened" and "assigned" and ignore the others.

Created:

• #423 [#6535] Ticket import for github fixes (2cp)
• #424 [#6535] Ticket import for github: pull events and handle pagination (4cp)
• #425 [#6535] Ticket import for github: import into existing project (2cp)

Closed #423. Pushed code to je/42cc_6535. Cleaned up branch a bit and rebased on master.

During testing #423, I've noticed that github has a request rate limit for the API. Unauthorized user can only hit 60 requests/hour, and user with Basic HTTP Auth or OAuth can hit 5000 requests/hour. AFAIS, exporter uses unauthenticated calls for now. Maybe we should consider the posibility to add the authentication for API calls in the future? I think 60 requests/hour won't be enough for production use.

See http://developer.github.com/v3/#rate-limiting for details.

Yes, I agree. We'll make a separate ticket with that, since I think it'll be fairly involved.

Closed #425. Force-pushed je/42cc_6535

• status: in-progress --> code-review

Closed #424. Force-pushed je/42cc_6535

• status: code-review --> closed
• Milestone: forge-backlog --> forge-sep-20