#6656 Github oauth application

v1.1.0
closed
nobody
import (116) github (25) 42cc (432)
General
Cory Johns
2015-08-20
2013-09-13
Dave Brondsema
No

To avoid low rate limits for anonymous API access, we should use an oauth app. http://developer.github.com/v3/#rate-limiting

As best I can tell https://pypi.python.org/pypi/requests-oauthlib is the best oauth v2 library to use. (The "oauth2" library we already use, despite its name, only is for oauth v1) It's license is BSD/MIT style, based on the very good 'requests' library, has good docs and has an active git repo.

I am not super familiar with oauth v2 and github's setup, but based on what I know, here's how I think it should work. Each Allura instance (e.g. your development host, SourceForge, etc) will need to set up a their own Github OAuth App. Then those keys can be placed in the ini file. Our github importer code will then do the oauth flow to authorize the user requesting an import. No scope is necessary since we're just doing public readonly fetching. We should store the appropriate user tokens (via user.set_tool_data) so that they are available for the background task, and also can be re-used if the user wants to run another import.

This should all go through a shared mechanism (e.g. override the base ProjectExtractor.urlopen in GitHubProjectExtractor) so that it's used for all github related API access. This code should also check the rate limit values and when it reaches the limit, log a warning, and sleep for the amount of time needed until the limit resets).

Of course, we can modify this as needed if my understanding of github oauth isn't correct.

Related

Tickets: #6656

Discussion

  • Igor Bondarenko - 2013-09-16
    • status: open --> in-progress
     

  • Igor Bondarenko - 2013-09-16

    Created #437: [#6656] Github oauth application (4cp)

     

    Related

    Tickets: #6656

  • Igor Bondarenko - 2013-10-24

    Workflow you described is good and looks like github's web application flow. http://developer.github.com/v3/oauth/

    But implementing it will require additional interaction with user, such as redirecting to github to authenticate/allow access, when user accessing import page and has no oauth token yet. Is that okay with you?

    I personally prefer the above, but there also a couple of options to implement server-server auth, that doesn't require interaction with user:

    1. send client_id and client_secret (oauth app's keys) with every request
    2. get auth token programmatically http://developer.github.com/v3/oauth/#non-web-application-flow

    (1) is very simple to implement and it increases rate limit, but the downside is we'll have rate limit for per entire app, not per user, as with tokens.

    (2) requires basic auth, so we should ask user for his github credentials, which is bad, or create github user for allura instance and put credentials to ini, which also seems pretty bad.

     

  • Dave Brondsema

    Dave Brondsema - 2013-10-24

    Yep I think having the user authorize our app on GitHub is the right way to do it is we don't have to deal with the limitations you describe in the alternate options

     

  • Igor Bondarenko - 2013-10-25

    Closed #437. je/42cc_6656

    • Create github app. Callback must be set to your host (e.g. https://sf-fortytwo-1030.sb.sf.net/). It's important, since GitHub matches redirect_uri passed from authorization calls to it.
    • Add keys to ini (see below)
    • Go to project import page or individual tool import page for github. You'll be redirected to github, asked to authorize app, then redirected back to form, and then you can run import as usual

    Ini options:

    github_importer.client_id = <client_id>
github_importer.client_secret = <secret>

    If app isn't set up (no app keys in config) it will still work, but use unauthorized, low rate limit access

     

  • Igor Bondarenko - 2013-10-25
    • status: in-progress --> code-review
     

  • Cory Johns - 2013-11-13
    • QA: Cory Johns
     

  • Cory Johns - 2013-11-13
    • status: code-review --> closed
    • Milestone: forge-backlog --> forge-nov-15
     

Log in to post a comment.