Apache Allura™

Closed #392. je/42cc_3154_link_api

Since this ticket is about ForgeLink API and not directly related to the export, I've created a separate branch for that. You may wish to review this changes separately and merge them to master before full export is ready.

Documentation for https://sourceforge.net/p/forge/documentation/Allura%20API/

Endpoint: _prefix/_

* GET _prefix/_ - returns the url
* POST _prefix/_ - updates the url
     * parameter url: the url (for example "http://google.com")

Closed #391. je/42cc_3154_blog_api

Docs:

Endpoint: _prefix/_

* GET _prefix/_ - returns a list of posts, including title and API url
* GET _prefix/year/month/title_ - returns a JSON representation of a post
* POST _prefix/_ - creates a post
    * parameter title - the title of the post
    * parameter text - the text of the post
    * parameter labels - labels of the post
    * parameter state - 'draft' or 'published'
* POST _prefix/year/month/title_ - updates an existing post
    * parameter parameters: 
    * parameter title - the title of the post
    * parameter text - the text of the post
    * parameter labels - labels of the post
    * parameter state - 'draft' or 'published'

merged je/42cc_3154_link_api

FYI, tested with curl-oauth:

HOST=sf-dbrondsema-1015.sb.sf.net
curl-oauth --domain $HOST --request-token --request-token-url https://$HOST/rest/oauth/request_token --consumer-key ABC --consumer-secret XYZ --request-token-authorize-url https://$HOST/rest/oauth/authorize --request-token-access-token-url https://$HOST/rest/oauth/access_token
curl-oauth --domain sf-dbrondsema-1015.sb.sf.net https://$HOST/rest/p/testit/docs/ -X POST -d 'url=http://slashdot.org'

Closed #394. je/42cc_3154 (forced update)

For the blog API:

• can you add the author username to BlogPost.__json__ ?
• when creating a new post, it'd be slightly better to return a 201 instead of 200. And include a Location: header in the response, pointing to the URL of the new post
• There seems to be some code duplication between the regular controllers and the API controllers. For example, creating a new BlogPost involves setting several fields, calling make_slug() making a Thread and committing (side not: post.viewable_by = ['all'] is unique and I'm not sure why). Can you check for better code re-use on post, update, list and viewing? It seems to me that we could refactor and get some cleaner code that uses the same methods. Or perhaps even one controller could just call the other controller with no new methods/helpers needed. And for listing posts, it wouldn't be bad if the API had pagination like the regular controller too (I don't remember offhand if the same pagination classes are used for API pagination though)

Closed #389. je/42cc_3154

Originally by: *anonymous

Was it resolved ?

I do not understand the "bla bla bla" which is comming to my e-mails.

Could you please state clearly can we back up our Tickets ?

Thank

We've been closing some of our subtasks for this issue. We're tracking a lot of detail on this ticket, but if you just want to know when it's all done, then wait for the "status" field of the ticket to be set to "closed". This is a big feature so I expect we'll also post on the SF blog when this is all done and ready to be used.

Created #407: [#3154] ForgeBlog API followup (1cp)

Closed #393. je/42cc_3154

Closed #390. je/42cc_3154

Closed #407. je/42cc_3154_blog_api (forced update)

• status: in-progress --> code-review

Closed #395. je/42cc_3154

Merged je/42cc_3154_blog_api

• summary: Users need a way to backup allura data [idt 824] --> Users need a way to backup allura data [idt 824] NEEDS INI

• QA: Dave Brondsema

I've made a few commits myself and rebased and resolved a conflict. That is all on db/3154 so please review those commits (give me feedback if anything's not good) and use that as a starting point for the followup work.

Overall this is working well and I like how it came together. Here's some areas to make it better before we merge:

• Add an ini option to control whether project export is enabled, which would control whether the sidebar entry is shown and admin page is accessible. The task should still be runnable so that it could be submitted at /nf/admin/task_manager/new
• The tmpdir/tmp_dir is constructed in several places by just adding the project shortname. The tmp directory should be configurable (e.g. add a bulk_export_tmp_path config). Then copy that to a tmp file created with tempfile.mkstemp in the final dir so its unique & secure (in case end-users can write to this dir too, we don't want them to cause problems with malicious symlinks etc). Then lastly, do an os.rename (which is atomic and thus also safe) from that mkstemp filename to a final filename which also includes the timestamp. Actually, for maximum flexibility with the final filaname, lets expand bulk_export_path to include the file name too. E.g. /tmp/bulk_export/{nbhd}/{project}/allura-backup-{date:%Y-%m-%d-%H%M%S}.zip (pass date=datetime.utcnow() into the format call).
• I got errors on exporting from u/admin1 since shortname has "/" in it. Should be able to drop u/ since nbhd has it. I also get errors on a subproject or neighborhood. Not sure what the best approach is for that. (Don't necessarily need to support neighborhoods, but should support sub-projects and user-projects).
• The bulk_export task needs to take a neighborhood and use that in the Project query, since the same shortname may be present in 2 different neighborhoods.
• We want to keep Allura and its test suites with minimal dependencies on other tools. So instead of TestExport using tickets & shorturl, can you use just wiki (which is already needed by core Allura)? You can use the Search app to test that it doesn't show up. exportable_tools_for can pass include_hidden=True to ordered_mounts so that hidden tools theoretically can export if they want. (I think it'd be good to do anyway).
• Continuing with the idea of hidden apps doing an export, the project metadata could be an export from the Admin app. Then it'd show up as a checkbox on the form, and within bulk_export() task it'll be handled in the main tools loop (no extra export afterwards needed).
• I'm glad to see the docstring on Application.bulk_export. Can you add a line in the main Application docstring for the 'exportable' var too?
• Remove the try/catch surrounding zip_and_cleanup. If that fails, we need to let it fail - there won't be a valid zip file for the user to get.
• Remove the sort by timestamp/mod_date in all the bulk_export queries and Discussion.__json__. It's not really necessary and if there's not an index on (app_config_id, mod_date) for example and the result is very large, then mongo can error on the sort.

• status: code-review --> in-progress

Created

• 408 [#3154] clean up export tests (2cp)
• 409 [#3154] neighboorhoods fixes for export (3cp)
• 410 [#3154] tmp file changes in export (2cp)
• 411 [#3154] INI option and small additions for export (3cp)

408 closed, branch mi/42cc_3154

closed #409 neighboorhoods fixes for export, branch mi/42cc_3154

had some merging issues, please check if merge is correct

closed #410 tmp file changes in export, branch mi/42cc_3154

Not sure if that's what you mean though.

closed #411 INI option and smal additions for export, branch mi/42cc_3154

• status: in-progress --> code-review