#3154 Users need a way to backup allura data [idt 824] NEEDS INI

v1.0.1
closed
nobody
General
nobody
2015-08-20
2011-11-02
Chris Tsai
No

https://sourceforge.net/apps/ideatorrent/sourceforge/ideatorrent/idea/824/

SF clearly says that the backup of the project data is left to project admins:

https://sourceforge.net/apps/trac/sourceforge/wiki/Backup%20your%20data

Fine.

However, there is currently no way to backup the SF2.0 tools, for instance the data from the ticket tool.

Backups would be the first step, then a way to restore (though we never offered that part in SF classic to my knowledge)?

Related

Tickets: #3154
Tickets: #4213
Tickets: #4527

Discussion

<< < 1 2 3 > >> (Page 2 of 3)
  • Igor Bondarenko - 2013-07-15

    Closed #392. je/42cc_3154_link_api

    Since this ticket is about ForgeLink API and not directly related to the export, I've created a separate branch for that. You may wish to review this changes separately and merge them to master before full export is ready.

    Documentation for https://sourceforge.net/p/forge/documentation/Allura%20API/

    Endpoint: _prefix/_
    
    * GET _prefix/_ - returns the url
    * POST _prefix/_ - updates the url
         * parameter url: the url (for example "http://google.com")
    
     
  • Igor Bondarenko - 2013-07-17

    Closed #391. je/42cc_3154_blog_api

    Docs:

    Endpoint: _prefix/_
    
    * GET _prefix/_ - returns a list of posts, including title and API url
    * GET _prefix/year/month/title_ - returns a JSON representation of a post
    * POST _prefix/_ - creates a post
        * parameter title - the title of the post
        * parameter text - the text of the post
        * parameter labels - labels of the post
        * parameter state - 'draft' or 'published'
    * POST _prefix/year/month/title_ - updates an existing post
        * parameter parameters: 
        * parameter title - the title of the post
        * parameter text - the text of the post
        * parameter labels - labels of the post
        * parameter state - 'draft' or 'published'
    
     
  • Dave Brondsema

    Dave Brondsema - 2013-07-19

    merged je/42cc_3154_link_api

    FYI, tested with curl-oauth:

    HOST=sf-dbrondsema-1015.sb.sf.net
    curl-oauth --domain $HOST --request-token --request-token-url https://$HOST/rest/oauth/request_token --consumer-key ABC --consumer-secret XYZ --request-token-authorize-url https://$HOST/rest/oauth/authorize --request-token-access-token-url https://$HOST/rest/oauth/access_token
    curl-oauth --domain sf-dbrondsema-1015.sb.sf.net https://$HOST/rest/p/testit/docs/ -X POST -d 'url=http://slashdot.org'
    
     
  • Igor Bondarenko - 2013-07-23

    Closed #394. je/42cc_3154 (forced update)

     
  • Dave Brondsema

    Dave Brondsema - 2013-07-23

    For the blog API:

    • can you add the author username to BlogPost.__json__ ?
    • when creating a new post, it'd be slightly better to return a 201 instead of 200. And include a Location: header in the response, pointing to the URL of the new post
    • There seems to be some code duplication between the regular controllers and the API controllers. For example, creating a new BlogPost involves setting several fields, calling make_slug() making a Thread and committing (side not: post.viewable_by = ['all'] is unique and I'm not sure why). Can you check for better code re-use on post, update, list and viewing? It seems to me that we could refactor and get some cleaner code that uses the same methods. Or perhaps even one controller could just call the other controller with no new methods/helpers needed. And for listing posts, it wouldn't be bad if the API had pagination like the regular controller too (I don't remember offhand if the same pagination classes are used for API pagination though)
     
  • Igor Bondarenko - 2013-07-24

    Closed #389. je/42cc_3154

     
  • Anonymous - 2013-07-24

    Originally by: *anonymous

    Was it resolved ?

    I do not understand the "bla bla bla" which is comming to my e-mails.

    Could you please state clearly can we back up our Tickets ?

    Thank

     

    Last edit: Anonymous 2015-12-14
  • Dave Brondsema

    Dave Brondsema - 2013-07-24

    We've been closing some of our subtasks for this issue. We're tracking a lot of detail on this ticket, but if you just want to know when it's all done, then wait for the "status" field of the ticket to be set to "closed". This is a big feature so I expect we'll also post on the SF blog when this is all done and ready to be used.

     
  • Igor Bondarenko - 2013-07-25

    Created #407: [#3154] ForgeBlog API followup (1cp)

     

    Related

    Tickets: #3154

  • Igor Bondarenko - 2013-07-25

    Closed #393. je/42cc_3154

     
  • Igor Bondarenko - 2013-07-26

    Closed #390. je/42cc_3154

     
  • Igor Bondarenko - 2013-07-30

    Closed #407. je/42cc_3154_blog_api (forced update)

     
  • Igor Bondarenko - 2013-07-30
    • status: in-progress --> code-review
     
  • Igor Bondarenko - 2013-07-30

    Closed #395. je/42cc_3154

     
  • Dave Brondsema

    Dave Brondsema - 2013-07-31

    Merged je/42cc_3154_blog_api

     
  • Dave Brondsema

    Dave Brondsema - 2013-07-31
    • summary: Users need a way to backup allura data [idt 824] --> Users need a way to backup allura data [idt 824] NEEDS INI
     
  • Dave Brondsema

    Dave Brondsema - 2013-07-31
    • QA: Dave Brondsema
     
  • Dave Brondsema

    Dave Brondsema - 2013-08-06

    I've made a few commits myself and rebased and resolved a conflict. That is all on db/3154 so please review those commits (give me feedback if anything's not good) and use that as a starting point for the followup work.

    Overall this is working well and I like how it came together. Here's some areas to make it better before we merge:

    • Add an ini option to control whether project export is enabled, which would control whether the sidebar entry is shown and admin page is accessible. The task should still be runnable so that it could be submitted at /nf/admin/task_manager/new
    • The tmpdir/tmp_dir is constructed in several places by just adding the project shortname. The tmp directory should be configurable (e.g. add a bulk_export_tmp_path config). Then copy that to a tmp file created with tempfile.mkstemp in the final dir so its unique & secure (in case end-users can write to this dir too, we don't want them to cause problems with malicious symlinks etc). Then lastly, do an os.rename (which is atomic and thus also safe) from that mkstemp filename to a final filename which also includes the timestamp. Actually, for maximum flexibility with the final filaname, lets expand bulk_export_path to include the file name too. E.g. /tmp/bulk_export/{nbhd}/{project}/allura-backup-{date:%Y-%m-%d-%H%M%S}.zip (pass date=datetime.utcnow() into the format call).
    • I got errors on exporting from u/admin1 since shortname has "/" in it. Should be able to drop u/ since nbhd has it. I also get errors on a subproject or neighborhood. Not sure what the best approach is for that. (Don't necessarily need to support neighborhoods, but should support sub-projects and user-projects).
    • The bulk_export task needs to take a neighborhood and use that in the Project query, since the same shortname may be present in 2 different neighborhoods.
    • We want to keep Allura and its test suites with minimal dependencies on other tools. So instead of TestExport using tickets & shorturl, can you use just wiki (which is already needed by core Allura)? You can use the Search app to test that it doesn't show up. exportable_tools_for can pass include_hidden=True to ordered_mounts so that hidden tools theoretically can export if they want. (I think it'd be good to do anyway).
    • Continuing with the idea of hidden apps doing an export, the project metadata could be an export from the Admin app. Then it'd show up as a checkbox on the form, and within bulk_export() task it'll be handled in the main tools loop (no extra export afterwards needed).
    • I'm glad to see the docstring on Application.bulk_export. Can you add a line in the main Application docstring for the 'exportable' var too?
    • Remove the try/catch surrounding zip_and_cleanup. If that fails, we need to let it fail - there won't be a valid zip file for the user to get.
    • Remove the sort by timestamp/mod_date in all the bulk_export queries and Discussion.__json__. It's not really necessary and if there's not an index on (app_config_id, mod_date) for example and the result is very large, then mongo can error on the sort.
     
  • Dave Brondsema

    Dave Brondsema - 2013-08-06
    • status: code-review --> in-progress
     
  • Anton Kasyanov - 2013-08-07

    Created

    • 408 [#3154] clean up export tests (2cp)
    • 409 [#3154] neighboorhoods fixes for export (3cp)
    • 410 [#3154] tmp file changes in export (2cp)
    • 411 [#3154] INI option and small additions for export (3cp)
     

    Related

    Tickets: #3154

  • Anton Kasyanov - 2013-08-13

    408 closed, branch mi/42cc_3154

     
  • Anton Kasyanov - 2013-08-13

    closed #409 neighboorhoods fixes for export, branch mi/42cc_3154

    had some merging issues, please check if merge is correct

     
  • Anton Kasyanov - 2013-08-15

    closed #410 tmp file changes in export, branch mi/42cc_3154

    Not sure if that's what you mean though.

     
  • Anton Kasyanov - 2013-08-15

    closed #411 INI option and smal additions for export, branch mi/42cc_3154

     
  • Anton Kasyanov - 2013-08-15
    • status: in-progress --> code-review
     
<< < 1 2 3 > >> (Page 2 of 3)

Log in to post a comment.