#5076 svn hooks need conversion

v1.0.0
closed
Cory Johns
support (446) p2 (24) migration (63) sf-4 (350)
General
Dave Brondsema
2015-08-20
2012-10-05
Chris Tsai
No

Due to differences in server configs, migrated svn hooks don't seem to work.

Errors with pre-commit hooks can cause commits to get blocked, ref: [forge:site-support:#874]

after upgrading the megameklab project, we can't checking anything into SVN now. We get the following error:
/svn/p/megameklab/code/hooks/pre-commit: line 3: /var/local/mastertree/host/sfp-svn/hook-scripts/check-case-insensitive.py: No such file or directory

I imagine other hooks would fail similarly. We should probably also remove any unnecessary hooks and update the data on other hooks. For example, the user in [forge:site-support:#862] had the following:

-bash-3.2$ pwd
/home/svn/p/callflow/code/hooks

-bash-3.2$ cat post-commit     
#!/bin/bash
# The following is required for site integration, do not remove/modify.
# Place user hook code in post-commit-user and it will be called from here.
curl -s http://sourceforge.net/auth/refresh_repo/p/callflow/code/

DIR="$(dirname "${BASH_SOURCE[0]}")"
if [ -x $DIR/post-commit-user ]; then  exec $DIR/post-commit-user "$@"

-bash-3.2$ cat post-commit-user 
#!/bin/sh
# THIS FILE IS AUTOGENERATED - DO NOT EDIT
/var/local/mastertree/host/sfp-svn/hook-scripts/sf-svn-stats-hook.py $*
/var/local/mastertree/host/sfp-svn/hook-scripts/keepsake -p "$1"
/var/local/mastertree/host/sfp-svn/hook-scripts/svnnotify --repos-path "$1" --revision "$2" --to "callflow-svn@lists.sourceforge.net" --subject-prefix "SF.net SVN: callflow:" --subject-cx --no-first-line --with-diff --viewcvs-url "http://callflow.svn.sourceforge.net/callflow/?rev=%s&view=rev" --user-domain "users.sourceforge.net" --footer "This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site." -l /usr/bin/svnlook --max-diff-length 100000
exit 0

  • We should probably first remove the "AUTOGENERATED" link, since that's no longer true.

  • I think we can probably just remove the sf classic specific hooks (sf-svn-stats-hook.py and keepsake).

  • For svnnotify, we should fix the path (iirc, it's in the standard location /usr/bin/svnnotify/ though just svnnotify works), and update the url to the code browser (ie., the --viewcvs-url option)

We have a few other pre-configured hooks users can add, but we didn't allow custom hooks for svn repos on classic, so it should be a finite list.

Discussion

1 2 > >> (Page 1 of 2)

  • Dave Brondsema

    Dave Brondsema - 2012-10-05
    • size: --> 1
     

  • Anonymous - 2012-10-16

    Originally by: maddes

    Hi there,

    I can not find any svnnotify on the server.
    Here's how I searched:

    -bash-3.2$ hostname
shell-24006
-bash-3.2$ pwd
/home/svn/p/regshot/code/hooks
-bash-3.2$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin
-bash-3.2$ which svnnotify
/usr/bin/which: no svnnotify in (/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin)
-bash-3.2$ find / -name "svnnotify"
-bash-3.2$

    Similar issue for check-case-insensitive.py and check-mime-type.pl in the converted pre-commit hook:

    -bash-3.2$ find / -name "check-case-insensitive.py"
-bash-3.2$
-bash-3.2$ find / -name "check-mime-type.pl"
/usr/share/doc/subversion-1.6.17/contrib/hook-scripts/check-mime-type.pl

    What are the correct pathes for:
    "svnnotify" in post-commit-user
     "check-case-insensitive.py" in pre-commit
    * "check-mime-type.pl" in pre-commit

    Or do have I to place them myself somewhere and call from there? If so, what is the recommended path?

    Maddes

    Current content of pre-commit after upgrade to Allura:

    -bash-3.2$ cat pre-commit
#!/bin/sh -e
# THIS FILE IS AUTOGENERATED - DO NOT EDIT
/var/local/mastertree/host/sfp-svn/hook-scripts/check-case-insensitive.py "$1" "$2"
/var/local/mastertree/host/sfp-svn/hook-scripts/check-mime-type.pl "$1" "$2"
exit 0
     

  • Chris Tsai - 2012-10-16

    Hello Maddes,

    The server for shell sessions is not the same as the server that runs commit hooks. On the servers that run the scripts, svnnotify is in the standard /usr/bin/svnnotify location (though just using svnnotify without the full path also works)

    As for the other scripts, I'm not certain that those are installed on that server yet, but I expect the work on this ticket to root those out and get them added properly.

    Regards,
    Chris Tsai, SourceForge.net Support

     

  • Anonymous - 2012-10-16

    Originally by: maddes

    Thanks for the quick info.
    Changed my post-commit-user file accordingly.

    Will track this ticket for "check-case-insensitive.py" and "check-mime-type.pl" to arrive. The first one is critical for projects with mixed development platforms (Windows/*nixes).

    Normally these are available in /usr/share/subversion/hook-scripts/ and must be executable.

    Maddes

     

  • Anonymous - 2012-10-16

    Originally by: maddes

    The scripts are available on the code server (found via pre-commit hook) but not at their usual place.
    Normally these are in /usr/share/subversion/hook-scripts/ and must be executable.

    -rwxr-xr-x 1 root root 6527 Oct  5  2007   /usr/share/doc/subversion-1.6.17/contrib/hook-scripts/check-mime-type.pl  
-rwxr-xr-x 1 root root 6527 Jun  1  2011   /usr/lib64/subversion/contrib/hook-scripts/check-mime-type.pl  
-rwxr-xr-x 1 root root 4794 Oct 17  2007   /usr/share/doc/subversion-1.6.17/contrib/hook-scripts/case-insensitive.py  
-rwxr-xr-x 1 root root 4794 Jun  1  2011   /usr/lib64/subversion/contrib/hook-scripts/case-insensitive.py

    Maddes

     

  • Anonymous - 2012-10-16

    Originally by: maddes

    /usr/lib64/subversion/contrib/hook-scripts/check-mime-type.pl:
    Is working for me. Still unusual place for hook script.

    /usr/lib64/subversion/contrib/hook-scripts/case-insensitive.py:
    The Python script is not working.

    Traceback (most recent call last):  
  File "/usr/lib64/subversion/contrib/hook-scripts/case-insensitive.py", line  
 64, in ?  
    from svn import repos, fs  
ImportError: No module named svn
     

  • Cory Johns - 2012-10-16
    • status: open --> in-progress
    • assigned_to: Cory Johns
     

  • Cory Johns - 2012-10-17
    • status: in-progress --> code-review
    • qa: Dave Brondsema⚛
     

  • Cory Johns - 2012-10-17

    allura:cj/5076

     

  • Dave Brondsema

    Dave Brondsema - 2012-10-18

    Can we put this in forge-classic? Monkey-patching even, if needed. This is very SF-specific.

    We need to handle these two hooks also:

    /var/local/mastertree/host/sfp-svn/hook-scripts/check-case-insensitive.py "$1" "$2"
/var/local/mastertree/host/sfp-svn/hook-scripts/check-mime-type.pl "$1" "$2"

    There's an extra slash in --revisionURI=http://localhost//p...

    We should check with siteops about what environment these run in so that we can confirm the paths (e.g. the new paths in this code branch aren't present for me on the hosts I expected, but neither was /usr/bin/svnnotify so I'm clearly looking on the wrong host) and confirm functionality (e.g. ImportError mentioned above).

    Should we do manual fixups for the existing repos? Not sure how many there are, but that might be easier than scripting it. Either way, we should do that in 'validation' stage or make a separate ticket.

     

  • Dave Brondsema

    Dave Brondsema - 2012-10-18
    • status: code-review --> in-progress
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-19
    • labels: support, p2 --> support, p2, migration
     

  • Anonymous - 2012-10-25

    Originally by: peterbecker

    I'm experiencing this problem with the 'toscanaj' project and the check-case-insensitive.py script. How do I work around this issue? I'm reasonably happy to drop the script completely, but I don't know how.

    Here's the exact error if needed:

      svn: E165001: Commit failed (details follow):
  svn: E165001: Commit blocked by pre-commit hook (exit code 1) with output:
  /svn/p/toscanaj/code/hooks/pre-commit: line 3: /var/local/mastertree/host/sfp-svn/hook-scripts/check-case-insensitive.py: No such file or directory
     

  • Anonymous - 2012-10-25

    Originally by: *anonymous

    Sorry -- I just realised the issue linked in the description answers my question. Fixed.

     

  • Cory Johns - 2012-10-25

    I believe you should be able to use the shell service to access the hooks directly at /home/svn/p/<project>/<mount>/hooks.

    Unfortunately, we do not yet have the hooks installed globally, but you should be able to put them in the hooks directory and call them from there.

     

  • Cory Johns - 2012-10-25

    I added the info to this thread as well in my reply above, as well as noting that, in addition to simply commenting out the offending hook call, you can also install it directly in your project's hook directory until we resolve the issue globally.

     

  • David Burley - 2012-10-26

    /var/local/mastertree/host/sfp-svn/hook-scripts/sf-svn-stats-hook.py $*
    /var/local/mastertree/host/sfp-svn/hook-scripts/keepsake -p "$1"

    ^-- those should be removed from the project's hooks anyways, they are classic forge specific.

     

  • David Burley - 2012-10-26

    svnnotify is present in /usr/bin/svnnotify, so the path just should need changed to reference it.

     

  • David Burley - 2012-10-26

    I have created all the scripts in the allura SVN environment, in the same place they were before (well, symlinked to the new location). I replaced the non-needed scripts with empty ones that just return to clear up errors.

     

  • David Burley - 2012-10-26

    note: I haven't tested them, however...

     

  • Cory Johns - 2012-10-26

    allura:cj/5076
    forge-classic:cj/5076

    Moved hook copying / processing code to forge-classic, added missing munging, fixed a couple of small issues, and generally cleaned it up.

    The production setup is already in place, and the default value for the new config option is correct, so this should be GTG out of the box.

     

  • Cory Johns - 2012-10-26
    • status: in-progress --> code-review
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-29

    Code and tests are looking good, but got an error when migrating a test project. As you can see in the pdb shell, the src & dst args have the right values. Also /git/p/testscm/part1.git/hooks/post-receive.sample actually exists when I check after this error. Before running the migration, I did run rm -rf /{hg,svn,git}/p/testscm to make sure I had a fresh clean start.

      File "/var/local/forge-classic/scripts/migrate_sfx_project.py", line 18, in main
    migrator.migrate()
  File "/home/dbrondsema/dbrondsema-7024/forge-classic/sfx/lib/migrate.py", line 220, in migrate
    self.migrate_repos()
  File "/home/dbrondsema/dbrondsema-7024/forge-classic/sfx/lib/migrate.py", line 299, in migrate_repos
    self.tool_position += SCMMigrator(self.allura_project).migrate(repos, self.tool_position)
  File "/home/dbrondsema/dbrondsema-7024/forge-classic/sfx/lib/migrate.py", line 995, in migrate
    self._post_process_repo(repo, app)
  File "/home/dbrondsema/dbrondsema-7024/forge-classic/sfx/lib/migrate.py", line 1134, in _post_process_repo
    dispatch[repo['tool_name']](repo, app)
  File "/home/dbrondsema/dbrondsema-7024/forge-classic/sfx/lib/migrate.py", line 1144, in _post_process_git_repo
    shutil.copy2(hook, target)
  File "/usr/lib64/python2.7/shutil.py", line 128, in copy2
    copyfile(src, dst)
  File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: u'/git/p/testscm/part1.git/hooks/post-receive.sample'
> /usr/lib64/python2.7/shutil.py(83)copyfile()
-> with open(dst, 'wb') as fdst:
(Pdb) a
src = /nfs/classic/sf-git/t/te/testscm/part1/hooks/post-receive.sample
dst = /git/p/testscm/part1.git/hooks/post-receive.sample
     

  • Dave Brondsema

    Dave Brondsema - 2012-10-29
    • status: code-review --> in-progress
     
1 2 > >> (Page 1 of 2)

Log in to post a comment.