Revenge of the reposturgeon!

Reposurgeon 1.5 is out. This is a major release based on experience gained converting the roundup repository.

The main new feature is code to help in fixing up fossil CVS and SVN commit references, turning them into action stamps. I had to think about the design carefully here, because the task combines a front end that humans do better than machines with a back end that machines do much better than humans.

The problem: You’ve lifted your Subversion repo to git with git-svn or some similar tool. But the comments still have references in them that look like, say, “r2317″. You want to replace these to point to the corresponding changesets in the git repo, but there may be lots of them, and even if patching each one by hand weren’t a huge pain in the ass it’s a fiddly job at which your error rate is likely to be significant.

The part of this task humans are good at is recognizing from context all the random forms a reference can take. There’s the canonical “r2317″, “SVN#2317″, “commit 2317″, “rev 2317″, and other variants. Machines aren’t good at reduciing ambiguity; I passed on solving the strong-AI problem and designed for a workflow in which the human first replaces all these variants with a uniform machine-parseable cookie – to wit, “[[SVN:2317]].

Then the machine does what it’s good at, which is crunching through the logic to replace that cookie with an action-stamp pointing at the same changeset. A human doing this by hand would be prone to boredom-induced detail errors and typos.

How does it get the mapping from revision number to time!committer? One way is if the repo comments contain metadata put in changeset comments specifically to support this, which git-svn does. (reposurgeon also has a command to strip out all that metadata when you’re done with it.)

Or – and here’s the tricky part – reposurgeon will sometimes be able to mine that information out of CVS keyword expansions. So, to take a real-world example from the roundup repo, let’s say a blob in the repo has this string in it:


$Id: ru.po,v 1.6 2004-07-03 13:51:03 a1s Exp $

Then reposurgeon knows that the reference cookie [[CVS:ru.po:1.6]] should be replaced by an action-stamp pointing at whatever commit this blob is attached to (if there are two or more such commits it just grabs the first; this is a bug and I’ll fix it in 1.6).

UPDATE: Duh…I had already fixed it to throw an error in that case! What I need to do is try to correctly handle cases where all but one of the possibilities can be discarded because they refer to the wrong branches.

Fear the reposturgeon!

UPDATE: Mike Swanson pointed out a Python 3 compatibility problem, so I snap-released 1.6 about a day later.

12 thoughts on “Revenge of the reposturgeon!

  1. Missing trailing ‘n’ in roundup.sourceforge.net, both the tagged text and the href

  2. Correktion; the tagged text is fine, just the href needs adjusted.

  3. Is it reposurgeon or reposturgeon? the former makes it sound like a cutting edge elite git tool. The latter makes it sound like a fish.

  4. >Resposturgeon, reposurgeon, which is it?

    Reposugeon is the program. The reposturgeon is the project’s emblem – a blue Atlantic sturgeon.

  5. >Resposturgeon, reposurgeon, which is it?

    It’s a state secret. (See Janet Kagan’s Hellspark for explanation.)

  6. Again I think action stamps that incorporate metadata, conflate semantics that are orthogonal in version control. Can’t one identify a specific set of changes in DVCS without referring to the author of the change? If not, then how does the author enable you to find changes. Doesn’t make any sense to me that an author stamp aids in any way, am I missing something? Please correct my ignorance.

  7. You’ve introduced a line that’s again incompatible with Python 3.2. On another note, the hash-bang opening should not need the -3 option to the Python interpreter; the -3 option only warns about Python3 incompatibilities and doesn’t actually change how the interpreter operates, additionally, the option doesn’t exist on Python 3.x proper and makes me have to edit that line just to run it (my distro defaults to Python 3)!

    --- reposurgeon.orig	2011-11-02 16:14:11.000000000 -0700
    +++ reposurgeon	2011-11-02 22:38:09.666219943 -0700
    @@ -1,10 +1,10 @@
    -#!/usr/bin/python -3
    +#!/usr/bin/python
     #
     # reposurgeon - a repository surgeon.
     #
     # By ESR, October 2010.  BSD terms apply.
     #
    -# Requires Python 2.7.2 or newer, running in Python 3 mode.
    +# Requires Python 2.7.2 or newer.
     #
     import sys, os, cmd, tempfile, subprocess
     import readline, time, calendar, signal, shutil, copy, shlex
    @@ -1909,7 +1909,7 @@
                 if sys.version_info.major >= 3:
                     raise extype(value).with_traceback(traceback)
                 else:
    -                raise extype, value, traceback
    +                raise extype(value).with_traceback(traceback)
             if self.fp.close() is not None:
                 legend = "%s%s returned error" % (self.command, self.legend)
                 if extype == Fatal:

    >Resposturgeon, reposurgeon, which is it?

    Reposugeon is the program. The reposturgeon is the project’s emblem – a blue Atlantic sturgeon.

    It’s increasingly sounding like this program needs a FAQ. The first question being this very thing. The second one, perhaps I’ll suggest “Are you getting tired of answering that question yet?” :P

    ESR says: Fixed the code formatting. Use “<pre lang=”Python”> …<pre>”.

  8. Well, that diff was eaten by the blog, but you can see the change anyway, I’m sure.

  9. >You’ve introduced a line that’s again incompatible with Python 3.2.

    Look at the context:

                # This is what we want, but it's only in true Python 3.x
                if sys.version_info.major >= 3:
                    raise extype(value).with_traceback(traceback)
                else:
                    raise extype, value, traceback

    I found that without that conditional 2.7.2 would barf on the traceback() method call. Is this code throwing an actual syntax error for you? If so I guess we need to wrap that else body in try: … except SyntaxError.

  10. Ah, I didn’t make a big attempt to notice the context last night. Yes, it’s throwing an actual syntax error on Python 3.2, so it seems the python interpreter isn’t caring much about context, either.

  11. >Yes, it’s throwing an actual syntax error on Python 3.2, so it seems the python interpreter isn’t caring much about context, either.

    Easily fixed. 1.6 will be more robust.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>