Well-designed software suites should not only be correct, they should be able to demonstrate their own correctness. This is why the new 2.10 release of reposurgeon features a new tool called ‘repodiffer’. And yes, that is what it sounds like – a diff tool that operates not on files but entire repository histories. You get a report on which revisions are identical, which are different, and in the latter case where the differences are, down to which files don’t match. Commits to be paired are matched by committer and commit date. Like reposurgeon, it will work on any version-control system that can emit a fast-import stream.
If you tried running repodiffer on two repositories for different projects the output would be noise and coincidences. What it’s really useful for is comparing two different attempts to lift a repository. Don’t trust reposurgeon? Fine – lift your repo twice, once with git-svn or whatever tool strikes your fancy, then run repodiffer to see the differences. All the differences, not just those in the master tip state. I’ve already found one bug in git-svn this way.
There are few other new goodies, like automatic translation of .cvsignore to .gitignore files (trivial, really – the syntax is upward-compatible). Also, translations from Subversion now emulate Subversion’s default ignore-pattern behavior.
Also note the new web page comparing reposurgeon to other translation tools. To be extended…
Does reposurgeon have an equivalent operation to git-svn rebase? Or does it lend itself better to one-time-only conversions?
>Does reposurgeon have an equivalent operation to git-svn rebase? Or does it lend itself better to one-time-only conversions?
There’s no rebase. It’s for amplifying the human judgment required to do really high-quality one-time conversions.
@esr, note that cvs2svn includes tool to do cvs2git translation: http://cvs2svn.tigris.org/cvs2git.html
Most of heavy lifting of git-cvsimport is done by cvsps.
There are also other tools mentioned on https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools#Interaction_with_other_Revision_Control_Systems
>@esr, note that cvs2svn includes tool to do cvs2git translation: http://cvs2svn.tigris.org/cvs2git.html
I’m aware of this. But I don’t trust the cvs2svn codebase it’s derived from – I’ve seen way too many weird artifacts and crud generated by it in the older parts of Subversion repositories.
>Most of heavy lifting of git-cvsimport is done by cvsps
Yes. I’ve actually submitted a patch to the cvsps maintainer that does an almost complete job of making it emit a fast-import stream directly – the only thing it’s missing is making parent pointers. I think I can finish that job if I can get him to answer a few basic questions.
>There are also other tools mentioned on https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools#Interaction_with_other_Revision_Control_Systems
Not only do I know that page, I’ve done a fairly large percentage of the recent updates on it. :-)
Oh no! The sturgeon ate my hometown!
>Oh no! The sturgeon ate my hometown!
That title was, of course, a reference to the classic The Creature That Ate Sheboygan.
@esr:
This. But I haven’t used it in awhile, and all of the artifacts it did for me were relatively benign. There wasn’t any missing information — just added cruft (files marked as changed during a revision when nothing was different, etc.)
Interesting that you mention svn2git issues.
We’ve been working on migrating Gentoo from cvs to git and are still struggling to get a clean migration with full history. Right now the two still-open issues I’m aware of are that unicode in commit messages sometimes gets mangled, and that cvs keywords get modified during the conversion (the latter tends to show up a lot in things like patch files that happen to contain keywords from the files they are patching).
I tossed together some code to do some of my own comparisons (not really suitable for general use – assumes no branching which is true in our case). I might try using your new comparision tool to see if it finds anything I have yet to spot, and I’ll see if reposturgeon does any better, as verified by both sets of tools.
>We’ve been working on migrating Gentoo from cvs to git and are still struggling to get a clean migration with full history. Right now the two still-open issues I’m aware of are that unicode in commit messages sometimes gets mangled, and that cvs keywords get modified during the conversion (the latter tends to show up a lot in things like patch files that happen to contain keywords from the files they are patching).
Ugh. That’s nasty. What conversion tool is showing these problems?
Congrats! That looks like an excellent tool. One nit in the features doc: I think “file-version numbers like 1.2 in Subversion repos that had a former life in Subversion” should be “file-version numbers like 1.2 in Subversion repos that had a former life in CVS”.
>Ugh. That’s nasty. What conversion tool is showing these problems?
We’re using git2svn, combined with some scripts to fix some of the issues we’ve run into with it (including the keyword issue – but the script only handles the keywords Gentoo puts in the files – not ones that have gotten into things like patches/etc because upstream uses them). The scripts also do things like converting cvs usernames to Name in the author field using a lookup table.
When I get back from my holiday travels I’ll probably point your tool at a cvs snapshot and see how it does. We’re fairly eager to get off of cvs and we’re tantalizingly close to being able to do it.
>When I get back from my holiday travels I’ll probably point your tool at a cvs snapshot and see how it does.
reposurgeon doesn’t read CVS repos yet. It is possible this might change shortly. Yes, please email me when you’re in work mode again, I want more detail on your bug report and might be able to help you develop a workaround. If the stars align, you might get to be the beta test for CVS support – which might not sound like a good thing until you realize that means your conversion will have my full attention.
@Glen Holmer, mine too!
This may sound like an odd request, but can the features page’s source be included in the reposurgeon archive? I’m just thinking it may be nice to install it alongside ;)
How does the diff tool work? Because unless the code it uses to read the repositories is implemented independently of the code that reposurgeon itself does, it sounds like there’s a bit of circularity in relying on it – any bug that affects correctness is also going to cause the diff tool to be incorrect in precisely a way that will cause it to claim it was correct.
I think I misread something; disregard the above.