Here’s a late New Year’s gift for all you repository-editing fiends out there: the long-awaited and perhaps long-dreaded reposurgeon 3.0.
In Heads up: the reposturgeon is mutating! I described the downside of a strategy of incremental small language changes aimed at preserving compatibility: you can wind up trapped by suboptimal early decisions. Sometimes, you have to bust out and do the big redesign, which I did and why there’s a bump in the major version number (the last time that happened was when reposurgeon got the ability to read Subversion dump files directly).
The biggest change is that the command language syntax has mutated from VSO to SVO. What? You’re not up on your comparative linguistic morphology and gave no idea what I’m talking about? That’s Verb-Subject-Object to Subject-Verb-Object.
My first gift of the new year. Read it here.
Not long ago I pulled the plug on one of the two CVS export utilities I was maintaining. One consequence of this is that I decided I needed to get the other one out of beta and into a state I would be willing to ship as 1.0.
And lo, it has come to pass. I just shipped cvs-fast-export 1.0. It has been well field-tested; a couple of weeks ago I used it to rescue the history of Gnu Troff.
There are several CVS exporters out there that suck pretty badly. (To be fair, the perversity of CVS is such that doing an even half-decent job of lifting CVS histories into a modern version-control system is quite difficult.) Now that this one is shipped I know of exactly two that don’t suck. The other one is Michael Haggerty’s cvs2git, which I’m working with him on improving.
Tradeoffs: cvs2git is slow and a bit clunky to use (I’m improving the latter but can’t fix the former). cvs-fast-export is blazingly fast (like, 3.7K commits a minute) but has a hard repository-size limit – above it you run out of core and the OS reaps the process in mid-flight. (Very few projects will hit this limit.)
For each tool there are weird CVS edge cases that it gets wrong. The sets of edge cases are different. cvs2git’s may be smaller, but I’m not sure of that; we haven’t set up head-to-head testing yet. Most projects will not trip over either set of problems.
cvs-fast-export is better documented, especially around error conditions.
Help stamp out CVS in our lifetime!
There’s an ancient Unix maxim to the effect that a tool that gets 85% of your job done now is preferable to one that gets 100% done never. Sometimes chasing corner cases is more work than the problem really justifies.
In today’s dharma lesson, I shall illustrate this principle with a real-world and useful example.
If you’re a regular at A&D or on my G+ feed, and even possibly if you aren’t, you’ll have noticed that I ship an awful lot of code. I do get questions about this; between GPSD, reposurgeon, giflib, doclifter, and bimpty-bump other projects it is reasonable that other hackers sometimes wonder how I do it.
Here’s part of my answer: be fanatical about automating away every part of your workflow that you can. Every second you don’t spend on mechanical routines is a second you get to use as creative time.
Soon, after an 11-year alpha period, I’m going to ship version 1.0 if one of my main automation tools. This thing would be my secret weapon if I had secrets. The story of how it came to be, and why it took 11 years to mature, should be interesting to other hackers on several different levels.
Though there haven’t been any huge dramatic improvements since Subversion analysis got good enough to use even on horribly gnarly repositories, reposurgeon continues to quietly get better and faster. I shipped 2.43 a few minutes ago.
Blogging has been light recently because I’ve been working very hard on a major GPSD release, which I just shipped. This is mostly new features, not bugfixes, and it’s probably the most new code we’ve shipped in one release since about 2009.
OK, this is embarrassing. I got a Mercurial command wrong in 1.11 – thanks to A&D regular Jay Maynard for pointing out the error. But I landed another minor feature. So there.
Yes, I know I shipped a point release of this yesterday. Then some new features landed as a direct result of that release. Mainly, smart defaults for user name and ID, and repo support for hg/bzr/svn. Also (cosmetic addition) the project now has a logo.
coverity-submit automates the process of running the Coverity static checker’s front-end tools and shipping the results to their public server for analysis.
One bug fix, two minor features. The build-version (-b) and description (-t) options now have sensible defaults. When run from a repository, the default for -b is the commit ID of the head revision. The default for -t is an ISO8601 release timestamp.
Actually, the build-version default presently only works in a git repo. I’ll cheerfully take patches that support other version-control systems.
There’s a genre of computer games called 4X (explore/expand/exploit/exterminate). well-known examples of which include the Civilization series and Master of Orion.
Ever wonder what the ur-progenitor of this genre was, the game at the root of 4X in the way Colossal Cave Adventure created the genre of dungeon-crawl games? It was Walter Bright’s game “Empire” from the early 1970s. You can read about it at his page on Classic Empire.
Since 1994 I’ve maintained an early Empire workalike written by Chuck Simmons in 1987 to run under the now-extinct VMS operating system; it was ported to Unix immediately, and remains to my best of knowledge the only open-source version or variant of Empire available.
Walter Bright does not acknowledge this version’s existence on his Empire page, which is fair because he didn’t write it and probably doesn’t consider it to be “Empire” at all. But it is close in gameplay and style to the earliest of Bright’s versions, except for being able to display its crude character-cell maps in color (I added that back when color terminals were cutting-edge technology).
If you love Civ or MOO, try this out for a look at what the computer 4X game was like before pixel graphics. The display and command interface are primitive by today’s standards, but the AI and general gameplay have held up surprisingly well. It’s instructive to see how many of the core tropes of later 4X games are already present in this one.
You can get version 1.10 of VMS-Empire here.
I released reposurgeon 2.20 today, with various minor improvements in the graph command and the behavior of repodiffer. Which, mainly, gives me all the excuse I need for this:
Image composed using Pulp-O-Mizer.
For those of you who have been following my work on tools to muck out the swamp that is CVS, a new shovel: cvs-fast-export. Note, this is an alpha release for testing purposes; double-check the quality of any conversions you do with it carefully.
The 2.13 release of reposurgeon has in it maybe the coolest new feature I’ve ever implemented in five minutes of programming – graphical visualization of a repository’s commit DAG.
Today’s very special non-world-ending software release, triggered if not originated from here at Eric Conspiracy Secret Laboratories is the amazing Zen simulation, Robot Finds Kitten. I bow in respect before Leonard Richardson and the other giants of kitten-finding history and am humbly proud to be counted among the select few who have contributed to this monumental, er, monument.
Get yer hot fresh tarball right here. It will improve your sex life, clear up your financial problems, cure your acne, and make you as a god among men. Would I lie?
Well, er, no. Actually, it attacks CVS.
Yes, that’s right, the just-shipped reposurgeon 2.11 can now read – though not write – CVS repositories. To get it to do this, I got my lunch-hooks on a relatively old program called cvsps that assembles changesets out of CVS repositories for human inspection. I gave it a –fast-export reporting mode that emits a fast-import stream instead, so now CVS has a universal exporter that will talk to any version-control system that speaks import streams. Oh, yes, and I’m maintaining cvsps now too – applause to David Mansfield, who both did a very good job on that code and sees clearly that its original use case is obsolete and –fast-export is a better way forward.
Two substantial releases of different projects in a day is a fast pace even for me. cvsps-3.0 and reposurgeon-2.11; two great tastes that taste great together.
Fear the reposturgeon!
Well-designed software suites should not only be correct, they should be able to demonstrate their own correctness. This is why the new 2.10 release of reposurgeon features a new tool called ‘repodiffer’. And yes, that is what it sounds like – a diff tool that operates not on files but entire repository histories. You get a report on which revisions are identical, which are different, and in the latter case where the differences are, down to which files don’t match. Commits to be paired are matched by committer and commit date. Like reposurgeon, it will work on any version-control system that can emit a fast-import stream.
I had said I wasn’t going to do it, but…I experimented, and it turned out to be easier than I thought. Release 2.7 of reposurgeon writes (as well as reading) Subversion repositories. With the untested support for darcs, which should work exactly as well as darcs fast-export and fast-import do, this now brings the set of fully-supported version-control systems to git, hg, bzr, svn, and darcs; reposurgeon can be used for repository surgery and interconversion on any of these.
I shipped reposurgeon 2.0 a few days ago with the Subversion support feature-complete, and a 2.1 minor bugfix release this morning. My previous release announcement was somewhat rushed, so here is a more detailed one explaining why anybody contemplating moving up from Subversion should care.
To go with this, there is a new version of my DVCS Migration HOWTO.
I’ve just shipped reposurgeon 2.0, a power tool for editing and interconverting version-control repositories. This is a major release, adding the capability to read Subversion dump files directly.