Apr 07

Out on the tiles

I’ve been experimenting with tiling window managers recently. I tried out awesome and xmonad, and read documentation on several others including dwm and wmii. The prompt cause is that I’ve been doing a lot of surgery on large repositories recently, and when you get up to 50K commits that’s enough to create serious memory pressure on my 4G of core (don’t laugh, I tend to drive my old hardware until the bolts fall out). A smaller, lighter window manager can actually make a difference in performance.

More generally, I think the people advocating these have some good UI arguments – OK, maybe only when addressing hard-core hackers, but hey we’re users too. Ditching the overhead of frobbing window sizes and decorations in favor of getting actual work done is a kind of austerity I can get behind. My normal work layout consisted of just three big windows that nearly filled the screen anyway – terminal, Emacs and browser. Why not cut out the surrounding cruft?

I wasn’t able to settle on a tiling wm that really satisfied, though, until my friend HedgeMage pointed me at i3. After a day or so of using it I suspect I’ll be sticking with it. The differences from other tiling wms are not major but it seems just enough better designed and documented to cross a threshold for me, from interesting novelty to useful tool. Along with this change I’m ditching Chatzilla for irsii; my biggest configuration challenge in the new setup, actually, was teaching irssi how to use libnotify so I get visible IRC activity cues even when irsii itself is hidden.

Continue reading

Mar 24

Python speed optimization in the real world

I shipped reposurgeon 2.29 a few minutes ago. The main improvement in this version is speed – it now reads in and analyzes Subversion repositories at a clip of more than 11,000 commits per minute. This, is, in case you are in any doubt, ridiculously fast – faster than the native Subversion tools do it, and for certain far faster than any of the rival conversion utilities can manage. It’s well over an order of magnitude faster than when I began seriously tuning for speed three weeks ago. I’ve learned some interesting lessons along the way.

Continue reading

Feb 26

Mode of the Reposturgeon!

It was inevitable, I suppose; reposurgeon now has its own Emacs mode.

The most laborious task in the reposurgeon conversion of a large CVS or Subversion repository is editing the comment history. You want to do this for two reasons: (1) to massage multiline comments into the summary-line + continuation form that plays well with git log and gitk, and (2) lifting Subversion and CVS commit references from, e.g., ’2345′ to [[SVN:2345]] so reposurgeon can recognize them unambiguously and turn them into action stamps.

In the new release 2.22, there’s a small Emacs mode with several functions that help semi-automate this process.

Fear the reposturgeon!

Feb 02

Is closed source worth it for performance?

The following question appeared in my mailbox today:

If a certain program (that you need) was proprietary, and its open-source counterpart was (currently) 40% slower. Which would you use, the open-source one or the proprietary one?

The answer is: it depends. I’m going to answer this one in public because it’s a useful exercise in thinking about larger tradeoffs.

Continue reading

Jan 03

Solving the CVS-lifting problem

Last month I added CVS-reading support to reposurgeon. The reason I haven’t blogged in ten days is that this pulled me down a rathole out of which I am just now beginning to emerge. And now I have a a request for help – I need to collect some perverse CVS repositories, preferably relatively small ones.

Continue reading

Dec 21

robotfindskitten – the Mayan Apocalypse Edition!

Today’s very special non-world-ending software release, triggered if not originated from here at Eric Conspiracy Secret Laboratories is the amazing Zen simulation, Robot Finds Kitten. I bow in respect before Leonard Richardson and the other giants of kitten-finding history and am humbly proud to be counted among the select few who have contributed to this monumental, er, monument.

Get yer hot fresh tarball right here. It will improve your sex life, clear up your financial problems, cure your acne, and make you as a god among men. Would I lie?

Dec 20

Reposturgeon Attacks Tokyo!

Well, er, no. Actually, it attacks CVS.

Yes, that’s right, the just-shipped reposurgeon 2.11 can now read – though not write – CVS repositories. To get it to do this, I got my lunch-hooks on a relatively old program called cvsps that assembles changesets out of CVS repositories for human inspection. I gave it a –fast-export reporting mode that emits a fast-import stream instead, so now CVS has a universal exporter that will talk to any version-control system that speaks import streams. Oh, yes, and I’m maintaining cvsps now too – applause to David Mansfield, who both did a very good job on that code and sees clearly that its original use case is obsolete and –fast-export is a better way forward.

Two substantial releases of different projects in a day is a fast pace even for me. cvsps-3.0 and reposurgeon-2.11; two great tastes that taste great together.

Fear the reposturgeon!

Dec 16

The Reposturgeon That Ate Sheboygan!

Well-designed software suites should not only be correct, they should be able to demonstrate their own correctness. This is why the new 2.10 release of reposurgeon features a new tool called ‘repodiffer’. And yes, that is what it sounds like – a diff tool that operates not on files but entire repository histories. You get a report on which revisions are identical, which are different, and in the latter case where the differences are, down to which files don’t match. Commits to be paired are matched by committer and commit date. Like reposurgeon, it will work on any version-control system that can emit a fast-import stream.

Continue reading

Dec 02

Beware! The Reposturgeon!

I had said I wasn’t going to do it, but…I experimented, and it turned out to be easier than I thought. Release 2.7 of reposurgeon writes (as well as reading) Subversion repositories. With the untested support for darcs, which should work exactly as well as darcs fast-export and fast-import do, this now brings the set of fully-supported version-control systems to git, hg, bzr, svn, and darcs; reposurgeon can be used for repository surgery and interconversion on any of these.

Continue reading

Nov 04

reposurgeon 2.0 announcement – the full-orchestra version

I shipped reposurgeon 2.0 a few days ago with the Subversion support feature-complete, and a 2.1 minor bugfix release this morning. My previous release announcement was somewhat rushed, so here is a more detailed one explaining why anybody contemplating moving up from Subversion should care.

To go with this, there is a new version of my DVCS Migration HOWTO.

Continue reading

Oct 25

Announcing autorevision

autorevision extracts metadata about the head version of your repository. This program is meant to be used by project build systems to extract properties that can be used in software version strings. It can create files containing variable and macro definitions suitable for C, C++, sh, Python, Perl, PHP, lua, Javascript, and header files suitable for use with preprocessing Info.plist files.

This was a sort of spinoff from irker, though I’ve decided it not to use it there because I want to keep irkerd and irkerhook.py in single self-contained files.

No, I don’t know dak180′s real name. He’s a pretty good collaborator, though.

Oct 21

I hate having to be the heavy…

I nearly issued a forking threat a few minutes ago. Only the second time I’ve felt a need to do that and the first was in 1993, so this is not something I do casually. And I drew back from the brink.

But I may have to if the maintainer I’m dealing with doesn’t clean up his act. His library is critical to one of my projects, but his behavior has been increasingly sloppy and erratic lately. He made a serious design mistake which he’s been trying to paper over with kluges; the kluges have made the code unstable and the latest shipped version is actually broken to the point of unusability without a patch.

Continue reading

Oct 12

End-to-end arguments in software design

My title is, of course, a reference to the 1984 paper End-to-End Arguments in System Design by Reed, Saltzer, and Clark. They enunciated what has since become understood as perhaps the single most central and successful principle of the design of the Internet. If you have not read it, do chase the link; it well deserves its status as a classic.

The authors wrote mostly about the design of communications networks. But the title referred not to “network design” but to system design, and some of the early language in the paper hints that the authors thought they had discovered a design rule with implications beyond networking. I shall argue that indeed they had – there is a version of the end-to-end principle that applies productively and forcefully to the design of (non-networked) software. I shall develop that version, illustrating it with a case study from experience.

Continue reading