I’ve just shipped irker 1.8, and I think this brings the wild ride I’ve been on for the last eleven days approximately to a close. I consider this release feature-complete; it achieves all the goals I had in mind when the CIA service died and I decided it was up to me to rescue the situation. I expect the development pace to slow down a lot from the almost daily release I’ve been doing.
The last really major feature was irkerhook support for Mercurial repositories. I’d be mildly interested in a bzr extractor class if anyone wanted to contribute one, and that probably wouldn’t be hard – the git and hg extractors are about 70 lines each. But with git, hg, and Subversion covered it’s good enough.
Uptake of irker continues at a pleasingly rapid pace. There’s now a second symbiote application, a poller daemon that watches the log of a specified Subversion repository and uses irkerd to ship notifications from it. This can be useful if you don’t have write access to the repo hooks and thus cannot install irkerhook.
Time for a pause and some reflection on lessons to be learned.
I think the most interesting aspect of the project, after I got the basic irkerd design in working shape, was hardening the implementation against denial-of-service attacks. I got some high-powered volunteer help with this, including from A&D regulars Daniel Franke and Peter Scott. Reasoning out the attack paths and the simplest possible countermeasures was just plain fun, possibly more fun because I’ve never had to do this kind of analysis before.
The willingness of hackers to step up and contribute to a project like this continues to be a wonderful thing about which I hope never to become jaded. Because this project spun up so fast and still doesn’t have a mailing list, I know many of my contributers mainly by IRC nicks. Thank you, AI0867, birkenfeld, KingPin, laurentb, dak180, nenolod, and everybody else who showed up on #irker to help and contribute and critique and report bugs and ask questions. It has been a pleasure working with all of you and watching a healthy micro-community form around this project within hours of my first release.
irker does have some has competition for its goal of replacing the CIA notification service. Some Debian people dusted off a project called “KGB”; I gave its principal designer a bit of a hard time when he showed up in A&D’s comments because KGB is heavier and more elaborate than it needs to be, but in truth it’s not actually horrible. CIA’s spaghetti architecture was horrible; KGB is merely somewhat overweight.
KGB is also, judging by the traffic on freenode #commits, losing the adoption race. They got to the party late with software that’s more difficult to grok and get running than irker is. Though, to be fair, irker’s minimalistic design might not have gained an advantage without also having better documentation. I never consider high-quality documentation a mere optional extra on my projects. This is certainly a lesson more developers could stand to learn…
I will admit, however, that one KGB feature I initially scoffed at turned out to be sufficiently useful and lightweight that I added it – that is, support for color-highlighting notication lines. The key to that decision was that I found a way to implement it in a handful of lines of code in irkerhook.py; the feature doesn’t touch irkerd at all.
This is an example of a larger theme in irker’s design – policy/mechanism separation. The irker daemon is pure mechanism; it has no options or control knobs of any kind (other than one to enable debugging messages and another to dump the version and exit). It’s just a message bus. All the policy stuff (choices about what to put in a notification) lives in irkerhook.py. And you damn betcha that I plan to keep it that way!
Because all irkerhook.py does is gather information that it ships as JSON, the addition of one simple option – to dump the JSON to stdout rather than trying to ship to the configured irker instance – makes changes in the the policy stuff very easy to test. Which is as unlike the huge, nigh-untestable and thus extremely failure-prone hairball that was CIA as it is possible to be and do an even remotely similar job.
Generally, policy needs to change more often than than mechanism. So, when you partition your system into a mechanism part (irkerd) and a policy part (irkerhook.py), the policy part is the part where there is greater need for testability. Or, to put it differently: when you partition your code and you find that the least stable part is most amenable to testing, you’re doing it right.
Yes, this is the old-time Unix gospel of minimalism and design for testability I’m preaching here, brothers and sisters, and I’m doin’ it for a reason. The total line count of irkerd and irkerhook.py code is 689 LOC (measured by sloccount), compared to 1957 LOC for KGB and probably tens of thousands of lines for CIA (I haven’t measured that). When you pay proper attention to separation of function and separation of policy from mechanism, your code gets not larger but smaller. With virtuous consequences, the most important of which is fewer failure modes.
Comprehensibility helps deployment, too. It’s not that I think every project administrator who has adopted irker has actually read the daemon code, but I’m certain its smallness and lightness – and the fact that you can read through irkerhook.py and grok what what it’s doing in just a couple of minutes – has made it an easier sale to people who are chronically short of time and attention to get all their tasks done.