Oct 06

irker is feature-complete

I’ve just shipped irker 1.8, and I think this brings the wild ride I’ve been on for the last eleven days approximately to a close. I consider this release feature-complete; it achieves all the goals I had in mind when the CIA service died and I decided it was up to me to rescue the situation. I expect the development pace to slow down a lot from the almost daily release I’ve been doing.

The last really major feature was irkerhook support for Mercurial repositories. I’d be mildly interested in a bzr extractor class if anyone wanted to contribute one, and that probably wouldn’t be hard – the git and hg extractors are about 70 lines each. But with git, hg, and Subversion covered it’s good enough.

Uptake of irker continues at a pleasingly rapid pace. There’s now a second symbiote application, a poller daemon that watches the log of a specified Subversion repository and uses irkerd to ship notifications from it. This can be useful if you don’t have write access to the repo hooks and thus cannot install irkerhook.

Time for a pause and some reflection on lessons to be learned.

Continue reading

Sep 30

irker takes off like a rocket

It was just three days ago that I shipped irker 1.0, but the project is already a huge hit out there in hackerland. It’s clear from traffic on the freenode #commits channel that irker installations are springing up everywhere. There’s already one symbiote, a proxy that takes XML-RPC requests in the CIA format and passes them to an irker instance (you have to supply your own mapping of projects to IRC channels for it to use). And at least one custom hook already written and in production – by the Python development list, as it happens.

I’m a bit boggled, actually. I don’t think I’ve ever had a project go from launch to all over the freakin’ landscape this fast before. Guess that’ll happen when you step up with a clean replacement for a service that lots of people were habituated to and have suddenly lost.

There’s more work to be done, of course. (There’s a public repository, and an #irker IRC channel, for people interested in following development.)

Continue reading

Sep 27

CIA and the perils of overengineering

The CIA commit-notification service abruptly died two days ago, a development that surprised nobody who has been paying attention to the recent history of the codebase and its one public server site. A screwup at the cloud service hosting the CIA virtual machine irretrievably destroyed the instance data; please don’t ask me for details, I don’t know how it happened and don’t care. The CIA codebase is so messed up that even reconstituting a virgin instance would be way too much work – and that I will talk about a bit later in this post.

Fortunately, I saw this coming and had started work on a CIA replacement in late August. I had been holding off releasing it because there was some effort going on to salvage the CIA code, but that possibility effectively vanished when the only instance was erased. I shipped my replacement just a few minutes ago, and expect to spend much of the next week helping forge-site operators install it so we can have our notification service back.

The remainder of this post is a finished version of a design analysis of CIA I started a couple of weeks ago when the death of the service was still only a theoretical possibility. Since that theory has become actuality, the message should be heard loudly and clearly: this was a truly classic case of over-engineering, code bloat, excessive centralization, and bad practice. Read on for the cautionary tale.

Continue reading

Sep 27

irker 1.0 (a functional CIA replacement) is shipped

OK, I’ve been hacking intensely for most of the last 24 hours and here’s the payoff: irker-1.0 is shipped. Code and documentation are at http://www.catb.org/esr/irker/.

Out of the starting box we have a hook script with tested support for git and (rather clumsily) Subversion; hg should be a piece of cake for anyone who wants to step up. Forge-site operators can begin installing the relay daemon and the repo hook immediately, and should do so.

Coming soon: a long essay I’ve been sitting on analyzing the now-dead CIA service as a case study in over-engineering. It’s not really very surprising that it collapsed under its own weight.

Also note that there is an XML-RPC proxy for people who have limited ability to change their hook scripts. I haven’t looked at the code myself but there’s a pointer in the irker README file.

Sep 27

An emergency replacement for the CIA service is coming.

A few hours ago I learned that, due to a virtual-server mishap, the cia.vc notification service is dead. And not coming back.

This was not entirely unexpected. The CIA codebase was a shambles, the service has been flaky and subject to outages, and the server-site operator who inherited it has for some time been muttering darkly that the end was probably nigh.

I’ve been sitting on a lightweight replacement for CIA since late August, holding off shipping until it was clear whether or not a salvage effort on the codebase was going to succeed. That option is off the table now, so I’m going into emergency overdrive to get a release out.

The main thing that still needs to be done is for me to finish and test a hook script for git repos, so that when I ship the admins at places like SourceForge and GitHub will be able to drop in both a server instance and the correct hook code. This script will also be a model for hooks serving other VCSes such as Mercurial, Subversion, and (ugh) CVS.

I’m working on that now and expect to ship within 48 hours. Watch this space.

Sep 24

Practical prophecy

Inspired by Dave Logan’s keynote on tribal leadership at AgileCultureCon, I did a breakout session and then an open-space followup on “Practical Prophecy 101″.

Recall that in Logan’s terms a “prophet” is a person who moves the behavior of his tribe towards greater cooperation and creativity by (his words) “preaching the inevitability of values-based change”.

Venessa Miemis took notes on my talk. Here’s a lightly edited and expanded version of those notes. In each item I have replayed a quote of mine that she recorded; where appropriate I have expanded a little on the thinking behind it.

Continue reading

Sep 19

Who were the prophets of the early hackers?

I learned a new way of thinking about social behavior at Agile CultureCon last week – Dave Logan’s taxonomy of tribal stages and his interestingly specialized notion of what a “prophet” is. For review, see Logan’s TED talk.

Logan explains the distribution of tribal stages as follows: Stage 1, “Life Sucks”, is the violent and profoundly dysfunctional tribalism of gangs and prisons (approximately 2% of tribes); Stage 2, “My life sucks”, is bureaucracy (about 22% of tribes); Stage 3, “I’m great (but you’re not)!” is most of business and academia (about 48% of tribes); Stage 4: “We’re great!” is where you start to see serious creativity, tribal self-awareness, and collective sense of mission (about 22% of tribes); and Stage 5 “Life’s great!” is high-creative behavior totally driven by values rather than ego or struggle against some adversary (about 2% of tribes).

A “prophet”, in Logan’s model, is somebody who expresses the deepest shared values of a tribe and invites people in it to change stage (and fuse with other tribes at the new stage). Because most people, most of the time, live in tribes with a stage 3 culture, the most common upward transition (and the most common kind of prophet) is from stage 3 to stage 4.

I noted in a previous post that hearing this in a talk made the hair on the backs of my arms stand up. Because I have lived through, and was one of the prophets of, the hacker culture’s transition from largely unconscious mixed stage-3/stage-4 to fully conscious mostly Stage 4 behavior (“We’re great!”) in the 1990s.

But. I am by no means sufficiently ignorant or egotistical to think I was our only prophet. Most obviously there was Richard Stallman a decade before me, issuing a stage 4 call to higher values around “free software”. But because I was a historian before I was a prophet, I can’t really stop there. I find myself asking who the earlier prophets were!

Continue reading

Sep 17

Culture hacking, reloaded

My last four days, at the Agile CultureCon split between Philadelphia and Boston, have thrown more new ideas and techniques at me than I’m used to encountering in a normal four months. Or more. It was very challenging and exciting, the more so because I was immersed in a culture at some distance from those where I usually hang out.

The organizers (Dan Mezick & Andre Dhondt) and various friends (now including me) are launching from agile software development into new ways of organizing work and communication that dynamite a lot of common assumptions about the necessity of power relationships and hierarchies. What makes this really interesting is not the theory but the working examples. They’re not dealing in vague platitudes, but in methods that can be taught and replicated. (And yes, I will describe some of them later in this post.)

Nobody in this crowd thinks politically (or at least if they do, it doesn’t show); it’s all framed as ways to fix corporate cultures to make them more productive and happier. But what this was, underneath occasional freshets of vaguely new-agey language, was a three-day workshop in practical anarchy.

Continue reading

Sep 06

A shout to the world’s technical journals

So, after my post on ground-truth documents, one of my commenters argued eloquently that I ought to clean it up and submit it to a journal read by people who manage programming projects. He suggested Software Practice and Experience.

This seemed like a pretty good idea, until I read SP&E’s submission procedures and was reminded that (like most journals) they want me to assign the copyright of my submission to the publisher.

My instant reaction was this: Fuck. That. Noise. I’m certainly willing to cede publication rights when I want to be published, but copyright assignment ain’t going to happen. Ever. Nobody gets to own my work but me. (Yes, I insist on this with my book publishers too.)

I have a message to all you technical journal publishers out there…

Continue reading

Sep 05

coverity-submit 1.2 is released

Coverity simplified their remote-submission procedure. Because of this, I have been able to remove the ugliest bits of configuration cruft from coverity-submit; you no longer have to specify either a public drop directory for your results tarball or a URL that advertises it.

Get your remote-static-checking goodness here.

Sep 03

This is a tease.

Yesterday I applied for allocation of a new public port number from IANA. It’s 6659. When the allocation is confirmed, I’ll publish the source code for a reference implementation of the server. It’s a bit over 300 lines of Python.

Let the speculation begin. :-)

Aug 31

Ground-truth documents

Sometimes good terminology, by making a distinction that wasn’t easily articulated before, can be very clarifying. I was in an IRC conversation about software engineering with A&D regular HedgeMage earlier today and found myself inventing a term that I think may be useful: the ground-truth document.

Continue reading

Aug 23

Defense Distributed

I suppose it was inevitable, in a good way. Some friends of freedom have begun a project dedicated to developing and sharing open-source designs for firearms that can be manufactured with a 3D printer. Read about it here at Defense Distributed.

I approve, of course. I approve of any development that makes it more difficult for governments and criminals to monopolize the use of force. As 3D printers become less expensive and more ubiquitous, this could be a major step in the right direction.

Aug 22

ciabot for git version 3.5 is released

I shipped an updated version of the ciabot hook scripts for git to the git maintainers this morning.

The cool new thing in this release is that the script no longer needs to be modified for installation as a hook. You can install one copy where any number of git repositories can see it; when it’s run it will collect the information it needs either by autoconfiguring or by looking at variables set in each project’s .git/config file.

UPDATE: Once I started looking at the code…I found a way to make it completely self-configuring in the normal case. So I’ve shipped 3.6. This will be helpful for forge sites like Savannah, because it means they’ll be able to install one standard git hook that Just Works.

Aug 21

How To Choose A Martial-Arts School

The responses to my progress report on searching for a new martial-arts school made it clear that many people are interested in advice on this topic. The problem is especially difficult for new students choosing a first school, as they have yet to develop the kind of trained eye that can evaluate technique.

I have been training in empty-hand combat and contact weapons since 1982; more or less continuously since 1990. I have studied shotokan, tae kwon do, aikido, wing chun kung fu, and Mixed Martial Arts at five different schools and trained in sword-centered Western Martial Arts at two more. Along the way I’ve picked up bits and pieces of iaido, kenjutsu, escrima stick fighting, penjak silat, shaolin kung fu, Greco-Roman wrestling, Okinawan karate, naginata-do, and lua. I hold a black belt in tae kwon do and have been an instructor in multiple styles. I report these things to establish that my experience of styles and schools is very broad, equipping me to give useful advice on how to choose one.

This how-to will be aimed mainly at people new to the martial arts trying to choose a first school, but the questions I suggest can usefully be asked even if you are a much more experienced student.

Continue reading