Cometary Contributors

I released reposurgeon 3.30 today. It has been five years and a month since the first public release.

In those five years, the design concept seems to have proved out very well, finding use in many repository conversions. But the project exhibits an unusual sociology; I don’t get lots of casual contributors, only a few exceptional ones.

Your typical open-source project sees a sort of exponential distribution in which small fix patches from people you see stop by only once are common, single feature-sized patches less so, and complex sustained work that reimagines entire subsystems is rare. There’s an obvious inverse relation between frequency and complexity scale. At intermediate and higher complexity scales you often get regular contributors who do extended work on different things over time. GPSD is like this.

On reposurgeon I see an entirely different pattern. Casual patches are rare to nonexistent. For long stretches of time I have no active collaborators at all. Then a hacker will appear out of the void and begin contributing very clever patches. He (no shes yet) will draw closer to the project, and for a few days or weeks we’ll be in an intense collaborative mode tossing ideas and patches back and forth. Some complex series of features will be implemented.

Then, his particular feature-lust fulfilled, said hacker will quietly vanish into the interstellar darkness never to be seen again, like some comet on a hyperbolic trajectory after a pass near the Sun. Never yet has there been more than one cometary hacker at a time.

OK, I exaggerate slightly. The project has some semi-regular hangers-on in the #reposurgeon channel (one of them is A&D commenter Mike Swanson). But those people tend to be power users rather than major code contributors; the pattern of large code drops by people who appear, do work that impresses the hell out of me, and then vanish, still dominates code contributions.

My wife Cathy called this one right when I remarked on it. Most people never use reposurgeon more than once, but the hands that find it are disproportionately likely to be very skilled ones. All of my half-dozen or so cometary contributors have been damn good hackers even by my elevated standards, careful and imaginative and tasteful. When people like this detect a deficiency in a tool, they fix it – and their idea of “easy” fixes would daunt lesser mortals.

It’d be nice if some of these hackers would stick around, because I love collaborating with people that bright, but oh well. They’re as in demand as only the capable can be. And at the end of the day, there are much worse things you can say of a software project than “it attracts high-quality work from high-quality people, er, even if they don’t tend to stick around”.

31 comments

  1. >> But the project exhibits an unusual sociology; I don’t get lots of casual contributors, only a few exceptional ones.

    Sounds pretty cut and dried to me–you’re the Alpha, they are not. Yet the survival of the pack is paramount, so they do their work.

    However, as capable as they may be, should they dare challenge your reproductive rights, it might not turn out so well for them. Better to pull the sled than challenge the lead dog.

  2. @esr:
    When people like this detect a deficiency in a tool, they fix it – and their idea of “easy” fixes would daunt lesser mortals.

    Also, keep in mind that for the use case that needs Reposurgeon, whether the fix is easy or daunting, hacking away a deficiency in Reposurgeon is likely to be the path of least resistance. What alternative would these cometary hackers have? Although my understanding of Reposurgeon’s problem domain isn’t really solid, my impression is that they would either have to rewrite Reposurgeon from scratch or do Reposurgeon’s job by hand, the latter project likely involving a completion date in the late 83rd century.

    1. >they would either have to rewrite Reposurgeon from scratch or do Reposurgeon’s job by hand, the latter project likely involving a completion date in the late 83rd century.

      Heh. Well put.

  3. Congratulations for this milestone. :-) If Emacs’ migration to git is any indication, reposurgeon is extremely good at what it does.

    Now, I don’t mean to be rude, but… isn’t it customary to include a link to the program’s resource page in these posts?

    And when you parody monster-movie titles, don’t you usually say “Reposturgeon”? ;-)

    1. >And when you parody monster-movie titles, don’t you usually say “Reposturgeon”? ;-)

      No. I checked. :-)

  4. Any chance that the “solving” of the Reposurgeon problem will enliven activity on the ForgePlucker front?

    1. >Any chance that the “solving” of the Reposurgeon problem will enliven activity on the ForgePlucker front?

      Oh, Goddess. It’s hardly “solved”.

  5. None of this comes as any surprise.

    Reposurgeon is a highly specialized tool folks are likely to use *once*. The cometary hackers are people who needed to use it, encountered an issue, thought it through, and contributed a fix. Having scratched their itch, they have no reason to stick around.

    I think the challenge to retaining such talent is understanding who they are, what their competencies are, and what *else* they might be able to make significant contributions to, and pointing them at it. But this has the additional challenge of knowing what projects *could* use their assistance and what might motivate them to get involved. The answer to a pointer might be “Don’t use it, no interest in it, and not something I want to devote time to.”

    There are many open source projects that could use your skills. What makes *you* get involved? What makes you *stay* involved? Most of what I recall you doing in recent years has been solving specific problems, and once you feel you’ve contributed a solution, you’re off to something else. What, besides Emacs, which you use on a daily basis, have you had a continuing involvement in? Why? What makes *you* stick around?

    I suspect you have an enviable contacts file of top quality hackers, but making use of that resource is another matter.

    (Incidentally, the Subject of the email about this post was “Reposurgeon against the Marians!”, which produced a WTF moment…)
    ______
    Dennis

  6. > No. I checked. :-)

    I assume you’re joking, but in case you’re not:

    14
    Another bite of the reposturgeon
    Attack of the 50-foot reposturgeon
    Beware! The Reposturgeon!
    Heads up: the reposturgeon is mutating!
    Mode of the Reposturgeon!
    Reposturgeon Attacks Tokyo!
    Reposturgeon and Santa Claus Against The Martians!
    Reposturgeon from the Black Lagoon!
    Return of the reposturgeon!
    Revenge of the reposturgeon!
    Son of the reposturgeon!
    Terror of the Reposturgeon!
    The Reposturgeon That Ate Sheboygan!
    The Reposturgeon from Beyond Space!

    7 (only two of which fit the “monster movie parody” mold)
    Coming soon: reposurgeon does Subversion
    Announcing reposurgeon – a tool for the good new days
    Reposurgeon Killed The Radio Stars!
    Reposurgeon Battles All Monsters!
    Lessons learned from reposurgeon
    Looking for reposurgeon test cases
    reposurgeon 2.0 announcement

    1. >I assume you’re joking, but in case you’re not:

      Oops. I screwed up my search term – I already used “the Martians”. Niow I’m actually going to have to change the post title. Oh well – I’d been thinking I should have called it “Cometary Contributors” anyway.

  7. Eric wrote: It’d be nice if some of these hackers would stick around, because I love collaborating with people that bright, but oh well. They’re as in demand as only the capable can be.

    Indeed – or to put it another way, I like to think of them as spending their time coding on some project even more valuable to us out there somewhere.

  8. @ ESR

    Sorry to go cosmically off-topic, but I think this is really important.

    Are you involved with any response/counter-attack/etc. in relation to the disgusting Contributor Covenant? On http://contributor-covenant.org/ it says…

    This code of conduct has already been adopted by over 10,000 open source projects.

    This a situation where your rep/cred are sorely needed.

  9. @Brian Marshall

    > This code of conduct has already been adopted by over 10,000 open source projects.

    Hmmm…. I searched the “A Sample of Projects” page for linux, perl, python, apache, gcc, llvm, libc, core [tools], emacs, busybox, vi (only got the ViM Cheat Sheet), ruby (got a couple of hits), ….

    You get the point, I trust – none of the important project are buying into this nonsense. “Fluffy Bunnies will be fluffy” — All Others Need Not Apply.

    IMNSHO, I don’t think we need to waste Eric’s time on this. (Although I am looking forward to his non-PC Code of Conduct.)

    P.S.Thank you, Eric, for the previewer!

  10. @ John Bell

    Well, the situation was described (in the November post) as being serious, and (in the comment to which you linked) ESR said he was working on an alternate Code of Conduct, so I was wondering what was happening.

    If the whole thing is just silly fluff, that is good.

    Actually, I was surprised that ESR thought that an alternate C-of-C was worth having, unless it was something like…

    Code of Conduct

    Note that we have reviewed the Contributor Covenant promoted by contributor-covenant.org and we have rejected it with extreme prejudice. If you think that this decision was a mistake, please feel free to get back to us when you grow up a little more.

    1.0 Humans are expected to act like civilized humans.
    1.1 Non-humans will be considered on a case by case basis.

  11. The main thing I see wrong with that hypothetical CoC excerpt is that it is not referred to as the “Cod of Conduct”. I consider this a dealbreaker.

  12. s/beat you with a stick/slap you around with a large trout/

    (Getting the trout waxed is optional.)

  13. That would make it more of a Trout of Conduct either way. You’re in danger of losing the referent.

  14. I will advance the theory that your cometary contributors are programmers working in the commercial software world, not really interested in open source projects. They have some assignment from their clients or bosses and they discover they need your OS tool to complete it…except there’s one problem that they need to fix. Once they fix it, they say goodbye; nobody is paying them for their time.

    1. >I will advance the theory that your cometary contributors are programmers working in the commercial software world, not really interested in open source projects. […] Once they fix it, they say goodbye; nobody is paying them for their time.

      I think that’s half right. I can believe that they don’t stick around because paid work is a higher priority, but…all of them have been very clued-in and respectful about the open-source style. Thus, I could more easily believe that if these guys are in the proprietary world, they at least welcome the chance to play in open source some of the time. When they can justify it.

  15. Apart from Julien “FrnchFrgg” Rivaud (see http://esr.ibiblio.org/?p=4861), who are those cometary contributors?

    By your own account, they’ve done “work that impresses the hell out of me“, so I’m curious who lands in that set (I suspect I fall well short of qualifying).

    1. >Apart from Julien “FrnchFrgg” Rivaud (see http://esr.ibiblio.org/?p=4861), who are those cometary contributors?

      The most recent one was some guy named Richard Hansen, and that’s all I know about him except that he’s a damn good hacker and talks like he’s been around the track a few times.

      The first one was some dude named Greg Hudson, who materialized, solved a problem that had been stalling Subversion support for quite a long time, and disappeared. I’m still not sure I completely understand his copy-on-write implementation and I’m very careful when I have to touch it.

      Daniel Brooks. Chris Lemmons. Edward Cree. Two of those guys comment on A&D occasionally, so I guess you could say they haven’t completely disappeared, but they haven’t contributed any code since their near pass.

      There might be one or two others I’m missing.

  16. Eric and LS are both correct in my case. Thanks for lumping me in with all these superb programmers, btw; I just found some juicy low-hanging fruit after I discovered that it took 40ish hours to run reposurgeon over my client’s svn repository (80k commits, as I recall).

    Another interesting coincidence is that Julien and I already knew each other from our mutual interest in Mozilla. We had a good time reconnecting in the irc channel.

  17. @ DocDoc

    Thanks for the Vox Popoli link. That was a good effort and good read. Many things worth considering there.

    @ Jay Maynard

    Your contributor agreement was great. Simple, focuses on code, offers to help [with code], and otherwise shows the door.

    1. >FWIW, Greg Hudson is probably this one:

      Yes, that seems extremely likely.

      If that’s the guy, I owe him a big one. The Subversion reader was both broken and slow before he contributed his CoW implementation. I’d been bashing my forehead on it for months to no good result. It was the worst case of stuck in a bad spot I’ve had in something like 15 years.

  18. (Warning: long ramble ahoy.)

    When I originally read this post, I thought “Heh. No way is he talking about me; it’s the real contributors that are cometary”. I decided I was just an ‘Earth-crossing asteroid’: similar orbit to comets (highly eccentric, natch ;) but not nearly as bright. I didn’t do any kind of original design, I just plugged some implementation into a framework so well laid out that I didn’t understand how anything but the bit I was hacking on worked and that didn’t matter.

    So when I re-read it today and saw esr’s comment above naming me, it well and truly flabbered my gast. To the point of ‘successfully resist[ing] a vague feeling that [I] ought to fall to [my] knees and cry out “I’m not worthy!”’, even. (Between this and the Dave Miller Incident, my ego is going to get seriously overinflated if I’m not careful.)

    As for LS’s theory… here’s one datum. My interest in the dread reposturgeon was indeed work-motivated, but not for anything proprietary. It’s an out-of-tree Linux driver; we’re planning to move it from hg to git in order to streamline getting patches from it into the in-tree version of the same driver. My flurry of patches resulted from my spinning up a proof of concept to show my boss “hey, we could move to git and it wouldn’t even be hard”. Meanwhile, I don’t have much need for reposurgeon outside of work, because all of my projects are in git and stayin’ that way. (Nonetheless, my patches were done on my time, rather than my employer’s.) Now that company politics have finally caught up enough to make the conversion happen, this comet may be headed round for another close approach…

    But to return to my overinflated ego, the thing I find really disturbing is this. If I’m really smart enough to impress esr, I feel like I ought to be doing more with myself than toy projects, games, and an obscure driver. It’s not that I’m failing to change the world, it’s that I’m not even trying. (Not for want of causes, either; there are plenty of things I’d change about the world if I could, and I suspect esr would approve of most of them.) Obviously without Eric’s extroversion I won’t be as influential as him, but… dangit, Eric, what’s your trick? You make having a disproportionate effect on the course of history look easy! Why can I never find anything important to hack on?

Leave a comment

Your email address will not be published. Required fields are marked *