Aug 25

The Great Beast has met its match

When I built the Great Beast of Malvern, it was intended for surgery on large repositories. The specific aim in view was to support converting the NetBSD CVS to git, but that project is stalled because the political process around NetBSD’s decision about when to move seems to have seized up. I’ve got the hardware and software ready when they’re ready to move.

Now I have another repo-conversion job in the offing – and it does something I thought I’d never see. The working set exceeds 32GB! For comparison, the working set of the entire NetBSD conversion tops out at about 18GB.

What, you might well ask, can possibly have a history that huge? And the answer is…GCC. Yes, they’re looking to move from Subversion to git. And this is clearly a job for the Great Beast, once the additional 32GB I just ordered from Newegg arrives.

Aug 21

On having good form

Sometimes, the best encouragement you can get in a martial-arts class is silence.

Once a month my school, which normally teaches a combination of wing chun kung fu and Philippine blade/stick fighting, gets a visit from Sifu Jerry Devone, who teaches pure traditional Wing Chun at a level a bit higher than our Sifu Dale Yeager.

Sifu Jerry is a nice guy, but it’s not difficult to find videos of him almost casually destroying other kung fu players in ring fights. He shows the same soft/hard combination as an instructor – never yells at anyone, but demands precision and perfection and often gets it, even from unpromising students.

Tonight he told the four senior students (including Cathy and myself) to line up facing him and do the Siu Nim Tao form with him watching for defects. From some instructors this would be a terrifying prospect, with anticipation of a humiliating ass-chewing to follow. While Sifu Dale wouldn’t exactly humiliate someone who screwed up, he might make snarky theatrical jokes about bad performance in a half-laughing-with, half-laughing-at manner. Neither of these is Sifu Jerry’s style – he’d just quietly correct in a way that would make you grimly determined to get it right next time.

Still, I felt rather stressed. I know the motions of Siu Nim Tao – it’s not a complex form, and it doesn’t require anything I’m bad at like high kicking – but it’s subtle. There are fine details in it, and the devil is in those details, and in getting the overall flow and timing right.

Continue reading

Aug 18

Yes, NTPsec is real and I am involved

A couple of stories by Charles Babcock and (my coincidentally old friend) Steven J. Vaughan-Nichols have mentioned the existence of an ‘NTPsec’ project being funded by the Core Infrastructure Initiative as an alternative and perhaps eventual replacement for the reference implementation of Network Time Protocol maintained by Harlan Stenn and the Network Time Foundation.

I confirm that NTPsec does exist, and that I am deeply involved in it.

The project has not yet officially released code, though you can view a preliminary web page at ntpsec.org. For various complicated political reasons a full public discussion of the project’s genesis and goals should wait until we go fully public. You probably won’t have to wait long for this.

I can, however, disclose several facts that I think will be of interest to readers of this blog…

Continue reading

Aug 01

Productive yak shaving

So here’s how my day went….

I started off trying to convert a legacy manual page to asciidoc. Found that pandoc (which could be the target of a whole separate rant, because it totally sucks at translating anything with tables in it) won’t do that.

@PUSH…

But it will convert DocBook to asciidoc. OK, so I can use my doclifter tool to convert the manual page to DocBook, then DocBook to asciidoc via pandoc I try this, and doclifter promptly loses its cookies.

@PUSH…

Huh? Oh, I see an [nt]roff request in there I’ve never seen before. Must fix doclifter to handle that. Hack hack hack – done. Push the fix, think “I ought to ship a doclifter release”

@PUSH…

I look at the doclifter repo. I see that the commit graph has an ugly merge bubble in it from where I forgot a –rebase switch when I was pulling last. It’s the pointless kind of bubble where someone else’s patch commutes with mine so the history might as well be linear and easier to read.

You know, before I ship it…I was planning to move that repo from the downstairs machine to GitLab anyway, I might as well fix that bubble in the process….

@PUSH…

Now how do I do that? Hm, yeah, this patch-replay sequence will do it. I ought to can that procedure into a script because I’m doubtless going to have to do it again. (I hates pointless merge bubbles, I hates them forever…) Script done. Hm. I should publish this.

@PUSH…

OK, throw together a man page and README and Makefile. Oh, right, I need a project logo for the web page. What should it be? Into my mind, unbidden, enters the image of the scrubbing bubble animated characters from an advertising campaign of my childhood. Win!

@PUSH…

I google for images, find one, and GIMP out a suitable piece within individual scrubbing bubble in it. Snrk, yeah, that’s funny. Scale it to 64×64 and go.

@POP…

Funny logo achieved.

@POP…

OK, version 1.0 of git-debubble gets published.

@POP…

git-debubble gets applied to the doclifter repo,

@POP…

…which I then publish to GitLab.

@POP…

Now I can convert the manual page to DocBook XML…

@POP…

…which can then be converted to asciidoc.

I have a lot of days like this.

I think there ought to be a term for a sequence of dependency fulfillments that is a lot like yak shaving, except that something useful or entertaining independently of the original task gets emitted at each stage.

Jul 13

How to submit a drive-by patch and get it accepted

I think it’s weird that I have to write this post in 2015, but earlier today I had to explain to someone with the technical skills to submit a good patch that he was doing the process wrong in some basic and extremely annoying ways.

Googling revealed that most explanations of patch etiquette are rather project-specific in their advice. So I’m going to explain the basics of patch submission that apply to just about any open-source project, with a focus on how to do it right when you aren’t a regular committer (that is, it’s what’s often called a drive-by patch). Here we go…

Continue reading

Jul 04

git-weave, a tool for synthesizing repositories from fossil tarballs

Welcome to my first new-project release of the year, git-weave. It’s a polished and documented version of the script I used to reconstruct the early history of INTERCAL five years ago – see Risk, Verification, and the INTERCAL Reconstruction Massacree for the details on that one.

git-weave can be used to explode a git repository into a sequence of per-commit directory trees accompanied by a metadata file describing parent-child linkage, holding committer/author/timestamps/comment metadata, and carrying tags.

Going in the other direction, it can take the same sequence of trees plus metadata file and reconstruct the live repository. Round-tripping is lossless.

What it’s really useful for is reconstructing a partial but useful ancient history of a project from before it was put under version control. Find its release archives, synthesize a metadata file, apply this tool, and you get a repository that can easily be glued to the modern, more continuous history.

Yes, you only get a commit for each release tree or patch you can dig up, but this is better than nothing and often quite interesting.

Nifty detail: the project logo is the ancient Egyptian hieroglyph for a weaver’s shuttle.

Jun 23

How to spot a high-quality repository conversion

In my last post, I inveighed against using git-svn to do whole-repository conversions from Subversion to git (as opposed to its intended use, which is working a Subversion repository live through a git remote).

Now comes the word that hundreds of projects a week seem to be fleeing SourceForge because of their evil we’ll-hijack-your-repo-and-crapwarify-your installer policy. And they’re moving to GitHub via its automatic importer. Which, sigh, uses git-svn.

I wouldn’t trust that automatic importer (or any other conversion path that uses git-svn) with anything I write, so I don’t know how badly it messes things up.

But as a public service, I follow with a description of how a really well-done repository conversion – the kind I would deliver using reposurgeon – differs from a crappy one.

Continue reading

May 28

Don’t do svn-to-git repository conversions with git-svn!

This is a public-service warning.

It has come to my attention that some help pages on the web are still recommending git-svn as a conversion tool for migrating Subversion repositories to git. DO NOT DO THIS. You may damage your history badly if you do.

Reminder: I am speaking as an expert, having done numerous large and messy repository conversions. I’ve probably done more Subversion-to-git lifts than anybody else, I’ve torture-tested all the major tools for this job, and I know their failure modes intimately. Rather more intimately than I want to…

Continue reading

May 18

Zeno tarpits

There’s a deeply annoying class of phenomena which, if you write code for any length of time, you will inevitably encounter. I have found it to be particularly prevalent in transformations to clean up or canonicalize large, complex data sets; repository export tools hit variants of it all the time, and so does my doclifter program for lifting [nt]roff markup to XML-DocBook.

It goes like this. You write code that handles a large fraction (say, 80%) of the problem space in a week. Then you notice that it’s barfing on the 20% remaining edge cases. These will be ugly to handle and greatly increase the complexity of your program, but it can be done, and you do it.

Once again, you have solved 80% of the remaining cases, and it took about a week – because your code is more complex than it used to be; testing it and making sure you don’t have regressions is about twice as difficult. But it can be done, at the cost of doubling your code complexity again, and you do it. Congratulations! You now handle 80% of the remaining cases. Then you notice that it’s barfing on 20% of remaining tricky edge cases….

…lather, rinse, repeat. If the problem space is seriously gnarly you can find yourself in a seemingly neverending cycle in which you’re expending multiplicatively more effort on each greater effort for multiplicatively decreasing returns. This is especially likely if your test range is expanding to include weirder data sets – in my case, older and gnarlier repositories or newer and gnarlier manual pages.

I think this is a common enough hazard of programming to deserve a name.

Continue reading

May 11

How to Deny a Question’s Premise in One Easy Invention

Now that the Universe Splitter is out, it might be that a lot more people are going to trip over the word “mu” and wonder about it. Or it might be the word only occurs in the G+ poll about Universe Splitter – I don’t know, I haven’t seen the app (which appears to be a pretty good joke about the many-wolds interpretation of quantum mechanics) itself.

In any case, the most important thing to know about “mu” is that it is usually the correct answer to the question “Have you stopped beating your wife?”. More generally, it is a way of saying “Neither a yes or no would be a correct answer, because your question is incorrect”,

But the history of how it got that meaning is also entertaining.

Continue reading

May 05

Sometimes progress diminishes

It’s not news to long-time followers of this blog that I love listening to virtuoso guitarists. Once, long ago in the 1980s I went to see a guitarist named Michael Hedges who astonished the crap out of me. The guy made sounds come out of a wooden flattop that were like nothing else on Earth.

Hedges died a few years later in a car crash, tragically young, and is no longer very well remembered. But I was on IRC yesterday taking music with a friend who mentioned a harmonica and a whistler doing Jimi Hendrix in a “laid back, measured, acoustic style”, and I brought up Hedges because I remembered his cover of All Along The Watchtower as an utterly amazing thing.

Afterwards, in a mood of gentle nostalgia, I searched YouTube for a recording of it. Found one, from the Wolf Trap festival in ’86, and got a surprise.

It was undoubtedly very similar to the performance I heard at around the same time, but…it just didn’t sound that interesting. Technically accomplished, yes, but it didn’t produce the feeling of wonder and awe I experienced then. His original Because It’s There followed on the playlist, and held up better, but…huh?

It didn’t take me long to figure this out. It’s because in 2015 I’m surrounded by guitarists doing what Hedges was doing in the late 1980s. It even has a name these days: “percussive fingerstyle”, Andy McKee, Antoine Dufour, Erik Mongrain, Tommy Emmanuel; players like these come up on my Pandora feed a lot, intermixed with the jazz fusion and progressive metal.

Sometimes progress diminishes its pioneers. It can be difficult to remember how bold an artistic innovation was once we’ve become used to its consequences. Especially when the followers exceed the originator; I must concede that Andy McKee, for example, does Hedges’s thing better than Hedges himself did. It may take memories like mine, acting as a kind of time capsule, to remind us how special the moment of creation was.

(And somwhere out there, some people who made it to Jimi Hendrix concerts when they were very young are nodding at this.)

I’m here to speak up for you, Michel Hedges. Hm..I see Wikipedia doesn’t link him to percussive fingerstyle. I think I’ll fix that.

Apr 16

A belated response to “A Generation Lost in the Bazaar “

Back in 2012, Poul-Henning-Kamp wrote a disgruntled article in ACM Queue, A Generation Lost in the Bazaar.
javporn
It did not occur to me to respond in public at the time, but someone else’s comment on a G+ thread about the article revived the thread. Rereading my reaction, I think it is still worth sharing for the fundamental point about scaling and chaos.

Continue reading

Apr 11

Penguicon 2015!

I’ve been sent my panel schedule for Penguicon 2015.

Building the “Great Beast of Malvern” – Saturday 5:00 pm

One of us needed a new computer. One of us kicked off the campaign to
fund it. One of us assembled the massive system. One of us installed the
software. We were never all in the same place at the same time. All of us
blogged about it, and had a great time with the whole folderol. Come hear
how Eric “esr” Raymond got his monster machine, with ‘a little help from
his friends’ scattered all over the Internet.

Dark Chocolate Around The World – Sunday 12:00 pm

What makes one chocolate different from others? It’s not just how much
cocoa or sugar it contains or how it’s processed. Different varieties of
are grown in different parts of the world, and sometimes it’s the type of
beans make for different flavor qualities. Join Cathy and Eric Raymond for
a tasting session designed to show you how to tell West African chocolate
from Ecuadorian.

Eric S. Raymond: Ask Me Anything – Sunday 3:00 pm

Ask ESR Anything. What’s he been working on? What’s he shooting?
What’s he thinking about? What’s he building in there?

We do also intend to run the annual “Friends of Armed & Dangerous” party, but don’t yet know if we’re in a party-floor room.

“Geeks With Guns” is already scheduled.

Apr 04

I have been nominated for a John W. Campbell Award

Last Sunday I was informed by email that I have been nominated for the 2015 John W. Campbell award for best new science-fiction writer. I was also asked not to reveal this in public until 4 April.

This is a shame.. I had a really elaborate April Fool’s joke planned where I was going to announce my nomination in the style of a U.S. presidential campaign launch. Lots of talk about a 50-state strategy and my hopes of appealing to swing voters disaffected with both the SJW and Evil League of Evil extremists, invented polling results, and nine yards of political bafflegab.

The plan was to write it so over-the-top that everyone would go “Oh, ha ha, great AFJ but you can’t fool us”…and then, three days later, the other shoe drops. Alas, I checked in with the organizers and they squelched the idea.

It is, of course, a considerable honor to be nominated, and one I am somewhat doubtful I actually deserve. But after considering the ramifications, I have decided not to decline the nomination, but rather to leave the decision on the merits up to the voters.

I make this choice because, even if I myself doubt that my single story is more than competent midlist work, and I want no part of the messy tribal politics in which I seem to have become partly swept up, there is something I don’t mind representing and giving people the opportunity to vote for.

That something is the proud tradition of classic SF, the Golden Age good stuff and its descendants today. It may be that I am among the least and humblest of those descendants, but I think both the virtues and the faults of Sucker Punch demonstrate vividly where I come from and how much that tradition has informed who I am as a writer and a human being.

If you choose to vote for Sucker Punch as a work which, individually flawed as it may be, upholds that tradition and carries it forward, that will make me happy and proud.

Apr 02

My Gitorious projects have moved.

Gitorious – which I preferred to GitHub for being totally open-source – is shutting down sometime in May. I had no fewer than 26 projects on there, including reposurgeon, cvs-fast-import, doclifter, and INTERCAL.

Now they’ve moved. This won’t affect most of my users, as the web pages and distribution tarballs are still in their accustomed locations at catb.org. If you’re a committer on any of these Gitirious repos, of course, the move actually matters.

Temporarily the repositories are on thyrsus.com; here’s the entire list. They may not stay there, but moving them to thyrsus.com was 90% of the work of moving them anywhere else and now I can consider options at my leisure.

Mar 23

On the duties of a geek-cred certification authority

Earlier today I was in an email exchange with a Tier 1 tech support guy at a hardware vendor who makes multiport serial boards. I had had a question in as to whether a particular board supported the Linux TIOCMIWAIT ioctl. Tier 1 guy referred the question to an engineer in their Linux development group, and Tier 1’s reply to me happened to include his email chain with the engineer.

The engineer wrote to Tier 1 “Is that Eric Raymond ‘ESR’? He’s a big deal in open-source circles.” This made me smile, because when I get made that way it usually means the engineer’s going to work rather harder to make me happy than he would for some random. This is helpful to get my work done!

But there is a duty which is the flip side of that privilege, and that’s what I’m here to write about today. Because if you are reading this at all, your odds of becoming a geek-cred certification authority someday are higher than average, and if that happens, it’s better if you consciously understand what you ought to be doing.

Continue reading

Mar 15

Remembering Terry Pratchett

I think Terry Pratchett’s death finally hit home for me today. I’ve been kind of numb about it before now, but today I learned abut this proposal for GNU Terry Pratchett. And as I was commenting about it on G+ I found myself crying.

Here’s a very slightly improved version of what I said on G+. I don’t think I knew Terry well enough to give him the tribute he really deserved, so this will have to do.

Continue reading