I’ve written a tool to assist intrepid code archeologists trying to comprehend the structure of ancient codebases. It’s called ifdex, and it comes with a backstory. Grab your fedora and your bullwhip, we’re going in…
I’ve been pretty quiet on what’s going on with NTPsec since posting Yes, NTPsec is real and I am involved just about a month ago. But it’s what I’m spending most of my time on, and I have some truly astonishing success to report.
The fast version: in three and a half months of intensive hacking, since the NTP Classic repo was fully converted to Git on 6 June, the codebase is down to 47% of its original size. Live testing on multiple platforms seems to indicate that the codebase is now solid beta quality, mostly needing cosmetic fixes and more testing before we can certify it production-ready.
Here’s the really astonishing number…
shimulate, vt.: To insert a shim into code so it simulates a desired standardized ANSI/POSIX facility under a deficient operating system. First used of implementing clock_gettime(2) under Mac OS X, in the commit log of ntpsec.
I checked first use by Googling.
When I built the Great Beast of Malvern, it was intended for surgery on large repositories. The specific aim in view was to support converting the NetBSD CVS to git, but that project is stalled because the political process around NetBSD’s decision about when to move seems to have seized up. I’ve got the hardware and software ready when they’re ready to move.
Now I have another repo-conversion job in the offing – and it does something I thought I’d never see. The working set exceeds 32GB! For comparison, the working set of the entire NetBSD conversion tops out at about 18GB.
What, you might well ask, can possibly have a history that huge? And the answer is…GCC. Yes, they’re looking to move from Subversion to git. And this is clearly a job for the Great Beast, once the additional 32GB I just ordered from Newegg arrives.
So here’s how my day went….
I started off trying to convert a legacy manual page to asciidoc. Found that pandoc (which could be the target of a whole separate rant, because it totally sucks at translating anything with tables in it) won’t do that.
But it will convert DocBook to asciidoc. OK, so I can use my doclifter tool to convert the manual page to DocBook, then DocBook to asciidoc via pandoc I try this, and doclifter promptly loses its cookies.
Huh? Oh, I see an [nt]roff request in there I’ve never seen before. Must fix doclifter to handle that. Hack hack hack – done. Push the fix, think “I ought to ship a doclifter release”
I look at the doclifter repo. I see that the commit graph has an ugly merge bubble in it from where I forgot a –rebase switch when I was pulling last. It’s the pointless kind of bubble where someone else’s patch commutes with mine so the history might as well be linear and easier to read.
You know, before I ship it…I was planning to move that repo from the downstairs machine to GitLab anyway, I might as well fix that bubble in the process….
Now how do I do that? Hm, yeah, this patch-replay sequence will do it. I ought to can that procedure into a script because I’m doubtless going to have to do it again. (I hates pointless merge bubbles, I hates them forever…) Script done. Hm. I should publish this.
OK, throw together a man page and README and Makefile. Oh, right, I need a project logo for the web page. What should it be? Into my mind, unbidden, enters the image of the scrubbing bubble animated characters from an advertising campaign of my childhood. Win!
I google for images, find one, and GIMP out a suitable piece within individual scrubbing bubble in it. Snrk, yeah, that’s funny. Scale it to 64×64 and go.
Funny logo achieved.
OK, version 1.0 of git-debubble gets published.
git-debubble gets applied to the doclifter repo,
…which I then publish to GitLab.
Now I can convert the manual page to DocBook XML…
…which can then be converted to asciidoc.
I have a lot of days like this.
I think there ought to be a term for a sequence of dependency fulfillments that is a lot like yak shaving, except that something useful or entertaining independently of the original task gets emitted at each stage.
Welcome to my first new-project release of the year, git-weave. It’s a polished and documented version of the script I used to reconstruct the early history of INTERCAL five years ago – see Risk, Verification, and the INTERCAL Reconstruction Massacree for the details on that one.
git-weave can be used to explode a git repository into a sequence of per-commit directory trees accompanied by a metadata file describing parent-child linkage, holding committer/author/timestamps/comment metadata, and carrying tags.
Going in the other direction, it can take the same sequence of trees plus metadata file and reconstruct the live repository. Round-tripping is lossless.
What it’s really useful for is reconstructing a partial but useful ancient history of a project from before it was put under version control. Find its release archives, synthesize a metadata file, apply this tool, and you get a repository that can easily be glued to the modern, more continuous history.
Yes, you only get a commit for each release tree or patch you can dig up, but this is better than nothing and often quite interesting.
Nifty detail: the project logo is the ancient Egyptian hieroglyph for a weaver’s shuttle.
Now that the Universe Splitter is out, it might be that a lot more people are going to trip over the word “mu” and wonder about it. Or it might be the word only occurs in the G+ poll about Universe Splitter – I don’t know, I haven’t seen the app (which appears to be a pretty good joke about the many-wolds interpretation of quantum mechanics) itself.
In any case, the most important thing to know about “mu” is that it is usually the correct answer to the question “Have you stopped beating your wife?”. More generally, it is a way of saying “Neither a yes or no would be a correct answer, because your question is incorrect”,
But the history of how it got that meaning is also entertaining.
It’s Penguicon 2015 at the Westin in Southfield, Michigan, and time for the 2015 Friends of Armed & Dangerous party.
9PM tonight, room 314. Nuclear ghost-pepper brownies will be featured.
I’ve been sent my panel schedule for Penguicon 2015.
Building the “Great Beast of Malvern” – Saturday 5:00 pm
One of us needed a new computer. One of us kicked off the campaign to
fund it. One of us assembled the massive system. One of us installed the
software. We were never all in the same place at the same time. All of us
blogged about it, and had a great time with the whole folderol. Come hear
how Eric “esr” Raymond got his monster machine, with ‘a little help from
his friends’ scattered all over the Internet.
Dark Chocolate Around The World – Sunday 12:00 pm
What makes one chocolate different from others? It’s not just how much
cocoa or sugar it contains or how it’s processed. Different varieties of
are grown in different parts of the world, and sometimes it’s the type of
beans make for different flavor qualities. Join Cathy and Eric Raymond for
a tasting session designed to show you how to tell West African chocolate
Eric S. Raymond: Ask Me Anything – Sunday 3:00 pm
Ask ESR Anything. What’s he been working on? What’s he shooting?
What’s he thinking about? What’s he building in there?
We do also intend to run the annual “Friends of Armed & Dangerous” party, but don’t yet know if we’re in a party-floor room.
“Geeks With Guns” is already scheduled.
Last Sunday I was informed by email that I have been nominated for the 2015 John W. Campbell award for best new science-fiction writer. I was also asked not to reveal this in public until 4 April.
This is a shame.. I had a really elaborate April Fool’s joke planned where I was going to announce my nomination in the style of a U.S. presidential campaign launch. Lots of talk about a 50-state strategy and my hopes of appealing to swing voters disaffected with both the SJW and Evil League of Evil extremists, invented polling results, and nine yards of political bafflegab.
The plan was to write it so over-the-top that everyone would go “Oh, ha ha, great AFJ but you can’t fool us”…and then, three days later, the other shoe drops. Alas, I checked in with the organizers and they squelched the idea.
It is, of course, a considerable honor to be nominated, and one I am somewhat doubtful I actually deserve. But after considering the ramifications, I have decided not to decline the nomination, but rather to leave the decision on the merits up to the voters.
I make this choice because, even if I myself doubt that my single story is more than competent midlist work, and I want no part of the messy tribal politics in which I seem to have become partly swept up, there is something I don’t mind representing and giving people the opportunity to vote for.
That something is the proud tradition of classic SF, the Golden Age good stuff and its descendants today. It may be that I am among the least and humblest of those descendants, but I think both the virtues and the faults of Sucker Punch demonstrate vividly where I come from and how much that tradition has informed who I am as a writer and a human being.
If you choose to vote for Sucker Punch as a work which, individually flawed as it may be, upholds that tradition and carries it forward, that will make me happy and proud.
I just dealt with an attempt to interview me by email that checked off pretty nearly every possible mistake in the form. As a public service, I now reproduce the advice I gave after bailing out halfway through the list of questions.
Every once in a while I have an experience that causes me to meditate on the question of how much variation in human behavior is genetically driven. I’ve written before about my gradually increasing awareness that I am genetically designed to enjoy combat. I had another experience something like this last night in a completely different domain – learning to solder.
For my blog regulars, first video from the Great Beast build.
Thanks once again to everyone who donated money to make this happen. I think you’ll see it was well spent. And there’s more video coming.
Yesterday I realized, quite a few years after I should have, that I have never identified in public where I got the seed of the idea that I developed into the modern economic theory of open-source software – that is, how open-source “altruism” could be explained as an emergent result of selfish incentives felt by individuals. So here is some credit where credit is due.
Now, in general it should be obvious that I owed a huge debt to thinkers in the classical-liberal tradition, from Adam Smith down to F. A. Hayek and Ayn Rand. The really clueful might also notice some connection to Robert Trivers’s theory of reciprocal altruism under natural selection and Robert Axelrod’s work on tit-for-tat interactions and the evolution of cooperation.
These were all significant; they gave me the conceptual toolkit I could apply successfully once I’d had my initial insight. But there’s a missing piece – where my initial breakthrough insight came from, the moment when I realized I could apply all those tools.
Also included, my nonfiction analysis of the effect of battlefield lasers on military airpower, a development likely to transform warfare in the coming century as radically as the deployment of automatic weapons did around the beginning of the last one.
The good folks from TekSyndicate showed up yesterday with a pile of parts and did final assembly of the Beast in my dining room. A&D regular John Bell remoted in last night to finish the setup. I’m actually blogging on it now as the last of my work environment transfers over from the old snark.
What a beautiful machine it is! The interior of the NZXT case is even more impressive live than it is in photos. It runs whisper-quiet.
During the next several hours we’ll be filming documentary and interview footage. I’ll announce here when and where the video is available.
UPDATE: I have now had a chance to profile performance on some of the benchmark repos. I’m seeing speedups of between a factor of three (on Emacs CVS) and twenty (on groff CVS). The entire NetBSD src repository, 288K commits and 37GB of content, converts in 22 minutes.
A lot of U.S. economic policy is distorted by the belief that manufacturing jobs are a magic bullet against declining incomes. Manufacturing’s false promise of a decent payday punctures that illusion.
One of the dumb, predictable responses to articles like this is “We need a stronger union movement”. Sorry, but no. Declining manufacturing wages aren’t an effect of the weakening of unions and can’t be reversed by strengthening them. Explanation follows.