Mar 13

Subversion to GIT Migration: A Tale of Two Gotchas

I’ve been wanting to migrate the GPSD codebase off Subversion to a distributed version control system for many months now. GPSD has a particular reason for DVCS; our developers often have to test GPSD sensors outdoors and aren’t necessarily in range of WiFi when they do it.

GPSD also needs to change hosting sites, for reliability reasons I’ve written about before. Though I’m a fan of Mercurial, I determined that moving to git would give us a wider range of hosting options. Also, git and hg are similar enough to make intermigration really easy – from SVN to either is 90% of the way to the other.

This blog entry records two problems I ran into, and solutions for them. One is that the standard way of converting repos does unfortunate things with tags directories. The second is that the CIA hook scripts for git are stale and rather broken.

Continue reading

Mar 10

On Learning Haskell

I’ve had learning the computer language Haskell on my to-do list for some time. I’m actually stepping up to learn it now, thanks to a temporary lull in my other activities and a vicious cold that has left me disinclined to strenuous work. I may associate Haskell with the taste of zinc gluconate for the rest of my days; both have an astringent and medicinal quality and produce fervent claims of effectiveness about which I am moderately optimistic but which I have not yet personally verified.

Haskell is very obviously a language built by mathematical logicians for mathematical logicians. Category theory lurks behind it in the same way that the lambda calculus lurks behind LISP. The following is an effort to make Haskell’s way of carving up the world intelligible, written partly to clarify my own thoughts.

Continue reading

Mar 07

Sink, the Bismarckian state!

Hmm: On 24th February I wrote, in a comment on this blog:

We’re about to reach the end of the historical era that began when Bismarck first implemented state socialism in the 1880s. The changes will be huge and wrenching; I wouldn’t even put the dissolution of the USA outside the realm of possibility.

Two days later, Mark Steyn writes this in Our own Greek tragedy:

The 20th century Bismarckian welfare state has run out of people to stick it to.

Coincidence or conspiracy? You be the judge.

Mar 04

Greed kills: Why smartphone lock-in will fail and open source win

In a previous post, How smartphones will disrupt PCs, I explained how and why I think small, ultra-portable, general-purpose computers that we’ll think of and use as “smartphones” are going to displace the PC. I promised then to explain why the software of these devices will be open source.

Go read Androids Will Challenge the iPad. It isn’t about smartphones, but the logic that will break the iPhone business model is clearly set out in it for anyone who’s paying attention. What we’re about to see in the smartphone and tablet markets is a repeat of the way the IBM PC shouldered aside the Apple II after 1980. Google’s deliberately slow-balled launch of Android via the G1 was just prelude; it’s with the Motorola Droid, the unlocked Nexus One and the generic Android tablets that the game begins in earnest.

Continue reading

Mar 04

Lies and consequences

In the horrible-inevitability department: Darwin Foes Add Warming to Targets.

My first thought when I read this was: “Creationists opposing AGW hysteria? Wow. So those stupid fucks are good for something after all.”

My second thought was: “It’d be a damn shame if AGW ends up discrediting good science with the public, but if it happens the scientific establishment has only itself to blame for not busting the AGW fraud first.”

In fact, I don’t think that bad outcome is likely to happen, for reasons I explained in Will the AGW fraud discredit science?. But this news story is a warning to all scientists: if you don’t want creationists to get traction, you can’t just treat this as someone else’s problem. You have to clean house. You have tolerated liars and rascals like Phil Jones and Rajendra Pachauri in your midst too long; you need to throw them out.

A diplomatic way for any random professional society to do this would be to demand that all climate science must be held to the strictest standards of methodological scrutiny. All data, including primary un-“corrected” datasets, must be available for auditing by third parties. All modeling code must be published. The assumptions made in data reduction and smoothing must be an explicitly documented part of the work product.

These requirements would kill off AGW alarmism as surely as a bullet through the head. But its credibility is already collapsing; the rising issue, now, is to prevent collateral damage from the scientific community’s failure to insist on them sooner. Every day you delay will strengthen the creationists and the flat-earthers and all the other monsters begotten from the sleep of reason.

UPDATE: and, in a nice bookend, ABC follows the money, suggesting strongly that the scientific establishment has failed to clean house because alarmism is just too damn lucrative.

Mar 02

Cocoa party! Yeah!

First there was the Tea Party movement. Which I’ve been keeping my distance from because while some of the small-government talk appealed to my libertarian instincts, it seemed to have a whiff of grouchy-old-fart social conservatism about it. While I’ve nearly achieved the calendar age required for grouchy-old-fart status, I don’t want anything to do with that mindset thank you very much.

Then there was the Coffee Party movement, which smelled of astroturf and humbug even before the founder was outed as a Democratic political operative and former New York Times staffer (but I repeat myself).

Now, there’s…the Cocoa Party movement! Yeah. Sign me up!

This satire is especially personally funny to me because in sober fact I am a cocoa-drinker by choice. I don’t like coffee, and while I enjoy tea well enough, I’d rather have a big mug of Godiva Dark Chocolate any day – extra strong, piping hot, no sugar or cream but a few shakes of cinnamon for sure.

Continue reading

Mar 02

The Nose of Peace

My regular readers will know that I’m sometimes fascinated by the extent to which mammalian body language crosses species lines — and, indeed, is so well conserved that families as far apart as felids and primates can communicate in ways that are emotionally satisfying to both parties. We take this for granted, but it’s really quite remarkable when you think about it; there’s a repertoire of mammalian gestures that must have maintained consistent social meanings since the earliest mammals in the late Triassic, 200 million years ago, all the way through dizzying changes in size, encephalization and ecological niche.

My wife Cathy reminded me of another one today: the Nose of Peace. “I thought you’d enjoy this photo of a kitten and a young deer exchanging the Nose of Peace” she wrote; our label for it comes from my attempt to describe to a friend over the phone the behavior of two cats who, after quarrelling for days, decided to get along. My friend understood what was intended by “the Nose of Peace” immediately, and that term for it has since spread to several other mutual friends.

Continue reading

Feb 28

Plug for a worthy project: Softbound

When I do my ESR-the-famous-geek road show, I get two kinds of questions: the public ones from my auduence, and the private ones from people who buttonhole me afterwards because they politely don’t want to burden the audience with their particular concerns. Of these, the single most common one is probably “How do I attract developers to my project?”

Last Wednesday I was giving a talk at University of Pennsylvania, which I do once in most years because it’s near me and one of the professors there, Dr. David Matuszek, is an old friend who likes to have me in once a year as a treat for his students.

One of his colleagues approached me with the familiar question. There isn’t a good general answer to this one, because how you attract developers depends in subtle and complicated ways on which developers you actually want. But this particular pitch interested me because it could be part of a significant change in the open-source tools infrastructure that I see coming down the pike.

Continue reading

Feb 25

How smartphones will disrupt PCs

I never bought the hype that laptops were going to obsolesce the conventional desktop PC, nor do I buy today’s version of the hype about netbooks. The reason I didn’t is simple: display and keyboard ergonomics. I use and like a Lenovo X61 Thinkpad happily when traveling, but for steady day-to-day work nothing beats having a big ol’ keyboard and a display with lots of pixels. I have a Samsung 1100DF, 2048×1536, and it may be a huge end-of-lifed boat anchor but I won’t give it up for a flatscreen with lower resolution and less screen real estate.

But now I’m going to reverse myself and predict that smartphones — not today’s smartphones, but their descendants three to five years out, will displace the PC. Here’s what I think my computing experience is going to look like, oh, about 2014:

Continue reading

Feb 24

Marginal Devolution

The recession got personal for me today, when I learned the reason a man I’ve been gaming with at my regular Friday night group hasn’t been showing up lately is because he’s broke and in a homeless shelter. I’m going to ask the Friday night gang for help for the guy — not money, but the job lead he needs much more. And it got me to thinking, about the two people I know who’ve actually been dragged under by the crappy economy.

You might wonder what took me so long. I knew it was getting bad out there, with the nominal unemployment rate at 10% and the actual hitting 17% and 6 applicants for every job. But my friends are mostly university-educated professionals in high-skilled tech jobs — last fired, first hired, and bright enough that if they had to change careers or found their own business they could probably hack it. Except…except for these two, who I’ll call A and B. What’s happening to them is bad. Very bad. And it illustrates a problem that’s going to get worse barring some drastic changes in the system.

Continue reading

Feb 19

On being against torture

I’m against the horror of therdiglob. All right-thinking people should be against therdiglob; civilized societies have long abandoned the practice. Therdiglob makes us look bad in the eyes of the world. If you’re not against therdiglob, someone may do it to you someday.

Eh…what?

The above statement is meaningless, because “therdiglob” is undefined. It has the form of moral indignation, but it is not morally serious. Actually, it’s vacuous except as a way of conveying the speaker’s desire to sound high-minded and morally superior. As read, it is actually faintly ridiculous — indignation without substance, tone without content, pure posturing.

Everything depends on the extensional meaning of “therdiglob”. If “therdiglob” were defined as “flaying the subject alive”, it would be difficult to object to the above. On the other hand, if “therdiglob” were defined as “giving the subject cotton candy”, it would become utterly rather than faintly ridiculous.

Now consider this:

I’m against the horror of torture. All right-thinking people should be against torture; civilized societies have long abandoned the practice. Torture makes us look bad in the eyes of the world. If you’re not against torture, someone may do it to you someday.

Continue reading

Feb 18

How you know you’ve made a difference

I received this a few seconds ago in my mailbox. It’s not especially unusual for me to get wow-you-changed-my-life email, I see two or three in a typical month, but this one is…well, I’ll say a bit more intense than usual. Also, 34 is an unusual age; they tend to be teenagers. Sender’s name masked and one bit turned into a live link. Otherwise unedited, typos and all.

I just finished reading your [Hacker HOWTO] for the second time in as many
days. All of my life I have wanted to be a programmer. I have never
done it. I dealt with the failure of that by downloading cracked
software and telling myself that I was a “hacker.”

When I clicked on the link to your guide, my intention was to get
started on the road towards what I envisioned a hacker to be.

By the end of your guide, I was wallowing in humiliation. I felt…
well basically I felt like a jack ass. I never even knew what a
hacker was. I wasn’t even a cracker, just some poser with no
ambition.

Your guide showed me that any of my few contributions to the cracker
community were meaningless, juvenile ploys for attention with no
progressive merit. They won’t matter in 6 months because I haven’t
‘built’ anything.

After reading your guide, I began to research and really appreciate
the open source community. I saw that the degree of separation
between us wasn’t to huge of a gap and I realized that I could,
through dedication and hard(but fun) work, be part of something that
is growing and giving. A place where individual contributions aren’t
lost in the grand scheme of things, but help shape that scheme.

I am 34 years old and I have no problem admitting to you that your
guide made me feel like a complete ass. It also directed me to an
ideal I will be proud to contribute to.

For that, I thank you Sir.

XXXXX XXXXXX

I replied by quoting “I saw that the degree of separation between us wasn’t to huge of a gap” and said “I’m glad you got that far. Good luck on your journey.”

And that, ladies and gentlemen, is how you know your life has made a difference.

Feb 18

Being an oracle is weird

In some areas of computer science, “oracle” is used as a term for a magic box that delivers solutions to undecidable problems. Or, sometimes more generally for any machine that generates answers by a process you don’t understand. One of the roles I play with respect to the Battle For Wesnoth fantasy game is as an oracle that generates plausible fantasy names on demand.

Here’s a (very lightly edited) transaction of this kind that I took part in a short while ago. There’s something going on behind it that is odder than will be immediately apparent.

Continue reading

Feb 16

Who bears the costs of moral vanity?

A high-ranking Taliban commander is captured in Pakistan, and the (now entirely predictable) dance begins. Says the Guardian:

Mullah Barader has been in Pakistani custody for several days, with US and Pakistani intelligence officials both taking part in interrogations, according to the officials. Though Barack Obama has banned US agencies from using forms of torture such as waterboarding, Pakistani questioning techniques are frequently brutal.

That’s right. Because the American chattering classes have their panties in a bunch about acts of “torture” that don’t do any permanent damage to the victim, Barader is in the hands of Pakistanis who are likely to fuck his shit up the old-school way, with knives and cattle-prods and blowtorches. And yet, this is supposed to count as a moral victory.

Continue reading

Feb 13

When you see a heisenbug in C, suspect your compiler’s optimizer

This is an attempt to throw a valuable debugging heuristic into the ether where future Google searches will see it.

Yesterday, my friend and regular A&D commenter Jay Maynard called me about a bug in Hercules, an IBM360 emulator that he maintains. It was segfaulting on interpretation of a particular 360 assembler instruction. But building the emulator with either -g for symbolic debugging or its own internal trace facility enabled made the bug go away.

This is thus a classic example of heisenbug, that goes away when you try to observe or probe it. When he first called, I couldn’t think of anything helpful. But there was a tickle in the back of my brain, some insight trying to break into full consciousness, and a few minutes later it succeeded.

I called Jay back and said “Turn off your compiler’s optimizer”.

Compiler optimizers take the output stream from some compiler stage and transform it to use fewer instructions. They may operate at the level of serialized expression trees, or of a compiler intermediate representation at a slightly later stage, or on the stream of assembler instructions emitted very late (just before assembly and linking). They look for patterns in the output and rewrite them into more economical patterns.

Optimizer pattern rewrites aren’t supposed to change the behavior of the code in any way other then making it faster and smaller. Unfortunately, proving the correctness of an optimization is excruciatingly difficult and mistakes are easy. Mistaken optimizations that almost always work are, though rare in absolute terms, among the most common compiler bugs.

Optimization bugs have a strong tendency to be heisenbugs. Enabling debugging symbols with -g can change the output stream just enough that the optimizer no longer sees the pattern that triggers the defective rule. So can enabling the conditioned-out code for a trace facility.

When I told Jay this, he reported that Hercules normally builds with -O3, which under GCC is a very aggressive (that is to say somewhat risky) optimization level.

“OK, set your optimizer to -O0,”, I told Jay, “and test. If it fails to segfault, you have an optimizer bug. Walk the optimization level upwards until the bug reproduces, then back off one.”

I knew of this technique because I’ve been in this kind of mess myself more than once – most recently the code for interpreting IS-GPS-200, the low-level bit-serial protocol used on GPS satellite-to-ground radio links. It was compromised by an optimizer heisenbug that was later fixed in GCC 4.0.

This morning Jay left a message in my voicemail confirming that my diagnosis was correct.

I said above that optimizer bugs have a strong tendency to be heisenbugs. If you are coding with an optimizing compiler, the reverse implication is also true, especially of segfault heisenbugs. The first thing to try when you trip over one of these is to turn off your optimizer.

You won’t hit this failure case very often — I’ve seen it maybe three or four times in nearly thirty years of C programming. But when you do, knowing this heuristic can save you many, many hours of grief.

Feb 12

LORAN, we hardly knew ye

There’s been some upset in the blogosphere over the shutdown of the LORAN system of radionavigation beacons. This post at Chicago Boyz is representative (hat tip to Instapundit). The author worries “I’m not totally sure that this was a good decision.” and various commenters are much more emphatic, bemoaning the lack of a backup for GPS.

In my capacity as lead of the GPSD project I’ve been required to become a topic expert on the strengths and weaknesses of GPS and various competitors to it, including LORAN. So here’s the straight scoop: yes, a backup for GPS would be a really good idea. No, LORAN was never plausible as that backup, so angsting about its passing on those grounds is just silly.

Continue reading