Apr 27

My open-source project is cooler than yours

Yes, GPSD is way cooler than your project. You know how I can tell?

Because my latest feature request is from a scientist who wants to use GPSD as part of the control software for an autonomous robot submarine. That’s how I can tell.

No word yet on whether the robot submarine will have a frickin’ laser mounted on its frickin’ head. But I’m hoping.

Apr 19

Incentives to be Open

A correspondent pointed me at a paper by Carliss Baldwin and Eric von Hippel, Modeling a Paradigm Shift: From Producer Innovation to User and Open Collaborative Innovation, which builds on my papers in some interesting ways. Here’s one of the money quotes:

Building on arguments of Ghosh (1998), Raymond (1999), and von Hippel and von
Krogh (2003), Baldwin and Clark (2006 b) showed formally that, if communication costs are low
relative to design costs, then any degree of modularity suffices to cause rational innovators that
do not compete with respect to the design being developed to prefer collaborative innovation
over independent innovation. This result hinges on the fact that the innovative design itself is a
non-rival good: each participant in a collaborative effort gets the value of the whole design, but
incurs only a fraction of the design cost.

Continue reading

Apr 08

IBM: Now trying to dig out…

IBM has issued another statement on the TurboHercules imbroglio. This one is reported by the Linux Foundation, but comes from Dan Frye. Dan Frye heads IBM’s Linux Technology Center and was actually at the top of my mental shortlist of likely voices of sanity over there. (Full disclosure: Dan kept me supplied with IBM Thinkpads for a couple of years as a thank-you gesture.)

The good news is that Dan says IBM will stand by the letter of its 2005 pledge. Furthermore, the second sentence of Dan’s pledge leaves no room for doubt that Hercules is a covered project. This is in flat contradiction to whatever brainless droid the Wall Street quoted yesterday on IBM reserving some right to decide that Hercules is ineligible. It also contradicts the previous implication that IBM is prepared to go to court over those two patents.

The bad news is that Dan leaves open the possibility that IBM may sue over the other close to 200 patents. I think it’s important not to overreact to this; his statement was clearly immediate damage control rather than a final ukase. The effect is that IBM now looks as though somebody with a clue has woken up to how much reputation damage their previous blunders have done them.

My guess is that the matter is now being debated (or soon will be) at a level higher than Frye or either of the pair of clowns who had previously made IBM’s posture look so very wrong. This might, still, blow over.

But IBM should hear this, loud and clear: the letter of your pledge is not enough. You cannot simultaneously hold yourself forth as an ally of open source and conduct patent warfare against an open-source project. Betrayal stings; we won’t abide it, and I wouldn’t argue that we should even if I thought I could win that argument. If you try to have this both ways, you will enrage the community more than if you had been a frank enemy all along.

Understand that our intransigence on this score this is only partly on behalf of Hercules itself. We detest the patent weapon, even in the hands of a sometime ally, because we fear it so much. What IBM does in this matter will set a precedent for the behavior of others; if IBM chooses to set the wrong precedent, it will make enemies of us.

Apr 07

IBM: Digging itself in deeper

Yesterday I blogged about the escalating dispute between IBM and TurboHercules SAS. I said, and will repeat now, that the central issue for the open-source community in this matter is not the antitrust allegations, but rather the fact that IBM has raised a patent threat alleging that Hercules violates its intellectual property. And especially, that IBM in doing so has cited two patents that were explicitly listed in its 2005 pledge to the open-source community.

IBM has now made matters worse. Much worse.

Continue reading

Apr 06

IBM: Back to the Bad Old Days?

Sadly, Florian Mueller’s scream of outrage (IBM breaks the taboo and betrays its promise to the FOSS community is not an April Fool’s joke. IBM has done what it swore not to in 2005 – picked up the patent weapon and aimed it to block an open-source project.

I was thoroughly briefed on this about ten days ago by Jay Jurata, a lawyer working for CCIA (Computer and Communications Industry Association), which is even now bringing an antitrust action against IBM over the matter. Assisting him was Jay Maynard, an Armed & Dangerous regular who happens to be the lead of a project called Hercules.

Hercules is an open-source emulator for IBM mainframes. Words cannot easily describe my degree of bogglement the first time Jay brought it up an a Linux laptop in my presence and I saw the unmistakable arcane runes of a 360 boot sequence – and in an old-school band-printer-style font, too. Now, after 11 years during which IBM nodded when its own employees used and contributed to Hercules, Big Blue has brought down the hammer.

Continue reading

Mar 31

Autotools must die

Me, on the GPSD mailing list:

Once upon a time, I did not hate autotools. Yea verily, it was the morning of the world and all things (even autotools) seemed fresh and new). I’d say this innocence lasted until about, oh, 1995.

But autotools was a kluge. And it did accrete kluges and crocks around it, adding layers of complexity until it became sore difficult to tell which end was up. And lo, it became a festering pile of special cases and obscure semi-documented rules, leading to a combinatorial explosion of unplanned interactions and obscure lossage.

Like, say, the fact that our make check insists on running gps-makeregress twice and in spite of being a genuine autotools expert with fifteen years of experience bear-wrestling the sorry fscker I cannot figure out why it is doing this.

Oh, there’s a reason, all right. And if I were willing to quintuple the three hours I just spent poking at our build setup I’m sure I could find it. But that was three hours wasted as far as solving any real problem was concerned. Life is too short for such nonsense.

Autotools has reached the Chandrasekhar mass limit of software and become a black hole of suck that consumes endless hours of bright peoples’ time without any good result. It is well past time for it to die.

Mar 24

Scenes from the Life of a System Architect

I’ve been doing some heavy work on the core code of gpsd recently, and realized it would be a good idea to explain the whys and wherefores to my co-developers on the project. After I wrote my explanation and reread it, I realized I had managed to generate something that might be relatively accessible, and perhaps even interesting, to people who aren’t intimate with the GPSD codebase.

I guess I’m aiming this at junior programmers and particularly curious non-programmers. It’s a slice of what software systems design — the thing that project leads and architects do — is like in the real world, where the weight of history is often as pressing as today’s requirements list. I think this note shows an example of doing it right, or at least not getting it too badly wrong.

If you find the technicalese in here difficult, it may be useful to refer back to some of my previous posts about this project:

GPSD and Code Excellence

GPSD-NG: A Case Study in Application Protocol Evolution

Why GPSes suck, and what to do about it

Continue reading

Mar 13

Subversion to GIT Migration: A Tale of Two Gotchas

I’ve been wanting to migrate the GPSD codebase off Subversion to a distributed version control system for many months now. GPSD has a particular reason for DVCS; our developers often have to test GPSD sensors outdoors and aren’t necessarily in range of WiFi when they do it.

GPSD also needs to change hosting sites, for reliability reasons I’ve written about before. Though I’m a fan of Mercurial, I determined that moving to git would give us a wider range of hosting options. Also, git and hg are similar enough to make intermigration really easy – from SVN to either is 90% of the way to the other.

This blog entry records two problems I ran into, and solutions for them. One is that the standard way of converting repos does unfortunate things with tags directories. The second is that the CIA hook scripts for git are stale and rather broken.

Continue reading

Mar 10

On Learning Haskell

I’ve had learning the computer language Haskell on my to-do list for some time. I’m actually stepping up to learn it now, thanks to a temporary lull in my other activities and a vicious cold that has left me disinclined to strenuous work. I may associate Haskell with the taste of zinc gluconate for the rest of my days; both have an astringent and medicinal quality and produce fervent claims of effectiveness about which I am moderately optimistic but which I have not yet personally verified.

Haskell is very obviously a language built by mathematical logicians for mathematical logicians. Category theory lurks behind it in the same way that the lambda calculus lurks behind LISP. The following is an effort to make Haskell’s way of carving up the world intelligible, written partly to clarify my own thoughts.

Continue reading

Mar 04

Greed kills: Why smartphone lock-in will fail and open source win

In a previous post, How smartphones will disrupt PCs, I explained how and why I think small, ultra-portable, general-purpose computers that we’ll think of and use as “smartphones” are going to displace the PC. I promised then to explain why the software of these devices will be open source.

Go read Androids Will Challenge the iPad. It isn’t about smartphones, but the logic that will break the iPhone business model is clearly set out in it for anyone who’s paying attention. What we’re about to see in the smartphone and tablet markets is a repeat of the way the IBM PC shouldered aside the Apple II after 1980. Google’s deliberately slow-balled launch of Android via the G1 was just prelude; it’s with the Motorola Droid, the unlocked Nexus One and the generic Android tablets that the game begins in earnest.

Continue reading

Feb 13

When you see a heisenbug in C, suspect your compiler’s optimizer

This is an attempt to throw a valuable debugging heuristic into the ether where future Google searches will see it.

Yesterday, my friend and regular A&D commenter Jay Maynard called me about a bug in Hercules, an IBM360 emulator that he maintains. It was segfaulting on interpretation of a particular 360 assembler instruction. But building the emulator with either -g for symbolic debugging or its own internal trace facility enabled made the bug go away.

This is thus a classic example of heisenbug, that goes away when you try to observe or probe it. When he first called, I couldn’t think of anything helpful. But there was a tickle in the back of my brain, some insight trying to break into full consciousness, and a few minutes later it succeeded.

I called Jay back and said “Turn off your compiler’s optimizer”.

Compiler optimizers take the output stream from some compiler stage and transform it to use fewer instructions. They may operate at the level of serialized expression trees, or of a compiler intermediate representation at a slightly later stage, or on the stream of assembler instructions emitted very late (just before assembly and linking). They look for patterns in the output and rewrite them into more economical patterns.

Optimizer pattern rewrites aren’t supposed to change the behavior of the code in any way other then making it faster and smaller. Unfortunately, proving the correctness of an optimization is excruciatingly difficult and mistakes are easy. Mistaken optimizations that almost always work are, though rare in absolute terms, among the most common compiler bugs.

Optimization bugs have a strong tendency to be heisenbugs. Enabling debugging symbols with -g can change the output stream just enough that the optimizer no longer sees the pattern that triggers the defective rule. So can enabling the conditioned-out code for a trace facility.

When I told Jay this, he reported that Hercules normally builds with -O3, which under GCC is a very aggressive (that is to say somewhat risky) optimization level.

“OK, set your optimizer to -O0,”, I told Jay, “and test. If it fails to segfault, you have an optimizer bug. Walk the optimization level upwards until the bug reproduces, then back off one.”

I knew of this technique because I’ve been in this kind of mess myself more than once – most recently the code for interpreting IS-GPS-200, the low-level bit-serial protocol used on GPS satellite-to-ground radio links. It was compromised by an optimizer heisenbug that was later fixed in GCC 4.0.

This morning Jay left a message in my voicemail confirming that my diagnosis was correct.

I said above that optimizer bugs have a strong tendency to be heisenbugs. If you are coding with an optimizing compiler, the reverse implication is also true, especially of segfault heisenbugs. The first thing to try when you trip over one of these is to turn off your optimizer.

You won’t hit this failure case very often — I’ve seen it maybe three or four times in nearly thirty years of C programming. But when you do, knowing this heuristic can save you many, many hours of grief.

Dec 04

GPSD and Code Excellence

There’s a wonderfully tongue-in-cheek project called the The Alliance for Code Excellence (“Building a better tomorrow — one line of code at a time.”) that sells Bad Code Offset certificates. They fund open source projects to produce good code that will, in theory, offset all the bad code out there and mitigate the environmental harm it does. They’ve asked software authors to write essays on how their projects drive out bad code, offering $500 dollar prizes.

I sat down to write an essay about GPSD in the same vein of high drollery as the Alliance’s site, then realized that GPSD actually has a serious case to make. We really do drive out bad code, in both direct and indirect ways, and we supply examples of good practice for emulation.

Continue reading

Nov 15

The pragmatics of webscraping

Here’s an amplification of my previous post, Structure Is Not Meaning. It’s an except from the ForgePlucker HOWTO on writing code to web-scrape project data out of forge systems.

Your handler class’s job is to extract project data. If you are lucky, your target forge already has an export feature that will dump everything to you in clean XML or JSON; in that case, you have a fairly trivial exercise using BeautifulStoneSoup or the Python-library JSON parser and can skip the rest of this section.

Usually, however, you’re going to need to extract the data from the same pages that humans use. This is a problem, because these pages are cluttered with all kinds of presentation-level markup, headers, footers, sidebars, and site-navigation gorp — any of which is highly likely to mutate any time the UI gets tweaked.

Here are the tactics we use to try to stay out of trouble:

1. When you don’t see what you expect, use the framework’s self.error() call to abort with a message. And put in lots of expect checks; it’s better for a handler to break loudly and soon than to return bad data. Fixing the handler to track a page mutation won’t usually be hard once you know you need to – and knowing you need to is why we have regression tests.

2. Use peephole analysis with regexps (as opposed to HTML parsing of the whole page) as much as possible. Every time you get away with matching on strictly local patterns, like special URLs, you avoid a dependency on larger areas of page structure which can mutate.

3. Throw away as many irrelevant parts of the page as you can before attempting either regexp matching or HTML parsing. (The most mutation-prone parts of ppsages are headers, footers, and sidebars; that’s where the decorative elements and navigation stuff tend to cluster.) If you can identify fixed end strings for headers or fixed start strings for footers, use those to trim (and error out if they’re not there); that way you’ll be safe even if the headers and footers mutate. This is what the narrow() method in the framework code is for.

4. Rely on forms. You can assume you’ll be logged in with authentication and permissions to modify project data, which means the forge will display forms for editing things like issue data and project-member permissions. Use the forms structure, as it is much less likely to be casually mutated than the page decorations.

5. When you must parse HTML, BeautifulSoup is available to handler classes. Use it, rather than hand-rolling a parser, unless you have to cope with markup so badly malformed that it cannot cope.

Actual field experience shows that throwing out portions of a page that are highly susceptible to mutation is a valuable tactic. Also, think about where in the site a page lives. Entry pages and other highly visible ones tend to get tweaked the most often, so the tradeoffs push you towards peephole methods and not relying on DOM structure. Deeper in the site , especially on pages that are heavily tabular and mostly consist of one big form, relying on DOM structure is less risky.

Nov 05

Structure Is Not Meaning

So, I announce ForgePlucker, and within a day I’ve got some guy from Y Combinator sneering at me for using regular expressions to parse HTML. Says it’s “crappy code”. The poor fool…he has fallen victim to a conceptual trap which I, fortunately, learned to avoid decades ago. I could spout a freshet of theory about it, but instead I’m just going to utter a maxim: Never confuse structure with meaning.

Continue reading

Nov 04

Announcing ForgePlucker

I’ve been strongly hinting in recent blog entries that I planned to do something concrete about the data-jail problems of present open-source hosting sites. Because I believe in underpromising and overperforming, I decided at the outset not to announce a project until I could not only show working code, but code with wide enough coverage to make it crystal-clear that the project goals are achievable with a relatively modest amount of effort.

That time has arrived. I am very pleased to announce ForgePlucker, a project aimed at developing project-state extractor software for backup, offline analysis, and (eventually) re-importation. The proof-of-concept code can extract complete issue-tracker state from Berlios, Gna!, or Savane — and issue trackers are probably the hardest part of the job. I expect extraction of repository histories and developer permissions tables to be easier. Extraction of mailing-list state is probably a bit trickier than either of those, but doable.

Continue reading

Oct 29

The future of software forges

I’m still not going to talk about my attack on the forge infrastructure problems quite yet; the software is coming along nicely, but I intend to announce only after it handles its fourth forge type (yes, that was a tease). But I will say this: I now think I know what the future of forges looks like. It’s called Roundup, and it is astonishingly elegant and potentially more powerful than anything out there. Anything, not excluding the clever decentralized systems like Fossil or Bugs Everywhere.

Here are the big wins:

1. Mailing lists, issue trackers, and online forums unify into *one* message queue that can be filtered in various ways.

2. Scriptable via XML-RPC or an email responder ‘bot.

3. Small base system with good extensibility – just three base classes (User. Msg, File) and the ability to define new classes. ‘Issue’ is a class built on top of these.

4. Arbitrary attributes per issue is basically free, with baked-in support for defining controlled vocabularies.

5. There’s a uniform way, called “designators”, for messages and other objects to refer to each other in text.

6. Small, clean implementation written in Python.

There are some things it needs, though… (Read the Roundup design document before continuing.)

Continue reading

Oct 26

Hacker superstitions about software licensing

Hackers have a lot of odd superstitions about software licensing. I was reminded of this recently when a project maintainer asked me whether he needed to get a sign-off from each and every one of his contributors before switching from Apache v1 to Apache v2. Here’s what I told him:

My opinion is this. Under U.S. law — and I believe European codes are not different in this respect, because both are controlled by the Berne convention — a license change on a collection is grounds for protest or legal action only if the rights of the contributors are materially affected by the change. That is, a court would have to be persuaded that the change caused a monetary loss or at least damage to a contributor’s public reputation. If there is no such possibility, then there is no harm and no grounds for complaint.

Continue reading