This is a consequence of my recent adventures in repository conversion – a detailed discussion of how to do a high-quality lift of a CVS or Subversion repo to DVCS-land, how to make both git and hg users happy, and what sorts of good practices to teach to keep things tidy.
I just released reposurgeon 1.2 and am continuing to develop the tool. In order to test some of the newer features, I’m looking for repository conversions to do. If you run an open-source project that is still using CVS or Subversion, or some odd non-distributed VCS, I may be willing to lift it to git for you (and from git to any other DVCS you might prefer is a pretty small step). Details of this offer follow; limited time only, first come, first served.
(Why have me do it? Well…especially for older projects with a complex revision history, it’s a messy and daunting job. The tools are somewhat flaky, the difference between a sloppy conversion and a good one is significant, and good conversions require experience and judgment.)
I upgraded to Ubuntu 11.04 a week or so back in order to get a more recent version of SCons. 11.04 dropped me into the new “Unity” GNOME interface. There may be people in the world for whom Unity is a good idea, but none of them are me. The look is garish and ugly, and it takes twice as many clicks as it did before to get to an application through their supposedly “friendly” interface as it did in GNOME Classic. No, dammit, I do not want to text-search my applications to call one up!
But the real crash landing was when I found out that the Unity dock won’t let you manage two instances of the terminal emulator separately. Oh, you can click the terminal icon twice and get two instances, and even minimize them separately, but they’re tied to the same dock icon when minimized. If you click it to unminimize, both pop back up. That did it; clearly Unity is a toy, not intended for anybody doing serious work.
I was miserable until I found out how to fall back to GNOME Classic. But then a few days later I upgraded to 11.10 and my real troubles began.
I have interesting friends. Two of them, who shall remain nameless because it is possible they have let slip to me information that is technically classified, recently told me the best GPSD deployment story since the robot submarine.
So, Friend A says “Hey, Eric, did you know GPSD is used in the on-board nav system of the Abrams tank?” Friend A is in a position to know, because Friend A has done troubleshooting of that nav system – once, over the phone with a tank actually in combat in Iraq. It seems GPSD is used as part of IFF (identification friend or foe) and without that module they are at unpleasant risk of heaving a shell at a friendly. (And no, I am assured the bug was not in GPSD itself.)
Protocol transitions are hard.
Yeah, I know the experienced software developers reading that are thinking “Well, duh!“. Shut up already, I’m venting here. I’ve just spent the better part of two years – actually, if you include design time and false starts it could be closer to five years – designing a new application protocol for my gpsd service daemon, implementing it, getting it deployed, dealing with problems and course-correcting.
For those of you in the cheap seats, an “application protocol” is a kind of language that a program uses to communicate with other programs. Normally humans never see these, but there’s one big example that non-geeks have often seen bits of. HTML, the Web’s markup language, is an application protocol. Designing these is not easy. There are difficult issues and tradeoffs around flexibility, economy, expressiveness, and extensibility for uses we haven’t imagined yet.
When you have a whole bunch of programs written by different people communicating with a particular application protocol, changing that entire ecology to use a different one is not easy. A good analogy in the physical world is the difficulty of changing an entire country’s railroad gauge. The long-term benefits may be huge, but the short-term cost in capital and service disruption is daunting.
Here’s hoping I’ve figured out how to extract a public permalink this time.
Normally I post essay-length stuff here, but I’ve weighed in on a G+ policy dispute and thought this post should be there. Comments there too please.
One of my commenters pointed me at an article by John Sonmez over at ElegantCode, Why Software Development Will Never be Engineering. The article makes one very shrewd, well-argued point, but then disappointingly fails to build on it. Read it and see if you spot the problem before I analyze.
Technological change has a tendency to look inevitable in retrospect – “It steam-engines when it’s steam-engine time.” Likely this is true in many cases, but I often think we underestimate the alarming degree of contingency lurking behind ‘inevitable’ developments. To illustrate this point, I’m going to sketch an all-too-plausible alternate history in which the World Wide Web never happened.
Someone on the gpsd-users list asked:
I was just wondering why gpsd doesn’t have a configuration file in /etc/gpsd.conf, like most other Unix/Linux software?
Because configuration files are evil, and not to be countenanced unless they become an absolutely necessary evil. Which in gpsd’s case is not yet, and I sincerely hope not ever.
I’ve gotten used to being cited in computer science and software engineering papers over the last decade, but here’s a new one. Today I read a draft in which I and the GPSD project get cited a bunch of times and it’s – er – not about open source. It’s about Marine AIS in disaster management. Broadening my deepwater horizons, as it were.
Some years back I wrote a book titled The Art of Unix Programming. My goal in that book was to convey the Zen of Unix to today’s generations of eager young Linux and *BSD programmers. In the spirit of that book, I feel impelled to point out out a program I’ve recently learned as a striking, near-perfect example of Unix style in the modern day. rsnapshot, you’re doing it right!
My post SCons is full of win today triggered some interesting feedback on scaling problems in SCons. In response to anecdotal assertions that SCons is unusably slow on large projects, I argued that build systems in general must scale poorly if they are to enforce correctness. Subsequently, I received a pointer to a very well executed empirical study of SCons performance to which I replied in the same fashion.
In this post, I intend to conduct a more detailed analysis of algorithmic requirements and complexity in an idealized build system, and demonstrate the implied scaling laws more rigorously. I will also investigate tradeoffs between correctness and performance using the same explanatory framework.
I’ve been thinking recently about writing a shared-memory export for gpsd. The JSON-over-sockets client interface we have is very powerful and flexible, but more than is needed when network access to the server is not required. For embedded deployments, in particular – it would be useful to have a lower-overhead way of shipping results to clients.
Consequently, I’ve been thinking about coherence techniques for shared memory. In this particular case, we have one writer (gpsd) and multiple readers (the application clients). Updates to the shared-memory segment are long enough that writes aren’t guaranteed atomicity. It is permissible for a client to miss an update if it’s not inspecting the segment frequently enough, but required that after a read from the segment the client can always tell when it has a coherent update (as opposed to having read the segment while a write is in progress).
The obvious way to ensure update coherence would be with a semaphore. But a technique that is non-blocking and wait-free would be preferable. I have invented a method I call “bookend consistency”. I present it here for public critique, also because I’m curious whether any of my commenters can identify it with a known, published algorithm. It was inspired by a vague, distant memory of pioneering work by Butler Lampson on lock-free algorithms.
A very curious thing happened with GPSD this week. In fact it’s so odd I’m still having trouble believing it. In software engineering we often have trouble getting seemingly simple things to work reliably. How does one react when an incredibly complex, fragile piece of bit-twiddling code works – perfectly – after six years without real-world testing, during which the surrounding architecture underwent such massive changes that any rational person would have expected the feature to bit-rot into garbage?
No, really, this one is weird. Let me unfold to you the strange tale of The RTCM2 Analyzer That Shouldn’t Have Worked. Really. At All.
Some months ago I wrote (in Flattening the Smartphone Market) about the real significance of the Android 2.2 announcement. That was the moment that Google made clear that it intended to take control of the smartphone feature list from the cell carriers. Subsequently, carrier-loaded crapware and suppression of features like hotspot and tethering have been in decline under market pressure. The release of the T-Mobile G-2 and the Samsung Galaxy S (marketed as “the pure Google experience”) have been indicators of this trend.
I should have added that 2.2 takes control of the smartphone feature list away from handset vendors as well. A leak by someone claiming to be a T-mobile employee in the know alleged that Samsung has been dragging its feet on 2.2 upgrades for the Samsung Vibrant, hoping customers will upgrade to the Vibrant 4G in order to get the 2.2 that ships with it. Now comes word that Samsung has folded under pressure from the maneuver and announced an OTA update schedule for 2.2 on the Vibrant.