I grew up on Internet RFCs, so technical standards issued under licenses that forbid free redistribution offend me. Every such document, whether intentionally or not, is a device for hindering open-source software projects and privileging closed-source developers with big budgets and lawyers to hand.
They offend me even more when (as, for example, when the GPS reporting standard NMEA 0183) the proprietary “standard” is so badly designed that a mob of crack-addled rhesus monkeys could have done a better job – and for this they want me to pay? They offend me the absolute most when the “standard” is distribution-restricted, expensive, badly written, and its topic is a safety-critical technology – so that people could actually die because some jerk wants to collect a trivial amount of secrecy rent on a standard that was crappy to begin with.
Fortunately, there is a way to monkeywrench the organizations that perpetrate this sort of thing – and I’ve spent a substantial part of the last couple of weeks doing exactly that. I’m writing about it here to encourage others to do likewise.
In connection with my ongoing work on GPSD I’ve recently been studying AIS, the marine Automatic Identification System. Since about 2004, maritime authorities worldwide have required every passenger vessel and every cargo ship of more than 300 tons displacement to carry a transmitter that periodically squawks navigational information about the vessel. AIS receivers in the vicinity can pick up this information and make it available for useful things like collision-avoidance systems.
A few AIS receivers have integral map displays meant to be read by a human, but these are largely toys for casual boaters. Most behave a lot like shipboard GPS sensors; they ship AIS sentences out a serial or USB port to be read by navigational systems that will do something useful with them…like, say, automated collision avoidance.
Well…you can do something useful with these sentences if you can read them. And why is that a problem? Because distribution of the core standard for the AIS reporting format (ITU-R 1371) is, you guessed it, restricted under terms that are proprietary and evil. Just like NMEA 0183. But what’s even worse, in this case, is that the International Telecommunications Union is charging secrecy rent on a standard that appears to have been designed mostly by the U.S. Coast Guard. So, a organization that is (a) private, and (b) foreign is charging Americans secrecy rent on a design built with U.S. taxpayer dollars.
When I learned this, I decided I was going to (a) do the open-source world a service, and (b) break their secrecy any way I could. Yeah, I know, some of you are going to tell me this sort of thing is pretty normal in every area of technical standardization other than the Internet; I even knew that, having run into it occasionally before. But this time I reached the “mad as hell and not gonna take-it any more” stage. I’m pushing back.
Here’s what I’ve done. Without looking at the two proprietary standards that bear on AIS (NMEA 4.0 and ITU-R 1371), I’ve collected all the public information on AIS messages and reporting formats into a document which essentially blows the lid off their secrecy. The “without looking” is important; at this point, I don’t want to see them so that I can”t be gigged for copyright infringement over my document. Here it is: AIVDM/AIVDO protocol decoding.
And how do I know this describes reality? Because it describes the masses of AIS sentences helpful people have been sending me. I have a working decoder in the GPSD suite now; it’s not integrated into gpsd itself yet, but you can use it to filter logs full of armored/encoded AIS sentences into readable text. Full support, and test clients that do cool stuff, will follow soon.
Actually, this thing wouldn’t have been trivial to put together even without the legal barriers. The public sources, and conversations with people who have seen NMEA 4.00 and ITU1371, reveal that these documents have been through multiple revisions with substantial changes in them – and the latter one is so densely and badly written that I was able to gather much of what I needed from technical corrigenda published by the U.S. Coast Guard. Also, I document common practice that the official standard doesn’t.
OK, so you should now go and do likewise with some other evil, locked-up-behind-a-paywall standard. We need to teach the ITU and other organizations like it that they will get p0wned and scorned every time they try to collect secrecy rent on these things, so they’ll stop trying.
And don’t worry that you’ll stop good standards from issuing; these groups are generally funded by vendor consortia and governments heavily enough that I can’t see the document fees they collect making any actual difference to them. And that, really, is the final turn of the screw – not only are they locking out open-source projects, they’re doing so for an amount of rent that’s tiny in comparison to the rest of their funding base.
How do I know this? Simple – because it has to be; after all, the Internet Engineering Task Force successfully organizes the same sort of standards work without charging one thin dime for viewing RFCs. So screw you, ITU and NMEA both; your paywalls aren’t just greedy, they’re stupid greedy.
(Oh, and anyone who’s got test sets pairing armored AIVDM sentences with text decodings of same should send them to me so I can add them to my regression test. Don’t bother if it’s just messages types 1-5, though, I’ve got plenty of test sets for those.)