I’m the lead of the GPSD project, a service daemon that monitors GPS receivers on serial or USB ports and provides TPV (time-position-velocity) reports in a simple format on on a well-known Internet port. GPSD makes this job looks easy. But it’s not — oh, it’s decidedly not — and thereby hangs an entertaining tale of hacker ingenuity versus multiple layers of suck.
Away back in the dark and backward abysm of time when GPS technology was first being made generally available (1993), only military-grade receivers were sensitive enough to use it where there were things like buildings and trees partly blocking the sky view. The first civilian customers to actually find a use for it were people messing about in boats. Thus it came to pass that the manufacturers of marine navigation systems were the first civilians to grapple with the question of how a GPS receiver should report TPV information over a wire to a navigational computer.
Our first layer of suck begins with the National Marine Electronics Association, or NMEA. They wrote a standard describing a protocol for GPSes reporting over serial ports called NMEA 0183 which, despite being a technical expert in the field, I’ve never dared to look at. The reason is that they made it proprietary and expensive, and their lawyers have been known to threaten legal action against people who quote it on the net.
To add injury to insult, NMEA 0183 was (and still is) a crappy standard. How crappy? Well, before I get into that, let’s note that there is one thing NMEA did right that later attempts to replace it got wrong. Each NMEA report is a text packet, or sentence, that begins with a dollar sign and ends with a carriage-return and line feed. The data elements in in NMEA sentences are just text fields separated by commas, like this:
$GPRMC,225446.33,A,4916.45,N,12311.12,W,000.5,054.7,191194,020.3,E,A*68
This means that log files of collected NMEA sentences are easy to read and edit. And that number on the right-hand end, after the “*” but before the CRLF? A data checksum, so you can tell whether you have a valid sentence or just line noise (and this is important: we’ll come back to it later). A GPS speaking NMEA emits sentences like this onto the wire, usually in once-per-second bursts.
The first layer of suck actually begins with what NMEA 0183 has you put in those packets. If you are a mathematician, you have a pretty good notion of what a TPV report is. It’s a 7-tuple
Internally, this is what a GPS sensor computes from the signal times to GPS satellites. Actually, to be pedantic, it doesn’t compute the error bars in exactly this form; rather, you get scale factors for the errors derived from the geometry of the satellites when the fix was taken, and have to multiply that by an experimentally-derived bugger factor dependent on things like how turbulent the radio-reflecting layer in the ionosphere is.
Now let’s look at what NMEA 0183 tells GPS devices to actually report. Here is a breakdown of the data in our sample sentence, which is in fact the most commonly used GPS reporting format for TPV:
1 225446.33 Time of fix 22:54:46 UTC
2 A Status of Fix: A = Autonomous, valid;
D = Differential, valid; V = invalid
3,4 4916.45,N Latitude 49 deg. 16.45 min North
5,6 12311.12,W Longitude 123 deg. 11.12 min West
7 000.5 Speed over ground, Knots
8 054.7 Course Made Good, True north
9 181194 Date of fix 18 November 1994
10,11 020.3,E Magnetic variation 20.3 deg East
12 A FAA mode indicator (NMEA 2.3 and later)
A=autonomous, D=differential, E=Estimated,
N=not valid, S=Simulator, M=Manual input mode
13 *68 Mandatory NMEA checksum
Alert readers will notice what’s missing here. Altitude, for starters — we’ve got no Z! People in boats, remember? They think they don’t need no steenking altitude. And no error estimates at all. And the T report is incomplete, giving only a two-digit year. Yup, that one got annoying real fast when the millennium turned. And it’s not like the designers couldn’t see that coming in 1993.
Eventually, NMEA wised up about the altitude thing. The sane way to proceed would have been to define a new sentence containing all the GPRMC information, plus altitude, plus a real four-digit year, even if error bars had to remain suppressed for some inexplicable reason. Here’s what we got instead:
$GPGGA,123519,4807.038,N,01131.324,E,1,08,0.9,545.4,M,46.9,M, , *42
1 123519 Fix taken at 12:35:19 UTC
2,3 4807.038,N Latitude 48 deg 07.038' N
4,5 01131.324,E Longitude 11 deg 31.324' E
6 1 Fix quality: 0 = invalid, 1 = GPS, 2 = DGPS,
3=PPS (Precise Position Service),
4=RTK (Real Time Kinematic) with fixed integers,
5=Float RTK, 6=Estimated, 7=Manual, 8=Simulator
7 08 Number of satellites being tracked
8 0.9 HDOP = Horizontal dilution of position
9,10 545.4,M Altitude, Metres above mean sea level
11,12 46.9,M Height of geoid (mean sea level) above WGS84
ellipsoid, in Meters
Now we’ve got X, Y, and Z…but T is even more damaged! You get a time of day, no month, no year, no century. No velocity report at all. We’ve got one number, HDOP, that tangles EDX and EDY together to give a circular horizontal error. And despite the fact that this sentence reports an altitude (Z), there’s an EDX/EDY and no report of EDZ!
For some inexplicable reason, NMEA also describes a GPGLL sentence that has all the brain-damage of GPGGA, but without the altitude. And a GPVTG that gives only a velocity report – no position, and naturally no error bars. Do I need to add that both have missing or incomplete timestamps? And oh, yes, there are actually two different incompatible variants of GPVTG.
Remember I said GPS receivers emit bursts of NMEA packets once a second? Well, the bursts typically consist of a GPRMC, followed by a GGA, possibly followed by a GPGLL and/or GPVTG. Er, no, I’m lying, they could be in a different order. The sentences in the burst have overlapping, incomplete information. The NMEA standard doesn’t specify even which ones must be sent, let alone the order they’re sent in.
Some NMEA GPSes part-repair the timestamp damage by shipping a sentence called GPZDA that gives you a full UTC timestamp with century. But the standard doesn’t require it, and most don’t, so you can’t count on it.
The first layer of suck was about what NMEA 0183 specifies. We are now passing into the second layer of suck, which is what it doesn’t specify. Like, the minimum set of sentences that have to be sent per reporting cycle. Oh, and nothing in the standard stops a GPS from simply omitting fields it doesn’t feel like reporting. It’s fairly common, for example, for receivers to not report magnetic variation or geoid separation (the geoid is an imaginary surface representing the difference between mean sea level and local sea level, which varies because the earth’s mass is not uniformly distributed.). GPS designers can save some absurdly tiny fraction of a penny per unit by not having these data tables in ROM, and they’re generally more than willing to shaft their customers to do it.
A mob of crack-smoking rhesus monkeys could have designed a better standard than NMEA 0183. It means that if you want to assemble a proper TPV report from NMEA sentences, you actually need to wait until you’ve seen an entire reporting cycle. Only…you can’t tell without knowing the type and firmware version of the GPS which sentences start and end the cycle! And even if you did know, buffering the partial data introduces latency that may be unacceptable for some applications.
A very practical way this manifests is that if you have a GPS client faithfully reporting the NMEA sentences coming over the wire, your altitude will typically flicker from known to unknown and back twice a second as it gets hit by alternating GPRMC and GPGGA sentences. That is, unless you buffer, in which case the altitude you see could be up to one second stale and associated with a previous fix.
The incomplete timestamps mean various sorts of lossage can bite you if you have a GPS client active at midnight. Unless your software is actually watching for the moment when the GGA timestamp goes to 00:00:00 and can compensate, it’s going to look like you’ve dropped back in time 24 hours until the GPRMC next comes in. Human eyes can just reject this, but what if you’re logging telemetry and try to graph against time? Similar anomalies lurk at the edges of years and centuries.
Yes, and if you want to report true altitude over ground correctly and consistently across devices, you better have your own geoidal separation table in software somewhere.
And I have nowhere near plumbed the stygian depths of the NMEA standard’s top two layers of suck. To spare the reader’s sanity, we shall lightly draw a veil over the spiky, vague, ill-documented horror that is NMEA error and status reporting and pass directly to the third layer of NMEA suck, the complete absence of any standardization of GPS control codes.
Here are some of the more important things there is no NMEA-standard way to tell a GPS to do:
- Report its vendor, model, and firmware version.
- Change the set of sentences it ships per cycle.
- Change the baud rate at which it reports.
- Change the number of samples it reports per second.
Of these, (1) is the most harmless-looking, but actually the deadliest. Many GPSes have vendor-defined commands to do (2) and (3), but it is far from trivial to figure out which set of vendor-defined commands might apply. If you are a GPS-using application, and you are handed the name of a port with a GPS on it, you have to either settle for the minimum common subset of GPS behaviors, or throw all the vendor-specific ID probes you know of at the device hoping it will respond to one of them. Hint: too often, it won’t.
But wait. Things get worse!
There are, broadly speaking, three major different ways that GPS vendors could have responded to the admitted fact that NMEA 0183, as given, is a festering pile of rancid camel vomit.
- They could have pressured NMEA into cleaning up the damn standard.
- They could have de-facto standardized on a decent set of extensions using the NMEA sentence packet format – a sentence that reports all 14 location parameters, a probe-for-ID query, a standard baud-rate change, etc.
- Or…they could invent a dozen mutually incompatible and poorly documented proprietary binary protocols, all of which throw away the transparency advantages of the NMEA textual packet format and each one of which introduces unique and special brain-damage of its own!
Guess which alternative most of them chose. Just guess…
You are now in a twisty maze of GPS reporting protocols, all different. Many devices have two different operating modes, one in which they emit NMEA packets and one in which they emit a vendor binary protocol that looks like nothing so much as line noise. At least one major vendor has dropped NMEA support entirely. If your location-sensitive application is naively expecting NMEA, you lose.
To be fair, one things the vendor binary protocols generally get right that NMEA doesn’t is shipping something close to a full TPV in one sentence per cycle. This at least avoids the nasty problems associated with integrating partial NMEA reports and worries about where the start of cycle is. However, I had to say “something close”; not one protocol ships the full and correct TPV 14-tuple. Usually one or more velocity components and error estimates are missing and have to be computed.
Let’s back off at this point and consider how people who use GPS sensors would, ideally, like their GPS sensors to behave. You plug it in, your software figures out what protocol it’s using, autoconfigures to match it, and starts collecting TPV reports and using them.
If “your software” is a GPSD-enabled application on a system with gpsd installed, it actually works this way. Those of you who have been following our descent into this fourth major layer of suck can be excused for wondering how in the flipping hell GPSD ever managed this trick in the twisty maze of vendor protocols, all different.
Certainly the vendors aren’t being much help here. Many of them (I’m looking at you, Garmin!) are cheerfully willing to assume that you will never use anything but their one idiosyncratic piece of GPS hardware, and that it will only talk to a limited, vendor-controlled selection of closed-source binary blobs provided by them or their business partners. Hello, vendor lock-in; goodbye, customer choice.
There is an Ariadne’s thread through this maze. It’s this: All the vendor protocols, like NMEA 0183, use packets with checksums and fixed header/trailer bytes. The intention is that they’re an integrity check so you don’t get fooled by line-noise-induced glitches. The side effect is that, if you’re sufficiently clever, you can do GPS protocol autodetection on the fly. It takes a fairly complex state machine that tangles together structural knowledge about every packet protocol in your supported set, but it can be done. In GPSD-land we call this piece of code the packet sniffer.
There’s something else the packet-sniffer does: it autobauds. Again, this is only possible because packet checksumming gives you a way to know for sure when you’re looking at valid data. When a serial GPS device is presented to gpsd, the packet sniffer doesn’t have to be told the baud rate the device is shipping at – it cycles through all possible combinations of speed, parity and stop bits looking for a combination under which it sees valid packets of some type. Normally this takes less than a second.
The packet sniffer is the real reason for the existence of gpsd — and I’d add “other programs like it”, except that there aren’t any others that I know of. Long ago, all the gpsd daemon did was serve as a multiplexer that read and buffered TPV reports from a single serial device so that several GPSD-aware applications could get simultaneous access to them. That’s all GPSD’s closest competitors today, like Gypsy, can do; they’re NMEA multiplexers. They typically can’t cope with non-NMEA devices at all; no packet sniffer.
The gpsd daemon also copes with the data management problems surrounding NMEA partial TPV reports, doing everything from supplying missing geoidal separation for altitude to computing and reporting error estimates from the geometry of the satellite skyview if the GPS doesn’t supply them.
Most of the suck surrounding GPSes can be summed up by “all this cleverness is actually necessary if you want to get clean TPV data out of more than one different kind of device”, or even out of just one kind of device that fails to supply a complete TPV. And, as we’ve noted, all of them fail in a dizzying variety of ways.
It’s true that in theory, every single GPS-aware application could include its own packet sniffer, the matrix algebra needed to compute missing error estimates, its own geoidal separation table, and all the other random logic needed to cope even with GPSes that are working nominally correctly. But have we mentioned yet that some…don’t? We know of at least three circumstances under which popular GPS chipsets return un-obviously corrupted NMEA – detectable, but you actually have to know how. Then there’s one chipset we know of that returns incorrect packet checksums when it doesn’t have a fix.
And we’re not done yet, because there at least two other sets of issues about extracting sense from these devices. One set is an artifact of the way USB GPSes are put together. Now, USB is generally a good thing in this context; unlike old-school serial ports, USB devices raise notification events on connect and disconnect, which clever GPS software can listen for and use to automatically hook up and sync to GPS sensors when they’re available.
However…naked GPS chips report serial data at TTL levels. The standard way to build a USB GPS is to hook up your GPS chip with a serial-to-USB bridge; there’s one in particular called a Prolific Logic 2303 that tends to show up on about 70% of the USB GPSes out there. There are two problems with this kind of design, one obvious and one subtle.
The subtle one is that both the bridge and the UART on the GPS chip have their own data buffers. Under most circumstances this doesn’t matter because the introduced latency from both together is very small – but some control operations (notably the serial-speed changes you’re going to be doing while you try to sync up with the device) need an amount of delay sufficient to flush both, otherwise you get odd race conditions that can result in garbage data coming back up the wire or your control operation silently failing.
The right combinations of OS-level buffer flushes and delays will avoid this problem, but clumsy ways to do it cause fix latency and application slowdown. The comment explaining these issues in the GPSD code leads off with “Serious black magic begins here.” and continues for 48 lines — because it needs to.
(This isn’t the worst thing you have to be careful of while hunting, though. Some Bluetooth GPSes with defective firmware will actually go so badly catatonic if you try to change their baud rate that you actually have to crack the case and unsolder the battery to unbrick them!).
Here’s the more obvious USB problem: there is no USB device class for GPSes. A USB GPS will present the vendor/product ID of the serial-to-USB converter. This means that, even if you’re fortunate enough to have an operating system that can do something reasonable with hotplug events, you can’t just tell it to watch for GPS devices going live and connect them to your software; you have to know which bridge-chipset IDs are likely to have GPSes behind them, sniff the data, and let go of the device if it’s not shipping GPS packets! Otherwise you might eat events from non-GPS serial devices that some other application badly needs to see.
And so on, and so on. Dealing with all this crap is further complicated by vendor documentation that is scanty if you can get it at all, and often written in rather broken English when you can. Part of of the problem is the structure of the GPS sensor market, which largely consists of dozens of tiny Pacific-Rim companies – each popping up out of nowhere, shipping the cheapest possible spin on one of about a half-dozen reference designs, and disappearing six to eighteen months later.
A friend who works in embedded systems tells me these little outfits aren’t even intended to last long; they’re actually run by giant electronics combines through several layers of shell companies as a way of providing deniability in case of lawsuits by patent trolls. They spin up, they ship, they funnel money back to daddy…and before there’s time for a process-server to show up, they disappear. All the engineers get rehired by a different sock puppet a week later. Lather, rinse, repeat. And…er…product support? What’s that?
It’s messy. Really messy. Those who love the law, sausage, or GPS devices really shouldn’t watch any of them being made.
Expecting GPS-aware applications to keep track of all this stuff would be just nuts. The best way to cope is to have a dedicated service layer that specializes in knowing about GPS idiosyncracies, hides all that ugliness, and presents a simple TPV-reporting interface to the application layer above. Ideally, the service layer should have a sharp crew of developers who are specialist GPS experts so that nobody else has to be.
And that’s exactly what the GPSD project is. It looks like a simple job…but it’s not.
Okkay, so GPS receivers suck in interesting ways, and the suckage explains why external applications have a hard time doing things like showing a consistant velocity and heading when traveling at a constant speed in a straight line.
Any idea why my Garmin GPS 496 occasionally shows a complete course reversal, or a major change in velocity, or a sudden climb or descent, when I’m flying straight and level? I would assume that Garmin uses its own devices to their best capabilities, after all. The 496 is WAAS-enabled, and should know where it is to a high degree of accuracy all the time. It’s rather disconcerting to have the thing occasionally warn, “Sink rate. Pull up!” when I’m in cruise at 7500 feet.
FWIW, the Garmin GNS 430W in the instrument panel never does that that I can tell.
Holy Mother of God!
GPSD Developers – Masochists only please!
>Any idea why my Garmin GPS 496 occasionally shows a complete course reversal, or a major change in velocity, or a sudden climb or descent, when I’m flying straight and level? […] FWIW, the Garmin GNS 430W in the instrument panel never does that that I can tell.
Fucked-up device firmware, most likely…though I suppose it could be an antenna problem, too. The fact that you’ve got another GPS reporting good smooth data from mere inches away disposes of most other potential causes.
And no, it isn’t jitter induced by NMEA partial reports, either; the error pattern from that looks different than what you’re describing, and Garmins report using a binary protocol with a decent single-sentence fix packet.
>GPSD Developers – Masochists only please!
Actually, I have to say it’s a fun little project. There’s a certain satisfaction in triumphing over teh suck, and at least it’s relatively easy to build test jigs. All the gpsd daemon ever sees is streams of serial data, and I long ago built a tool called ‘fakegps’ that starts it up connected to software-controlled pty devices, so if you can get a packet log along with a bug report it’s usually a piece of cake to replicate the circumstances.
The only real annoyances are that extracting decent documentation from vendors is difficult and getting them to ship us test hardware and eval kits without charging us an arm and a leg seems basically impossible. These clowns don’t seem to get that we’re adding market value to their hardware.
The original intention of NMEA 0183 was to link different navigation electronics, like linking a Decca receiver to a autopilot. All that on slow moving ships where electricity is expensive, with electronics that were often not that smart. NMEA predates the GPS, and has to provide backwards compatibility to an installed base of devices that are sometimes decades old.
That is one of the reason of the odd choices of “sentences”. The different sentences aim at different types of peripherals. It makes it possible for, eg. an autopilot to just filter out the $GPXTE sentences, and ignore the rest. That NMEA would be used more and more to feed GPS position data to computers is something nobody foresaw in the early 80ies.
So a lot of the suckage comes from trying to use a protocol for a purpose it was never really designed for. A bit thus like using Windows to do productive work…
Sorry for my extreme ignorance, but for me a GPS is something with a touchscreen I stick up onto the windscreen and it tells me aloud “on the next corner, turn left, turn left, turn le… I said LEFT, moron, now just turn back wherever you can” [1] i.e. a satnav i.e. a GPS receiver + a computer running map, route planning, speech synth and other software built into a small touchscreen device.
AFAIK a GPS is all about mobility (when you are at home you KNOW where you are :) ) and is there any point in carrying a laptop around with a standalone GPS receiver plugged in with USB instead of a neat little touchscreen device? If not, what’s the purpose of the project? If yes, what is that purpose?
It’s not a criticism, it’s curiosity. Probably the reason you spend your time on it is because you found some uses for it where a normal, simple satnav won’t be useful and the clumsiness if carrying a laptop around either doesn’t matter or well worths it – what are your uses? Or what are the most fun or most useful uses in general?
[1] OK current satnavs aren’t so outspoken YET, but… :-)
>AFAIK a GPS is all about mobility (when you are at home you KNOW where you are :) ) and is there any point in carrying a laptop around with a standalone GPS receiver plugged in with USB instead of a neat little touchscreen device? If not, what’s the purpose of the project? If yes, what is that purpose?
I got interested originally because I wanted to make signal-strength maps with wardriving software. That required connecting a GPS sensor to the PC running the wardriving software. Touchscreen GPSes are fine if you have no need to integrate location with other data sources; when you do, they’re too limited.
A lot of our technical contributors are people doing applications involving either remote telemetry from GPS-carrying sensor packages or embedded-systems work. We’ve had guys doing navigation on oil tankers and precision irrigation work in Australia. GPSD helped navigate a DARPA Grand Challenge entrant in 2004.
Wow, great writeup. Makes me miss embedded systems work.
But, the obvious question…
Given the churn with those shell startups and their products coming and going, how will your (presumably limited) base of developers ever keep up with it and keep gpsd from becoming stale? Seems a bit like the problem of keeping tax software up to date.
Is the situation getting any better? Seems like the industry would drift towards standardization.
Eric, my take from this is that, long-term, you should come up with a sane ISO standard for this communication. You’d be the perfect guy to do that, and to evangelize it.
Is there perhaps a list of (fairly inexpensive) GPS receivers/brands that you have encountered that you would actually recommend (at least above the rest) for usage with computers?
>Given the churn with those shell startups and their products coming and going, how will your (presumably limited) base of developers ever keep up with it and keep gpsd from becoming stale? Seems a bit like the problem of keeping tax software up to date.
It’s not quite as bad as that. They’re all using the same handful of reference designs – they have to, otherwise product development would eat up too much NRE. This means what GPSD has to keep up with is not so much individual products as it is GPS chipsets and chipset firmware revs. There are a lot fewer of those.
>Is the situation getting any better? Seems like the industry would drift towards standardization.
Well, yeah. SiRF has 70-80% of the GPS chipset market now and completely owns consumer-grade GPS mice. Not handhelds with screens – Garmin is the big cheese in that market, and also in aviation GPSes. There’s a tendency for each of the GPS market segments to develop a whale-and-minnows pattern where one chipset vendor dominates the mass market while the minnows serve specialty niches, of which the most obvious is higher-cost/higher-precision sensors.
>Is there perhaps a list of (fairly inexpensive) GPS receivers/brands that you have encountered that you would actually recommend (at least above the rest) for usage with computers?
The fact that these products use the same handful of reference designs also means large classes of them are indistinguishable by anything but the shape and color of the plastic case. In today’s market, I’d say look for any random SiRF-chipset receiver. There are scads of these. I like Haicom, because they make a good solid product and actually sent me test hardware; my bread-and-butter SiRF test receiver is a Haicom 204-S.
>Eric, my take from this is that, long-term, you should come up with a sane ISO standard for this communication. You’d be the perfect guy to do that, and to evangelize it.
I confess that similar thoughts have on occasion crossed my mind. Maybe after another round of redesign of the GPSD reporting protocol.
O rly? :)
>The original intention was to link different navigation electronics,…
>So a lot of the suckage comes from trying to use a protocol for a purpose it was never really designed for
Like our customer ,that insisted the raw data from a high res sonar array was available as NMEA because that’s the standard!
The sonar puts out 500,000 ranges/second, we have to use UDP so the ethernet can keep up.
But the customer decided on NMEA format so it would seemlessly work with all their other systems.
> “on the next corner, turn left, turn left, turn le… I said LEFT, moron, now just turn back wherever you can”
Heh. Someone really should make a satnav that abuses you when you make a wrong turn. It would be a hilarious and probably successful marketing gimmick for people looking to buy one as a gift.
gpsd has a hard time because you’ve aimed it at broken hardware. This is like trying to run linux on broken hardware, entirely possible, but in the end, a useless exercise.
Just use non-broken hardware: http://gps.psas.pdx.edu/OpenGnssProjects/
In the case that my comment is not-very obvious:
http://www.gpscreations.com/NewFiles/GPS600%20Brochure.pdf
http://www.gpscreations.com/Products_GPS1A.html
http://osgps.wiki.sourceforge.net/
>gpsd has a hard time because you’ve aimed it at broken hardware. This is like trying to run linux on broken hardware, entirely possible, but in the end, a useless exercise.
It’s the hardware actual users have actually available, Software-controlled radios are cool and in principle the idea of an open-source GPS reporting with a protocol that doesn’t suck is great…but until we can match the production economies of scale of (say) a Haicom-204S, it’s no more than a lab demonstration.
How are things looking with the Galileo project?
Okay so this explains a lot about why my OpenMoko GTA01 *still* can’t function as a GPS device with some arcane hokeypokey being required at the command line after a fresh install .. *GAH* … FRUSTRATING!
Galileo should be in service (all satellites up&running&communicating) by 2013. It will be compatible to NAVSTAR-GPS (GPS III), so a firmware update will be sufficient to let receivers work with it.
Expect the same quality in consumer hardware that you get now for GPS.
Right now the second test satellite is up, and a test service (4 satellites) is planned for 2010.
> AFAIK a GPS is all about mobility (when you are at home you KNOW where you are :) ) and is there any point in carrying a laptop around with a standalone GPS receiver plugged in with USB
But then you have these poweful and mobile devices which can run (and actually do run) gpsd, like the nokia N800/N810 internet tablets
> This is like trying to run linux on broken hardware,
Actually, Linux is full of workarounds for broken hardware. It’s sad, but software supporting broken hardware is the only way to go on the real world.
How does all of this tie in with the Royal Phillips Spot and SiGe software-GPS chips? AFAIK, only the front-end is in hardware, and then a chunk of software does all the baseband tasks. Presumably this software could be tweaked to provide whatever output would please you? Or is it all bound up in licensing?
Eric,
I posted your article on my local LUG mailing list and got a response from another expert in the field.
http://lists.lilug.org/pipermail/lilug-lilug.org/2009-February/000359.html
Both your article and his response been very enlightening reads.
> until we can match the production economies of scale of (say) a Haicom-204S, it’s no more than a lab
> demonstration.
Do you understand that the BOM on those devices is far, far less than the BOM on your precious Haicom device? If the software for them was only a bit better, it would be possible for a “GPS mouse” to be less than 25 EUR.
You are to complain about how broken GPS devices are, but you are yet uninterested in working with a device that is not broken, because it costs more today? When the software for devices like the GPS1A is complete, Hacom and other vendors will be forced to manufacture them.
Perhaps the problem is really that devices like the GPS1A are a threat to gpsd?
>Do you understand that the BOM on those devices is far, far less than the BOM on your precious Haicom device? If the software for them was only a bit better, it would be possible for a “GPS mouse†to be less than 25 EUR.
When I can buy one, I’ll be interested.
To give Garmin™ some credit, as far as they know they do publish much of their Special Proprietary Garmin Protocol documentation at no charge (I’m not entirely sure how “Free” they really are, but they are or at least were readily available).
This is in contrast to Magellan who used to do the same, but are now actively pushing towards a “consume-only” model for their units. As far as I can tell, you can’t even save your own data to the internal SD card to reverse-engineer the file format for general use – the SD card is only for “consuming” maps that you “buy”(license) from them. Ugh.
Of course, neither of them document their map format as far as I know…
Does anybody make a really good handheld GPS for active participants rather than mere consumers?
>Or is it all bound up in licensing?
Not as far as I know. I’ve written up what I think a better GPS protocol would look like; it would be cool if it were actually implemented somewhere.
Why don’t YOU create a new standard that GPS receivers can implement that works right? What is stopping you from starting an open consortium?
ooh, nvm, I just saw the last post. Perhaps no one cares :-(
I think the probable reason why the software part of GPS sucks is that software specified/developed by hardware companies tends to suck – I have my similar, altough not _so_ bad stories about bar code printers, bar code readers, credit card readers, customer terminals for POS… plain simply, due to their company culture, they tend to have an electrical engineering mindset, not a software engineering mindset.
A clear example of this here is that the record has no unique identifier, and that timestamp field that’s the closest thing to one doesn’t have millisecs (which would be, although not unique in the strict sense, “unique enough”).
An electrical engineer usually thinks like this: “Here, this is the signal, the signal correctly reflects the current state of the system, therefore, job done.”
A software engineer thinks like this “I have to make it clear whether this record is a new one or the correction of the previous one, because the software realized the previous one was in error. I’ll make it clear by using a new unique id or at least a new timestamp to the highest possible precision, or reusing the same old one.”
No offense meant to electrical engineers – the whole GPS stuff sounds like a miracle to me, it’s on the general HW/system level a job very well done. It’s just that, the same way as for bar code printer etc. vendors, their managers probably should hire a software engineer and not dump the job on one of the electrical engineers.
Darn you, Eric, now I’ve gone and discovered OpenStreetMap. I’ve been tracing Yahoo! maps so far, but since they might be offset, I’m a bit nervous about how good or bad the data might be. I’m actually considering acquiring an old GPS (the Garmin eTrex appears to go for about $65 on eBay) and compulsively walking every single road, footpath, railway and riverbank in the county, since the project’s data in this region is mostly automated.
Hmm. But then again, I’m short on cash, and I might be able to mod my TomTom (not sure which model it is; it’s not here at the moment) to record GPX tracks instead, which would certainly be cheaper.
Add Dead Reckoning to the mix and you’ve got tons more headaches. When the gps module looses gps fix for too long, it locks out. Then we have to run a hack telling it to recalibrate. The NMEA isn’t enough, we are handling both protocols on the same stream! Just imagine parsing NMEA text packets, which sometimes have a ; end of the message and sometimes don’t. Yes they half-assed the NMEA protocol! I won’t mention who the chip is, but then among that poorly implemented NMEA we are parsing for binary UBX protocol. Their customer service seems unable to provide specifics of how certain parts of there chips work under conditions, such as the DR lockout. The main problem with issues like that is that the chip is geared towards a specific usage, cars. Such obscure, undocumented, features may make the chip work a little more as expected in a car application, but causes hell for, say a train. Long story short, GPS is all hacks and headaches.
> GPS is all hacks and headaches.
you forgot the ‘D’
Having a copy of the (quite fairly) maligned NMEA 0183 in front of me, I can see it also defines:
* a “GSA”/”GRS” sentence pair that describes the satellites used in the solution, along with three errors (PDOP, HDOP and VDOP) and range residuals to each satellite. In this one the timestamp field is specified to match the one in the GGA / GNS sentence to which it applies.
* A “GST” sentence that gives more complete error information – standard deviations in X, Y and Z as well as a horizontal error ellipse, and the root mean square of the standard deviations in the satellite ranges. Again the timestamp here matches the corresponding GGA / GNS sentence.
Don’t know if many real-world systems send these, though!
I’d also echo what Krist says about the focus of NMEA 0183, too. GPS (and GLONASS) stuff takes up about 10 pages out of more than 100 – the rest is about things like describing a sentence for changing the frequency of your VHF radiophone.
@John Chapman
> Do you understand that the BOM on those devices is far, far less than the BOM on your precious Haicom device? You are to complain about how broken GPS devices are, but you are yet uninterested in working with a device that is not broken, because it costs more today?
I’ve spent thousands of dollars of my own money to buy new receivers so I can add/validate support for them. My hardware budget is tapped out for the forseeable future, otherwise I’d be buying more.
Also, cost to gpsd developers is less relevant that the cost to your mom … or “Aunt Tillie” as Eric likes to name the hypothetical non-techie user. Gmail only keeps my deleted messages for 30 days, so I can’t give you an accurate count of how many users have asked me “will this super-cheap-ass piece of crap that I just bought work?” It’s a big number though. We’re less concerned with supporting Brand X, Y and Z than we are with supporting relevant hardware.
> If the software for them was only a bit better, it would be possible for a “GPS mouse†to be less than 25 EUR. When the software for devices like the GPS1A is complete, Hacom and other vendors will be forced to manufacture them.
Looking forward to seeing mass production. Until then your product is better than vaporware, but still not relevant or useful to Aunt Tillie.
> Perhaps the problem is really that devices like the GPS1A are a threat to gpsd?
Hardly. I like writing drivers… send me one and I’ll happily work on adding support for it in gpsd.
>> Perhaps the problem is really that devices like the GPS1A are a threat to gpsd?
>
>Hardly. I like writing drivers… send me one and I’ll happily work on adding support for it in gpsd.
Chris, you just innocently confirmed Chapman’s model. His maliciousness greatly exceeds his degree of clue; he appears to believe you and I are so territorial about “our” project that we don’t want sane GPS hardware to succeed. It’ s called “projection”.
>Don’t know if many real-world systems send these, though!
GSA, yes – it’s one of the de-facto standard set (RMC, GGA, GSV, GSA) that all but the most brain-dead NMEA GPSes ship. If there’s a sixth, it’s likely to be VTG or, in rare cases, GLL. Sadly, ZDA is not in this set.
GRS/GST, no. I’ve never seen a GPS ship these.
>I’d also echo what Krist says about the focus of NMEA 0183, too. GPS (and GLONASS) stuff takes up about 10 pages out of more than 100
True, but no excuse for the fact that those 10 pages are so brain-dead. Designing a better protocol took me, oh, about 20 minutes.
> Chris, you just innocently confirmed Chapman’s model.
yes, but not in the way that eric suggests. Rather, the gpsd developers appear to spend a lot of time coddling broken hardware, and a broken spec. The last time I looked, gpsd has its own Kalman filter.
Rather than fighting the hardware broken, wouldn’t it be better to put some effort into developing great hardware, hardware with a manufatured BOM that is demonstrably lower than the current “GPS mice”? Hardware that actually has a chance of better performance, *and* a decent interface to the world? (It could have a NMEA-0183 interface as needed, too.)
I don’t believe that you’re territorial about gpsd beyond a certain “can’t see the forest for the trees” mentality, and even if that, I understand how you got there.
@John Chapman
> Perhaps the problem is really that devices like the GPS1A are a threat to gpsd?
That’s an odd attitude to take… you appear to be selling the hardware for GPS1A for about USD$500.
The crew behind gpsd provide free, open source code to use with 3rd party hardware, bluetooth versions of which retail around the USD$30 mark.
I think the balance of altruism is pretty clear here.
By all means, I applaud the sentiment of open source hardware, and am facinated with FPGA powered SDR’s, but I have USED gpsd in real projects and by handling all the gps/nmea crap, they have improved my life – thanks guys!
As a generalised protocol for communicating positioning data it’s completely brain-dead, but I would argue that within its intended application domain it was quite adequate (eg. the ship’s autopilot doesn’t care what the date is). The RSA sentence for Rudder Sensor Angle doesn’t give an error estimate for the rudder angle, either.
The problem as I see it was co-opting this standard for a purpose for which its completely inadequate.
Have you documented your better protocol? I’d be quite interested to have a look.
If anybody wants to try using a GPS receiver and doing some mapping on OpenStreetMap, consult the Events Calendar at http://wiki.openstreetmap.org/ — for example I’ll be in Baltimore this weekend running mapping parties on both days, Pittsburgh next, and Philadelphia the weekend after.
>Have you documented your better protocol? I’d be quite interested to have a look.
Here: http://gpsd.berlios.de/replacing-nmea.html
I describe three NMEA-format sentences that would, dammit, *solve the problem*!
Eric: in your scheme, is there any difference between the GPSGP sentences emitted by a controller to instruct a receiver to switch to SGPS or NMEA, and the sentences emitted by a receiver to indicate its new status? If not, it seems to me there could be a problem if you have more than one receiver attached – or can you guarantee that this situation will never arise?
>can you guarantee that this situation will never arise?
Hm. Maybe not. Are you assuming that the receivers see each others’ sentences, e.g. that the controller and receivers are sitting on some kind of multidrop serial bus rather than having pairwise point-to-point links? If so, then, yes, the protocol needs to be patched so the requests and responses can’t be identical.
I’ve never seen a setup like that, so the possibility hadn’t occurred to me before now.
> Are you assuming that the receivers see each others’ sentences, e.g. that the controller and receivers are sitting on some kind of multidrop serial bus rather than having pairwise point-to-point links?
Yes, exactly.
> I’ve never seen a setup like that, so the possibility hadn’t occurred to me before now.
Wasn’t that the original NMEA use case – a serial bus full of lots of different bits of nav gear, each identifying itself by its sentence header? Justin Dearing’s link above suggests as much.
BTW, there’s at least one obvious use case for having two SGPS talkers attached to the same computer: having one GPS receiver and one Galileo receiver, and taking a weighted average of their readings for increased accuracy. I guess you’d probably use pairwise point-to-point links here, but it would be nice not to have to :-)
John, your project is a Good Thing. It’d be an Even Better Thing if it was cheap and commonly available. Until then, there will be a need for software to deal with the broken hardware that people can actually afford and obtain. Having already spent $2600 on my Garmin GPS 496, I’m not about to spend $500 on your board to do those things that the 496 will let me do with my computer, just so I wouldn’t have to use gpsd. I’d say that gpsd is far more a threat to your product than the other way around.
>Wasn’t that the original NMEA use case – a serial bus full of lots of different bits of nav gear, each identifying itself by its sentence header?
Yes, I believe you’re right about this. Now that I’m really thinking about it, I vaguely remember reading something about NMEA originally being designed around a multidrop variant of RS422 in order to support configurations like this. I’ve updated the proposal accordingly.
A “consume-only” model. I like that turn of phrase. Consume! Obey!
I’m reminded of the :CueCat here–certainly the starkest example I can think of in terms of hardware that’s utterly useless in “consume-only” mode, but plenty useful even today when made general-purpose.
That’s an excellent read; it’s exactly the sort of thing that I was going to ask if you’d written. Just a note, though–there’s something wrong with the revision history at the top; the revisions are listed as 1.1, 1.0, 1.1, 1.2, and so on.
You know, if someone (John Chapman?) links me to somewhere I can get a $25 GPS1A device for evaluation, I’ll buy one on behalf of the gpsd developers. If the only thing you can offer is a $500 prototype, then you’re just selling vaporware at the moment.
John, I honestly wish you all the success in the world, but you’re flaming people who could likely help you, and certainly have no desire to harm your project. The problem you’re seeking to solve won’t be fixed by the gpsd developers; it’ll be solved by actually getting the hardware out there. If you send prototypes to the gpsd developers, they’ve already offered to support it. You’re barking up the wrong tree.
John Chapman wrote:
> Rather than fighting the hardware broken, wouldn’t it be better to put some effort into developing great hardware
Only if one had hardware skills. You’re talking to software hackers. They’re doing what they can to improve the situation, just like you. Your better bet is to get them interested in supporting you. Give them prototypes, so GPS1A will work with existing software at launch.
> I’ve updated the proposal accordingly.
Excellent. Glad to have been of assistance.
>If you send prototypes to the gpsd developers, they’ve already offered to support it
Actually, I don’t think we have offered in so many words, but…hell yeah! If they gave me one of those puppies, first thing I’d do would be to implement SGPS both on it and in gpsd.
>BTW, there’s at least one obvious use case for having two SGPS talkers attached to the same computer:
GPSD already supports gathering data from multiple simultaneous GPSes. Clients can query what devices are attached and optionally listen to named channels. If there’s only one GPS device attached, or if your client doesn’t care which it’s listening to, you can ignore this feature.
It’s actually even cooler than that, because on a system with hotplug capability channels can pop into and out of existence as devices are disconnected and reconnected. Clients get notified when this happens, but unless they’ve chosen to bind to a specific device they always get reattached to the most recently active one if the one they’ve been listening to closes.
I don’t know whether anyone actually uses these multidevice features, mind you. I just knew It Had To Be Done and wrote it that way.
> Only if one had hardware skills. You’re talking to software hackers. They’re doing what they can to improve the situation, just like you.
> Your better bet is to get them interested in supporting you. Give them prototypes, so GPS1A will work with existing software at launch.
There are several points of confusion..
1) I don’t sell any GPS hardware. Making the point that I do is laughable. If you think so, to quote esr, “you are an idiot”.
2) the hardware exists (I gave a link). yes, its more than “Aunt TIllie” wants to spend, today. But if you look at the BOM, there isn’t anything expensive on it.
2b) its not the only hardware like that, but the other hardware is just as expensive, and far less portable.
3) the hardware is waiting for solid software, the existing software is GPLed. Its “open source”.
If you want an open source GPS, or a GPS that actually works, or one that implements Eric’s version of NMEA-0183, or… then the solution is at hand.
I don’t think the GPSD developers are interested in hardware that is so different. Perhaps they like struggling with difficult/broken hardware.
BTW, the GPSD device support code should pick the BEST (lowest error) signal, not the last one that attached.
>I don’t think the GPSD developers are interested in hardware that is so different. Perhaps they like struggling with difficult/broken hardware.
No. I can state unhesitatingly that I’d be much happier if the reporting protocols didn’t suck. The hardware, in general, isn’t too bad — it’s the software design choices in the firmware that suck.
We’re very interested in any hardware that people choose to actually, like, send us. Beyond that, lots of factors enter. If it’s got lots of market share it’s interesting. If it’s got a cool feature we haven’t encountered before, it’s interesting. If it ships a protocol we haven’t encountered before, that’s interesting too; Chris and I both enjoy writing the drivers for these things.
>BTW, the GPSD device support code should pick the BEST (lowest error) signal, not the last one that attached.
I’ve actually been rethinking that. It occurred to me earlier today that I should probably by default allocate the device that’s been reporting fixes the longest, rather than the most recently allocated device, because fix quality tends to improve after the device has been through at least one 50bps-subframe cycle since cold start.
But pre-sorting by fix quality makes sense too – in particular, we probably want to prefer devices with a DGPS fix to devices without.
Hm…I think I’ve got the allocation policy isolated in a single function. Time to go look…
the hardware exists (I gave a link). yes, its more than “Aunt TIllie†wants to spend, today. But if you look at the BOM, there isn’t anything expensive on it.
It’s not just more than Aunt Tillie wants to spend…it’s more than anyone but a Stallmanite “free software” zealot would spend. The rest of us just want to get the job done, and that’s the market that the GPS1A is competing in. We’re not interested in spending 17 times the money to get something with open source firmware.
John, if you don’t sell it, then why, exactly, do you care enough to object as strenuously as you do? Methinks thou doth protest too much. It’s certainly not a conspiracy of gpsd developers, who all would happily go on and do other things if they didn’t have to deal with so much real-world broken hardware.
>We’re not interested in spending 17 times the money to get something with open source firmware.
Agreed. Jumping in early to get a Googlephone is about my limit, there. I might be willing to pay a 15% or 20% price premium for open firmware, but not a 1700% one.
My sales resistance to “Ooh! It’s open source! Pay extra money!” varies inversely with the degree to which I think I need the device to be flexible and programmable. Thus: I won’t run a closed-source OS on my PC under any circumstances; I’m interested in open source for my cellphone; but not so interested for my router downstairs (yes, I know it’s possible).
Hm, I wonder if that my-license-will-rule-the-world character who thinks I’m a “zealot” is still lurking? Maybe his brain will explode when he reads the previous paragraph…
Looks like this was a serendipitously timed post: Microsoft is patent-trolling Linux, targeting a GPS maker as its first defendant.
> I’m interested in open source for my cellphone; but not so interested for my router downstairs (yes, I know it’s possible).
I’m surprised you’ve never had the inclination to hack your router. I finally gave up even on DD-WRT because it wasn’t flexible enough. I use my desktop system as my router now. OTOH, I’ve never had much interest in using my phone for anything besides phone calls.
>I finally gave up even on DD-WRT because it wasn’t flexible enough.
Flexible enough to do what? Maybe if I understood more about the router version of stupid pet tricks the appeal of things like DD-WRT would be clearer to me.
Well, the main issue is that the firmware on most commercial routers is buggy as hell. I’ve never used one which I didn’t encounter one or another incomprehenisble failure mode where connections to random groups of ports get reset a second after being opened, or its WPA implementation is somehow brain-damaged and my wifi card can’t associate, or it has years-old buffer overflow vulnerabilities, or it just goes totally catatonic twice daily…
I even read one report that Belkin was selling a router that replaces every thousandth HTTP request with a Belkin ad.
But even when they work, there are things I want to do with my router that off-the-shelf firmware just isn’t designed for, like
Add a hook when my IP address changes so that I can update it with my IPv6 tunnel broker (already doing this)
Run an IDS so that I can blackhole the dozen nincompoops that run dictionary attacks on my SSH server every day (doing this with my office network)
Have my DHCP server send updates to my DNS server, as a less-annoying alternative to zeroconf (intended project for this weekend)
Offer an open access point on a separate SSID from my secure one, but do traffic shaping to keep people from abusing it (this one is hypothetical)
> Maybe if I understood more about the router version of stupid pet tricks the appeal of things like DD-WRT would be clearer to me.
ddwrt is to routers what gpsd is to consumer gpses. it’s something written because someone thought “why does the vendor personally have a vendetta against me? why must they make my life so difficult?” or because it’s just fun to wring more out of the hardware…
it solves problems for some people and is fun for other people. just like gpsd.
> No. I can state unhesitatingly that I’d be much happier if the reporting protocols didn’t suck. The hardware, in general, isn’t too bad — it’s the software design choices in the firmware that suck.
and the same {hardware, software} can run {software,hardware} with varying levels of suck…
iLotus sells a motorola M12 clone based on a SiRF part.
ublox used to ship a receiver based on a SiRF part.
Some Jupiter chipsets were SiRF parts.
SiRF has apparently crippled their own software – removing raw measurement capability from GSW3. That’s now a value-add feature.
FastraX iSuite runs on uNav, SiRF and Sony chips
uNav chips will run iSuite as well as the more minimalist Orion.
Garmin has successfully crippled SiRF and GlobalLocate chip with their firmwares
And didn’t I read something about someone cramming an open firmware into a Novatel SuperstarII?
> Rather, the gpsd developers appear to spend a lot of time coddling broken hardware, and a broken spec.
Yes.
We got started doing that because the broken hardware was mostly affordable, available and useful. So we dipped some pigs in concealer, glued on false eyelashes and a wig, added some lipstick and thought… “well, it’s better than no prom date.” Yes, we know we’re paddling upstream. We grumble about it a fair bit. And the 5 or 6 of us who commit to gpsd aren’t the only ones getting burned by crappy hardware; we’ll sacrifice some evenings and weekends to make gps less painful for everyone else.
> The last time I looked, gpsd has its own Kalman filter.
Nope. DOP calculator, but not kalman filter.
> Rather than fighting the hardware broken, wouldn’t it be better to put some effort into developing great hardware, hardware with a manufatured BOM that is demonstrably lower than the current “GPS mice� Hardware that actually has a chance of better performance, *and* a decent interface to the world? (It could have a NMEA-0183 interface as needed, too.)
Manufactured BOM isn’t terribly relevant. i’ve seen different vendors selling what appears to be the same SiRF reference design for a $50 spread. It cost them the same to make the board, but BobGPScorp wants $50 more for it than AliceGPSinc. Or look at Novatel’s pricing for features. You flash a new firmware or send an unlock code and your 1Hz NMEA receiver is now 10Hz, Carrier Phase, Binary and Precise Timing capable. I’ll be much more interested in this “great hardware” when I can buy it at the same price point as the current gps mice.
As for non-sucky software, I’m doing that too. We’re looking at adding raw data capability to gpsd so that you can plug whatever you like in, get raw measurements out and pass the measurements off to your favorite software solver… and I commit or send patches to a few other software solvers too.
>I’ll be much more interested in this “great hardware†when I can buy it at the same price point as the current gps mice.
Well…I might be willing to pay a bit of premium, but not a lot.
The trouble with open-hardware projects is that they fetishize the part that’s cool, sexy, and easy – the design stage. OK, great, so you’ve got a winning design with an attractively low BOM. Now show me how you’re going to raise enough capital to tool up for mass production to capture economies of scale.
Then, show me what you think your market and your value proposition to that market is. Finally, show me the people who can execute on this.
In other words, show me a business plan. Atoms aren’t bits; you can’t just handwave away the cost of deployment, because they don’t fit through the Internet.
Eric,
When PCB printers can be had at Radio Shack for a hundred bucks, they’ll be ready.
>When PCB printers can be had at Radio Shack for a hundred bucks, they’ll be ready.
Not gonna happen. There’s been a huge amount of capital put into making PCB fabrication inexpensive; diminishing returns set in long ago. It’s going to take some leap forward like inexpensive molecular assemblers to change the rules, at which point “open hardware” designs based on conventional PCBs will be about as relevant as open hardware designs carved in ivory or flint.
There is a well known solution to what you describe. It is called a “device driver”.
There is also another well know solution to this problem. It is called a “configuration file”. (For example, curses.)
Why wrap all this code up into one complex tangle when these other modular solutions would be a better approach? (For example, you describe a complex state machine to determine the type of GPS device. What happens when a new one comes out? Do you have to pick out the knitting to add it? Wouldn’t it be simpler to simply write a new device driver? Or add an entry to a config file?)
I suggest that what happened is that you saw a standard, and assumed it was good. Turned out that it was not a “standard” at all, and you have been caught in this trap ever since.
>There is a well known solution to what you describe. It is called a “device driverâ€.
Yes. GPSD has them; happily, they don’t have to live in kernel space because GPS communications are slow and there’s no hard real-time requirement. The daemon actually consists of three parts (1) a set of device drivers, (2) the packet sniffer, which dispatches to the device drivers, and (3) a thin layer of socket and D-Bus handling that acts as a subscription broker between user clients and the devices, shipping TPVs up to subscribed clients as they come in.
>It is called a “configuration fileâ€.
Which is unacceptable if the behavior you want is that Aunt Tillie plugs her GPS into her laptop and it just works. GPSD does that; if it required users to tweak configuration files rather than performing discovery on its environment and autoconfiguration on itself, it couldn’t.
>What happens when a new one comes out?
If it’s built from an existing chipset, likely it just works. If it’s a new one, we need to write another driver and weave another recognition path into the packet sniffer.
> Which is unacceptable if the behavior you want is that Aunt Tillie plugs her GPS into her laptop and it just works.
Does this reflect actual user demographics, or is it just a self-imposed requirement? How many Aunt Tillies use GPSD?
>Does this reflect actual user demographics, or is it just a self-imposed requirement?
Self-imposed. When I took over the project in 2004 I was thinking a lot about Macintosh applications, autoconfiguration, and simple UI. I took GPSD as a challenge to see how far I could push autoconfiguration. Pretty far, as it turns out; the daemon has no config file and I like it that way.
Part of my success was due to the serendipitous fact that GPS protocols all use variations on a stereotyped packet format; this made autobauding possible (it’s not when your incoming data is less structured). I exploited this opening ruthlessly.
>How many Aunt Tillies use GPSD?
We think we have two cohorts of users — one large one we seldom hear from and one small one we hear from a lot. The large one is people using Linux laptops with USB GPSes; for them it just works — which was the goal. Most of them probably don’t know GPSD exists, they’re using apps that talk to it to get TPVs (kismet, roadnav, gaia, gpsdrive, etc.).
The small one is techies doing embedded systems, navigation, and remote telemetry.
I wonder if the GPS in the Android phone is available via gpsd?
Probably, if you root your android phone. I haven’t yet written a gpsd compatibility layer for my android yet…
>How many Aunt Tillies use GPSD?
She might not use it directly, but she probably uses it in her TomTom.
By the way, Eric, what’s your take on Microsoft suing them?
>By the way, Eric, what’s your take on Microsoft suing them?
Right, I should blog about this
> I wonder if the GPS in the Android phone is available via gpsd?
Its not, and this (along with the implementation of spreading GPS info via a software layer) is the thin edge of the wedge. GPSD will go the way of stand-alone GPS devices.
A few questions about SGPS proposal:
* Latitude and longitude are signed, and in degrees. Do I understand correctly thta you use signed instead of N/S postfix (for latitued) or E/W postfix (for longitude) like in NMEA for easier parsing?
* In example you have “4916.45,” for latitude, which in NMEA $GPRMC would be 49 degrees and 16.45 minutes, while SGPS states that it has to be value in decimal degrees, not degree/minute/second or degree/decimal minute; 4916.45 does not make sense as value in degrees (up to 360 max), especially not latitude (-90..+90). I guess that it is copy and paste error from NMEA example.
* You have GPSVU for getting satellite status. Is there a way in GPS to get information about DGPS (differential GPS) stations used, or other augmentation information used?
> A few questions about SGPS proposal:
Does it work with GLONASS?
>I guess that it is copy and paste error from NMEA example.
Correct, thanks for pointing this out.
>You have GPSVU for getting satellite status. Is there a way in GPS to get information about DGPS (differential GPS) stations used, or other augmentation information used?
In the current version, no, I’ve considered adding a fix-quality field to the SGS report, but the information users really want about quality is in the error bars.
not(me): I think it’s more likely that Android will change than that gpsd won’t ever work on it. Android is an interesting hack. Enough people are disinterested in a java-only platform that its keep will not hold. It’s too open to be less than completely open. Most of what’s done on it will suck, just like most of what’s been done on the OpenMoko sucks, but one of them won’t (and I doubt very much that the non-sucky one will be the one Google has chosen.)
>Enough people are disinterested in a java-only platform that [Android’s] keep will not hold.
Google has already informally announced plans to support apps written in C and other languages.
I have just deleted all the comments in this thread (including a few of mine) that wandered off into flaming about open-source economics.
The parent post was about GPSes. Keep it on topic.
I will tolerate more drift in the comments on non-technical posts. But stuff that I think is off-topic flamage is subject to deletion at any time, and I’ll prune away entire forests of that even if it means removing my own responses because they no longer have context.
I repeat: The parent post was about GPSes. Keep it on topic.
The wonders of Committee developed standards!
A number of the GPS chipsets can be setup to only output certain sentences and at rates you define, this can help somewhat with receiving different sentences with similar data in them. You really have to prioritize what sentence you will use for each field and stick with it for the life of the app.
>A number of the GPS chipsets can be setup to only output certain sentences and at rates you define, this can help somewhat with receiving different sentences with similar data in them. You really have to prioritize what sentence you will use for each field and stick with it for the life of the app.
This doesn’t work unless you know the device type in advance and can thus be certain what sentence mix it will emit. Because we want to support hotplugging a USB GPS, we can’t make that kind of assumption.
“an USB GPS”?
Don’t tell me you pronounce it “uss-buh”.
Or do you pronounce it without the initial [j] (pardon my IPA), as in Oo-Es Ah-Mee?
I had a Latin teacher once who had a habit of occasionally using “an” before inappropriate nouns, just to see if anyone would call him on it. I think most either knew what he was doing, or assumed he had arcane Classical reasons.
All that commentary over a typo. Some people have too much time in their hands. Fixed…
Haven’t you heard? We’re all such starry-eyed esr fanboys that we can’t possibly conceive of the possibility of the great esr uttering anything so mundane as a typo :)
> Do you understand that the BOM on those devices is far, far less than the BOM on your precious Haicom device? If the software for them was only a bit better, it would be possible for a “GPS mouse†to be less than 25 EUR.
Erm, I just bought a SiRF Star III based GPS mouse for 12 EUR… It has no brand name on the box, only the type code “BU-353” and “Made in Taiwan”, but it seems to work fine.
Well, just wait until you look at correction sources!
There’s RTCA, RTCM-2, RTCM-3, CMR, LB2, just to name a few.
If you have some spare time, look at RTCM-2 and you’ll realise that NMEA is heaven in comparison.
At least they got it more or less right in RTCM-3. Well, more or less :)
>If you have some spare time, look at RTCM-2 and you’ll realise that NMEA is heaven in comparison.
Er, gpsd actually implements RTCM2 decoding, and partial RTCM3 decoding. I know all about this horror, believe me.
The main thing stopping me from finishing RTCM3 support is lack of a test load. Do you know anywhere
I can get a set of RTCM3 packets together with dumps of their content in some reasonable ASCII format?
Has anyone else feel like the global downturn has hurt the board industry?
The “Pacific Logic 2303” mentioned in the article should be a “Prolific 2303” (and they certainly are).
> http://gpsd.berlios.de/replacing-nmea.html
The current version (1.5) still has the incorrect latitude example of “4916.45”.
ok
dpm
OK. I propose a solution: Create a new clean sane standard. We obviously can not depend on NMEA or the GPS vendors to do this, so we’ll need to step up. We’ll need the following:
1. Host to Sensor Commands.
2. Sensor to Host command responses.
3. All sensors are clean USB or IP (UPnP). No serial layer.
4. All sensors can get firmware upgrades through this standard.
5. All sensors are un-brickable.
6. All sensors report Full TPV + Optional Gyro + Optional Accel.
7. All sensors support both the push and the pull models.
8. Protocol needs to accomodate new and alternative Postion detection systems.
Gallileo, Differential near-field, Etc.
We need a “Brand” so that end users can help drive demand for devices that impliment this standard.
We need to market this “Brand”. Think DLNA, Dolby, THX.
Just to give anyone who might not know a heads up, but NMEA has been busy with a new standard, NMEA2000 based on CAN. While it has a number of improvements, there are some caveats:
The hardware is still marine only which means heavy and expensive.
CAN interface hardware is not standard on many PC’s.
If interested, here’s a manual for a currently available receiver (see appendix A):
http://www.maretron.com/support/manuals/GPS200UM_1.1.pdf
NMEA2000 may never become a standard for consumer devices.
GPS:Global Position System
GPSD ？？？
My GPS Hemisphere R131 is interfaced with a Linux PC. The serial port of the PC receives many formats including GPRMC, GPGGA, GPZDA etc. However, an application i run on the PC does not capture GPRMC. When GPS of other make are interfaced with the PC, the application is able to capture GPRMC string. Can anyone explain this ?. I think the reason is that NMEA standards are not exactly followed by all manufacturers of GPS and hence the differences.
I was reading your post, and when I read
Report its vendor, model, and firmware version.
Change the set of sentences it ships per cycle.
Change the baud rate at which it reports.
Change the number of samples it reports per second.
I immediately thought “CANopen has all of these and many more, already codified”. I searched CiA (CAN in Automation) site for profiles and I noticed there is no profile for GNSS. CANopen goes even further by allowing to change “sentences” (PDOs) composition, if the vendor chose a stack supporting that. It will also support multireceiver configurations out of the box.
If I make up a profile draft based on your “Towards a better GPS protocol”, could you be interested into examining it and giving me a feedback?
>If I make up a profile draft based on your “Towards a better GPS protocol”, could you be interested into examining it and giving me a feedback?
Sure.