Plug and Pray in GPS-land

Welcome, ladies and gentlemen, to another darkly humorous tale of the seamy side of GPS interfacing. GPSD working with USB GPS mice has, when properly installed, lovely plug-and-play self-configuring behavior. That is, you plug a USB GPS into a USB port, the hotplug system notifies the gpsd daemon that the GPS is available, the daemon records this fact…and subsequently when you start up any GPSD client application It Just Works. Well, usually. There’s a dangerous weakness in the machinery, and yesterday it came around and bit us in the ass.

The root of the problem is a hole in the the USB standard. For most other kinds of USB devices (printers, mice, mass storage, smart cards, hubs, what have you) there’s a defined device class code which gets presented to your USB subsystem; so, for example, your print spooler can enumerate all the USB printers attached to your system and ignore everything that isn’t a printer. But the standard does not define a device class code for GPSes.

If there were a GPS device class, and GPS manufacturers followed the standard, then GPSD’s hotplug rules could simply say “hand everything with that type to GPSD” and all would be happiness. But there’s no such class, so we have to whitelist a bunch of vendor/product IDs and assume that if the device presents one on this list, it’s a GPS.

In truth, we’d probably have to do this even if there were a GPS device class. You see, all the GPS mice we’ve ever seen are are lash-ups consisting of a receiver module that’s wedded by a sort of bastardized TTL version of RS-232 with an RS-232-to-USB adapter chip. The USB device class is presented down the wire to your PC by the adapter chip, which would have to be custom-modified to show something other than its own device class and vendor/product ID. These devices are produced on such thin margins that the vendors might not spend the extra pennies for the customization.

In point of fact, USB to serial adapters don’t have their own USB class either (they return 0xFF for “Vendor Specific”). So we have to whitelist the vendor/product ID pairs of the USB adapter chips most commonly used in GPS mice; by far the most common of these, incidentally, is a doohickey called the Pacific Logic 2303.

So here we are with our udev rules file that’s watching for a plugin event from anything on the whitelist. The udev rule calls a Python script written by yours truly which launches a gpsd instance if one is not already running, then stuffs a notification up the daemon’s special control socket that said device might be a GPS. Later, when a GPSD client app wants location data, the daemon will actually open every device on its stash list and attempt to recognize GPS information flooding from them (not a trivial process; the GPS information could arrive in any of 13 mutually incompatible formats, which are untangled from each other by a big honkin’ state machine also written by yours truly).

Astute readers will already see the problem. We don’t actually have any guarantee that the thing behind any given whitelisted adapter is, in fact, a GPS. But for six years this failure case remained theoretical. Until yesterday.

Yesterday, we got a plaintive bug report from a guy who was trying to mate a cute little Linux SBC called a Gumstix with a GSM modem. A perfectly reasonable thing to do if you were, say, trying to remote-control some kind of instrumentation package over a GSM cell network. Our guy complained that every time he plugged in one of his modems, gpsd would spam a bunch of data at it. This sometimes gave the modem indigestion.

Well, guess what? Both of the USB-to-serial adapters he tried using with his GSM modem were in our whitelist, and gpsd duly sent them the crapload of vendor-specific probe strings it uses to try to prod a GPS into returning an identifying response. So our friend tried removing the GPSD package from the Gumstix, only to find that the KDE environment depends on it. FAIL.

The workaround is to uninstall GPSD’s udev rules. But now he’s going to have to peg GPSD’s version, or his modems will fall down go boom again right after his next upgrade pulls a new GPSD package installation and installs a new copy of our udev rules.

This basically sucks all around. And there’s no good fix. And if there were one, the skeezy Pacific-Rim fly-by-nights who pump out the world’s production of GPS mice probably wouldn’t apply it. Sigh…another wonderful day in the life of GPSD.

UPDATE: The person who reported the bug says I misunderstood his use case, though apparently not in an essential way. You can read his description.

UPDATE2: Ah, seems I conflated two bug reports from different people, one of which really did involve a Gumstix.

125 thoughts on “Plug and Pray in GPS-land

  1. OK, so I have a Gumstix on order, and I’m going to interface it to GSM. Good to know these things before they bite you, so thanks.

    But why on earth would KDE depends on GPSD?

  2. What sort of dependency chain would cause KDE to break if GPSD was removed? Does KDE depend on something that also depends on GPSD, or did the removal of GPSD bork something it shouldn’t have in udev (possibly due to the user doing lots of fiddling there to get his modem to work)?

    I also appreciate this because I’m soon ordering my toys for this year, which will include a Gumstix Overo Tide with some expansion modules.

  3. Away from KDE sources so I can’t check right now, but I’d guess that the only parts of KDE that could use GPS are either KStars (astronomy/planetarium app) or digikam (photo manager, for geotagging). Neither of these are needed for the base desktop and apps, so any decent package manager should be able to remove them. Or rebuild them from source without GPS support.

  4. Is there any way to add GPS class of devices to USB standard?

  5. >Is there any way to add GPS class of devices to USB standard?

    Supposing we could, we’d still have to whitelist by vendor/product ID for the huge volume of existing devices that return class FF.

  6. Could you not limit the rate at which GPSD tries to “poke a GPS device” into identifying itself? (I assume the modem is failing due to being spammed with stuff at a [i]rate[/i] it cannot handle, not that the strings themselves cause the problem)
    The exact rate to use would, of course, be trial and error per device, but a configuration option is surely not impossible, with the default at “as fast as possible”.

  7. >Could you not limit the rate at which GPSD tries to “poke a GPS device” into identifying itself?

    We already rate-limit automatically, because some GPSes have tiny receive buffers.

    >configuration option is surely not impossible, with the default at “as fast as possible”.

    Configuration options are death if what you’re aiming at is plug-and-play operation.

  8. Interesting.

    The PL-2303, which has terrible implementation issues as it is, (and rather crappy drivers if you end up having to use it in Windows), tends to be found in a lot of things. I would have thought you’d have hit a case like this before.

    It’s very common in cellphone “tethering” cables (for those that don’t speak USB natively), as well as being incredibly common in cheap USB to “RS-232″ cables. It’s also been showing up on some of the Arduino boards or “TTL” cables that some of them use, so I’d expect that as the Arduino continues to catch on, eventually this’ll become more problematic (though a lot of those still use FTDI chips, thankfully)

  9. Presumably the reason you haven’t hit such a case before is that most devices can handle being spammed.

    Whilst not ideal, if the device responds in some recognisable way, rather than sent it all the various vendor-specific stuff to try and get it to return a response, can you not first check to see if it’s a known bad device?

    If it is, don’t sent it the vendor-specific stuff and remove it from the list of devices.

    There may be some common probes that could be used to identify non-GPS devices, but then again that approach would have its own set of potential problems so probably not something I’d recommend.

  10. Although even more unpleasantly hackish, might it be possible to have your autoconfig script grep dmesg for some indication that the device isn’t a GPS system? Perhaps you’d note that it detected that USB device, and then moments later a modem driver was associated therewith, or that it reported modem-like outgoing activities? Again, its Modemlichkeit should surely be reflected somewhere in dmesg? :-)

  11. I must be galactically dense.

    Are the modems this guy using returning the same vendor/product ID as the GPS device he is using?

    If not, then why can’t gpsd have a config file section to “blacklist” those IDs?

    If so, could you turn the problem on its head and come up with a probe string that would identify the modem without borking it or any GPS device? This could go into a “graylist” section.

    To be nice, you’d have the user run a command with a name like gpsd-blacklist, which would pop up a screen that says “Please plug in the non-GPS device now, then press Enter.” It would then send a message to gpsd telling it to ignore the next USB device detected. Then gpsd-blacklist would read the information about the device, check a database of IDs to determine what probe to send to safely ID the device, and add it to the black/graylist. Then whenever that particular device is plugged in, gpsd would see it in the black/graylist and not cause any problems.

    But to really do this right, the business of IDing devices like this ought to be abstracted away from gpsd. I’m thinking a udev plugin that determines by the vendor/product IDs what is the most persnickety piece of hardware known that could be using it, then probes for it, either positively IDs it or rules it out, then moves down the line to the next one. Of course, it’s quite possible that there are two or more devices within an ID group that can each be safely probed only with a string that will bork the other. In that case, there isn’t much to be done other than a popup that asks the human that just plugged the device in “Is this thing you just plugged in $foo or $bar?”, followed by “Do you promise never to plug in that other thing, so you won’t have to be asked this question ever again?”

  12. But now he’s going to have to peg GPSD’s version, or his modems will fall down go boom again right after his next upgrade pulls a new GPSD package installation and installs a new copy of our udev rules.

    This sort of case seems to come up occasionally; would it really be impossible for a package manager to understand “Yes, I want to have the latest and greatest package FOO but I want you to ignore dependency BAR until further notice regardless of what FOO says?”

    Or does that way lie madness?

  13. Shenpen: “What would a small, embedded Linux distro need KDE for?”

    I was wondering the same thing, but figured there was some reason for it. Gumstix does have a display module setup, so perhaps it’s for that.

    The Monster: “Are the modems this guy using returning the same vendor/product ID as the GPS device he is using?”

    The problem, as has been pointed out, is that most of these cheap GPS “mice” devices look like a generic PL-2303 serial port. I have a couple of those, a cellphone cable, and a USB RS-232 adapter that all show up AS THE EXACT SAME DEVICE, with the “base” vendor/ID of the PL-2303. And I know I’m not alone, having compared notes with more than a few people.

    Now, tell me how to write udev rules for something that’s the norm, not the exception?

    The Monster: “But to really do this right, the business of IDing devices like this ought to be abstracted away from gpsd. I’m thinking a udev plugin that determines by the vendor/product IDs what is the most persnickety piece of hardware known that could be using it, then probes for it, either positively IDs it or rules it out, then moves down the line to the next one. ”

    I’d agree with you if there weren’t so many devices that show up as the default ID. And by “so many”, I mean possibly dozens.

  14. This is from memory, but when I was using these devices at a previous job a couple of years back, almost every USB to serial adapter used the same chipset, from a company that escapes me, but I could probably track down. And the vendor had a utility that you could use to change the the product ID that the USB device reported. We ended up not going down that route, but I experimented with it for reasons similar to yours, and it did seem to work. That’s probably not a good solution in general, but it might be workable as a one-off, if it’s the same chipset.

    As an aside, Windows CE 5.0 really, really doesn’t like to have modems on a removable port.

  15. >Whilst not ideal, if the device responds in some recognisable way, rather than sent it all the various vendor-specific stuff to try and get it to return a response, can you not first check to see if it’s a known bad device?

    How? All we can tell about it is that there’s a PL-2303 in front of it. We have no information about the device itself; that’s the exact problem.

  16. Well the modem case is pretty easy – you send it an “AT” and if you get an “OK” back you’re almost 100% certain it’s a modem. But I don’t like sending strings to devices when I don’t know what they are, you never know when some random input crashes a device.

  17. In truth, we’d probably have to do this even if there were a GPS device class. You see, all the GPS mice we’ve ever seen are are lash-ups consisting of a receiver module that’s wedded by a sort of bastardized TTL version of RS-232 with an RS-232-to-USB adapter chip.

    *facepalm*

    Why are GPS manufacturers so stupid?

    Okay, so it’s a modem, right? Good. I wrote serial comm routines in the 1980s in assembler, so I’m an expert in this area. A modem, if it’s not online (connected to a remote host) will have its carrier-detect line cleared, while most other RS-232 equipment will have its carrier-detect always set — this is the case for RS-232 plotters and terminals (ttys) and I suspect it might be the case for GPS mice, but I don’t know for sure.

    Could you possibly check the carrier-detect line on the RS-232?

  18. >Could you possibly check the carrier-detect line on the RS-232?

    No, because the RS232 lines only exist as circuit-board traces inside the sensor case. All we got to see is the USB lines.

  19. Are you sure about this? All the RS-232 stuff is present in linux-source/drivers/usb/serials/pl2303.c Specifically there is this struct definition:

    /* All of the device info needed for the PL2303 SIO serial converter */
    static struct usb_serial_driver pl2303_device = {
    .driver = {
    .owner = THIS_MODULE,
    .name = “pl2303″,
    },
    .id_table = id_table,
    .usb_driver = &pl2303_driver,
    .num_ports = 1,
    .open = pl2303_open,
    .close = pl2303_close,
    .dtr_rts = pl2303_dtr_rts,
    .carrier_raised = pl2303_carrier_raised,
    .write = pl2303_write,
    .ioctl = pl2303_ioctl,
    .break_ctl = pl2303_break_ctl,
    .set_termios = pl2303_set_termios,
    .tiocmget = pl2303_tiocmget,
    .tiocmset = pl2303_tiocmset,
    .read_bulk_callback = pl2303_read_bulk_callback,
    .read_int_callback = pl2303_read_int_callback,
    .write_bulk_callback = pl2303_write_bulk_callback,
    .write_room = pl2303_write_room,
    .chars_in_buffer = pl2303_chars_in_buffer,
    .attach = pl2303_startup,
    .release = pl2303_release,
    };

  20. The 1980s were a long time ago, and let me tell you, you’re lucky if a device manufacturer gets TxD, RxD and GND right.

    Having taken apart more than a few USB-serial device widgets for hardware hacking purposes, the ONLY things that tend to end up connected between the device in question and the USB-serial chip are, in fact, TxD, RxD and GND. Hardware handshaking barely happens any more.

    This schema also assumes we only wish to differentiate between modems and GPS units, when we mostly wish to differentiate between GPS units and lots of other things that are *unlikely to be modems* as well.

  21. Well, JCB, I would assume since the pl2303.ko kernel module has a struct that shows these details, they must be able to be read from the device. I’m quite sure that the author of the Pacific Logic 2303 driver (someone at IBM, apparently) didn’t put that in there for S&G or hysterical raisins or whatever.

    Of course your last point is very valid, but something to consider is that RS-232-to-USB adapters are likely to be used mostly for modems, GPS, and other devices that are of a more niche nature. Given that in the years that GPSD has been around, this is the first problem of this kind, it’s reasonable to assume that we don’t see the PL2303 in too much else.

    I’m going for practicality here, not theoretical possibilities. Theories are for college textbooks. :)

  22. @esr: How? All we can tell about it is that there’s a PL-2303 in front of it. We have no information about the device itself; that’s the exact problem.

    What I meant was that it sounds like you’re trying to identify the GPS devices by sending a device various bits of information and expecting that if it sees something it recognises it’ll send something back (and thus you know it’s a device you can handle), the problem being that this particular device doesn’t handle that well.

    If this device responds in a particular way (e.g. if it responds to manufacturer, or even device, specific messages) if you send something specific to it (e.g. if (very simplified obviously) you send it HELP it responds with ACME CORP MODEM), you would have a way of identifying it and thus ignoring it.

    It’s the same problem as trying to identify the GPS device, except in this case you want to identify that it’s not a GPS device (and in fact is a specific device that isn’t a GPS).

  23. Mr Greywolf,

    Apparently I was unclear.

    The chip and the struct itself have those features, yes. Not debating that. I have the datasheet for it right here, in fact.

    What I am asserting is that they’re not getting used at a hardware connectivity level, and no, I’m not debating the theoretical. I’m saying that I have seen terrible interface decisions having been made, and I’m also saying that you cannot make any kind of assumption that the hardware pins are going to be set up in a sane and logical way. I still deal with RS-232 (and many bastardized implementations thereof) on a daily basis as part of my work.

    I’m basing my statements on things I’ve actually seen done. I’ve seen this chip in many applications, I’ve seen what embedded USB-Serial setups look like, and let me tell you, generally speaking, they’re not well behaved. At all. So please keep your attempts at belittlement to yourself in claiming I’m only speaking in “theoretical possibilities”.

    Just because GPSD hasn’t had to deal with it doesn’t mean it’s not problematic elsewhere.

  24. .dtr_rts = pl2303_dtr_rts,
    .carrier_raised = pl2303_carrier_raised,

    It is interesting that these methods exist, but by definition they’re only carrying information passed over the USB bus, not the actual RS232 wire state. There are three problems with relying on them for anything:

    1. As someone else has already pointed out, the RS232 handshake lines frequently aren’t connected to the USB interface chip. The GPSD project already knew this, because it affects a precision timekeeping feature called PPS. Many serial GPSes send a high-precision once-per-second pulse over an otherwise unused handshake line. We’ve never been able to observe this pulse on USB devices.

    2. Even if the hardware handshake were passed through to the interface chip, it’s not given that that the interface-chip driver would implement the method, nor that the implementation would be correct, nor even how one would test the implementation for correctness. This is exactly the sort of dark corner where bitrot strikes first and worst.

    3. Supposing by some special dispensation of a benevolent Hermes Trismegistus you overcame the first two problems….what semantics would you assign the state of the line, exactly? If it’s device-dependent (as it certainly will be) how do you make use of it in the all-to-frequent case that the GPS cannot be made to respond with a self-identification string?

  25. >It’s the same problem as trying to identify the GPS device, except in this case you want to identify that it’s not a GPS device (and in fact is a specific device that isn’t a GPS).

    OK, now I get it. Holy shit – you’re suggesting, in effect, that GPSD (or some utility spun off from GPSD) should know all the request/response pairs that might identify any device behind a class FF USB device, not just the GPSes.

    I, er, don’t think this is practical. Our GPS-only corner of it is difficult enough already. Just finding out what these strings might be is hard; they tend to be poorly documented or undocumented.

  26. “1. As someone else has already pointed out, the RS232 handshake lines frequently aren’t connected to the USB interface chip. The GPSD project already knew this, because it affects a precision timekeeping feature called PPS. Many serial GPSes send a high-precision once-per-second pulse over an otherwise unused handshake line. We’ve never been able to observe this pulse on USB devices.”

    It really wouldn’t help you even if they did. I tried hacking this up once on a USGlobalsat puck, tying the LED output (which is tied to the 1PPS out) to a handshaking line. The problem is that getting 1PPS timing via USB is useless due to the fact that timing isn’t preserved from the chip’s pin across USB, and you can get really oddball intervals between perceived handshaking pin changes, especially if the USB controller is busy.

    Eric, is this one of those cases where just merely shutting up GPSD, having it listen first instead for a NMEA string when the port is open (and eventually timing out) isn’t enough? Or do you have to poke those vendor-specific strings in there to even get them to speak NMEA? I can’t claim to know every USB GPS puck out there, which is why I ask.

  27. Monster,

    > I must be galactically dense.

    Since galaxies are mostly empty space, i.e. the complete opposite of dense: Don’t flatter yourself. :) No one is that much of a genius!

    Yours,
    Tom

  28. @esr: OK, now I get it. Holy shit – you’re suggesting, in effect, that GPSD (or some utility spun off from GPSD) should know all the request/response pairs that might identify any device behind a class FF USB device, not just the GPSes.

    Not exactly, I’m suggesting you only do this for the devices that cause problems — most of them appear to not be problematic, you only need worry about those that crash or behave in weird ways and that number should be manageable (implied by the fact you have only one reported case of this).

  29. >It really wouldn’t help you even if they did.

    Yes, I’m aware of this. That was sort of my point 2,. though I could have been more explicit about the timing issue.

    >Eric, is this one of those cases where just merely shutting up GPSD, having it listen first instead for a NMEA string when the port is open (and eventually timing out) isn’t enough? Or do you have to poke those vendor-specific strings in there to even get them to speak NMEA?

    Usually the puck will ship NMEA before you poke it, though I know of a couple of oddball exceptions to that rule (like the San Jose Navigation FV18). The main reason reason we ship probes at start is to find out if the receiver has an alternate binary mode that ships more information than the NMEA mode (SiRF and Zodiac chips, for example). Also, we can sometimes deduce the existence of capabilities to change the interface baud rate and the device’s sampling interval.

  30. >Not exactly, I’m suggesting you only do this for the devices that cause problems

    Yes, that would pretty much be class FF. Anything in that is a potential problem.

  31. @JCB

    What I am asserting is that they’re not getting used at a hardware connectivity level, and no, I’m not debating the theoretical. I’m saying that I have seen terrible interface decisions having been made,

    Sorry, I didn’t mean to belittle you. I didn’t quite get what you were talking about; I don’t have any GPS mice here at the moment, but if you say that they’re not wired that way, then I’ll have to assume it must be so barring any evidence to the contrary. The only thing I can say is that if GSM modems at all try to be “Hayes compatible”, they have to implement things the way dialup modems do, and that would include things like the DTR and DCD lines.

    As someone else poiinted out, if all else fails you can try sending a GSM modem an ‘ATI’ command and it will report back the manufacturer, revision, and other status information. But given that GSM modems at least outwardly appear compatible with the Hayes AT command set, then checking DCD or DTR as I’ve mentioned above might still be helpful in separating modems from GPSes.

  32. >As someone else poiinted out, if all else fails you can try sending a GSM modem an ‘ATI’ command and it will report back the manufacturer, revision, and other status information.

    I’m assuming there’s a fixed trailer of \r\n on an ATI response, but will it also have a fixed header part that our packet sniffer can count on? From this page, for example, it looks like there wouldn’t be.

  33. > Yes, that would pretty much be class FF. Anything in that is a potential problem.

    Potentially, yes, but at present you only know of one device — why not just check for this single specific device (by whatever method you choose) and either ignore it if you find it, or proceed as normal if it’s not that one?

    If / when you get reports of similar problems with different devices re-evaluate at that point — it might be another six years before you get another one!

  34. IIRC, sending an init string of “ATV1\r” should return back “OK\r\n”. (V1 turns verbose mode on, V0 return verbose mode off) If you check for that, that should be enough to assume it’s a modem.

    If the modem is not in verbose mode, then it will only send back numeric return codes followed by a bare ‘\r’.

    You’re right that the results of ATI will vary; but so long as no GPS devices support a Hayes AT command set, you can be pretty confident that the ‘OK\r\n’ isn’t a GPS device. Whether or not it’s a really modem or who the maker is, etc., is probably someone else’s problem.

  35. OTOH, if the modem is connected, then sending any AT commands to probe it may be problematic. Most AT compatible modems will jump to command mode if you send “+++”, wait 0.5 second, and then send “+++” again. Then you have to get it to return to ‘online’ mode by ‘ATO’ I think.

    Could be dicey though. :-/

  36. Why don’t you just ask the user? E.g. “You have connected a USB device that may be a GPS to your computer. Would you like to treat it as a GPS device, or have gpsd ignore it?” This reeks of bad design, regardless of your desire for plug-and-play compatibility.

  37. >Why don’t you just ask the user?

    Because the whole point of autoconfiguration is to spare users from having to know or think about such things.

  38. udev rules just start a program when a device matches. Whether this program starts sending data to the USB port or asks confirmation to the user before doing so doesn’t matter to udev.

  39. @wrm & Tim:
    KDE itself does not depend on gpsd, Fedora’s KDE does. It’s probably some kind of compile-it-all monolithic install and not modular. I use KDE on Gentoo and gpsd in not installed.

  40. udev rules just start a program when a device matches. Whether this program starts sending data to the USB port or asks confirmation to the user before doing so doesn’t matter to udev.

    udev rules run firstly at boot, and secondly when a USB device is plugged in. GPSD doesn’t know and doesn’t even care if the user’s machine is running a desktop environment. For all GPSD knows or cares, the user is running headless, which might very well be the case in some automotive, marine, or aerospace applications.

  41. IIRC, sending an init string of “ATV1\r” should return back “OK\r\n”.

    You are correct, but you’re also potentially changing something. Perhaps the user wanted that modem in the numeric response mode.

    So you send “AT\r” and either get a 0 or OK back. I would say that’s enough to ID the modem without any chance of borking it. If gpsd gets either response, stop probing and classify the device as ‘not GPS’.

  42. OTOH, if the modem is connected, then sending any AT commands to probe it may be problematic.

    Yes, but we’re talking about udev rules for when the device is first plugged in. At least that’s what we should be talking about.

    The problem here is these Frankenserial things that try to marry the RS-232 to USB that have no standard for identifying what’s behind the adapter. That, in turn, is because there are no standards for RS-232 devices IDing themselves. ESR’s goal of autoconfiguration that Just Works is simply not possible, given that all it takes is any device that doesn’t play by the rules incorporated into gpsd is subject to being hosed. That means you have to have some way to locally blacklist devices and have gpsd ignore them.

  43. What does “GPS mouse” mean?

    Plainly it’s some sort of term of art for some set or subset of USB GPS devices, but I’ve never seen it before, so the exact extent is unclear.

    I suppose my question is best re-phrased as “does GPS mouse just mean USB GPS device”?

    (I have a USB GPS device that I use with GPSD on OSX, but it’s a plain ol’ dongle and I don’t see any thing mouselike about it in any context. It obviously ain’t an HID device, so it’s not mouse-like at the USB level, and it’s not physically like a mouse…

    Does the term just mean “GPS device” and come from the hardware implementations that look like a puck on a string, and thus vaguely like a mouse?)

  44. >That means you have to have some way to locally blacklist devices and have gpsd ignore them.

    Interesting idea but how are you going to do that if (say) both the GPS and the whatever-else are presenting PL2303 identification and aren’t on fixed ports?

  45. >What does “GPS mouse” mean?’

    A GPS mouse is a GPS sensor with no outputs other than a serial or USB or CF connector, or Bluetooth capability (well, maybe it has an activation LED as well.). That is, as opposed to a handset or dashboard device with its own display.

    >Does the term just mean “GPS device” and come from the hardware implementations that look like a puck on a string, and thus vaguely like a mouse?)

    I think so. I’m not aware of any other etymology.

  46. Interesting idea but how are you going to do that if (say) both the GPS and the whatever-else are presenting PL2303 identification and aren’t on fixed ports?

    If a specific machine is faced with two devices that identify themselves identically, then you simply must abandon It Just Works and provide the user a means of identifying these devices to gpsd.

    I guess we can call that “Yellowlisting”: A yellowlisted device will not be interrogated, lest it be borked in the process. Instead, a popup will appear on the screen asking whether this device is a GPS, to be connected to GPSD, or something else entirely.

    I know you don’t like it, but LIS, it isn’t possible to provide It Just Works with non-identical devices that inexplicably identify themselves identically. And borking devices is bad form.

    Eventually, we might be able to get some pressure on the hardware guys to provide some mechanism by which you could send a command to the device to write an ID string to some NVRAM, so that proper udev rules could distinguish between them. That or have a serial number in addition to the vendor and model ID strings.

  47. Basically, the use of a RS232 serial adapter kinda screws you because RS232 serial devices have never had any standards for identification, with exception of stuff like modems (Hayes AT command set) and GPS devices (NMEA 0183), etc. All of these things are loosely defined, loosely followed standards. :-/

    Since we are talking about udev rules here, you probably don’t have to worry if a modem is connected or not, just send it an ‘ATV1\r’ and check for ‘OK\r\n’. That guarantees it’s a modem, not a GPS. For other serial devices, you’re basically hosed; there is no (supported) way to get an HPGL plotter, for example, to identify itself.

  48. >If a specific machine is faced with two devices that identify themselves identically, then you simply must abandon It Just Works and provide the user a means of identifying these devices to gpsd.

    OK, so how is this going to work, exactly?

    If you unconditionally spawn a popup for every class FF device, that destroys It Just Works for the overwhelmingly common case in which all the user ever plugs in is a USB GPS (remember, this bug report is our first such in six years, and GPSD is very widely deployed). This would not be an acceptable result.

    The only alternative to this would be to somehow conditionally raise a popup. But how would you do that? What information would you use?

  49. Because the whole point of autoconfiguration is to spare users from having to know or think about such things.

    So, because of your narrow view about user interfaces anyone who has gpsd installed has to put up with your software sending unwanted probes at unknown devices? This is clearly the wrong behaviour.

    There is no reason you cannot ask the user whether they just plugged a GPS in or not, and in the case that they have done so, perform your usual autoconfiguration. If the GPS is plugged in on startup the message would have to change, but the same interactiion applies.

  50. So (as has been mentioned) we can use ATV1 followed ATI0 to tell a hayes modem to give a product code. Assuming that it’s theoretically possible for some GPS manufacturer to implement hayes commands and still be a GPS, you could default to “its a modem” if ATV1 says 0/OK and then keep a list of product codes that aren’t modems (in this case the file would be wanting to say “XXX” = gpsd or something similar).

    Is there a GPS equivalent to ATI0? (i.e. a command that tells it to dump some form of device info) If there was I’d suggest a decent option would be to code into this spun off util (bearing in mind that we’re talking at udev level, just after device connection) as many ways as possible of retrieving a product code from the device and then have a udev-like list for “0xff product codes”. This will work best if for any possible 0xFF device you can get something thats a plausible product code.

  51. >Is there a GPS equivalent to ATI0? (i.e. a command that tells it to dump some form of device info)

    /me laughs helplessly.

    Er. No. No, there isn’t. If there were, we wouldn’t be throwing random probe strings at devices.

  52. >So, because of your narrow view about user interfaces anyone who has gpsd installed has to put up with your software sending unwanted probes at unknown devices? This is clearly the wrong behaviour.

    Yes, it’s so desperately wrong that it took us six years to trip over an instance. I tremble in shame. Not.

  53. >If you unconditionally spawn a popup for every class FF device, that destroys It Just Works for the overwhelmingly common case in which all the user ever plugs in is a USB GPS (remember, this bug report is our first such in six years, and GPSD is very widely deployed). This would not be an acceptable result.

    I’m not certain how this would be a problem. Simply have the standard behavior of gpsd be to ask the user whether they just plugged in a GPS. This is consistent with how many other drivers work; for instance, if you plug in a camera, it asks you if you want to open a photo manager to deal with it. When I plug in an iPod, it also gives me that dialog, because the iTouch uses the digital camera protocol; but since mine does not have a camera, I simply click no. It’s a small annoyance, but not a usability breaker. I suppose if you’re running on an embedded or command-line-only system it would be problematic, but surely it’s possible to detect that case and use the old behavior.

    In general, I think you’re probably trying to be too pure in Just Working. If the default configuration options are acceptable, or it autoconfigures to an acceptable condition, then I don’t see the problem with having other configuration options behind the scenes for more complex use cases.

  54. Yes, it’s so desperately wrong that it took us six years to trip over an instance. I tremble in shame. Not.

    No, it took 6 years for someone to report it. You do not know the true extent of the problems this may have caused.

  55. Personally, I’d be inclined, on a plugin event, to wait five seconds before doing anything. If a NMEA0183 sentence is received during that time, it’s a GPS, and you can probe it to your heart’s content. If not, probe with “AT\r” and wait a second for an “OK\r\n” or “0\r\n” response; if received, it’s a modem, ignore it; it not, send a probe string to get it to send a NMEA0183 sentence if it’s one of the GPSes that needs such, and act accordingly.

    ESR says: Ah, now there’s some creative thinking!

  56. @esr:

    In the Ubuntu modemmanager source package there is some Python source that queries hal/dbus for modem information. I wonder if there’s anything in there you could use?

    $ apt-get source modemmanager

    The file is modemmanager-*/tests/mm-test.py

  57. That assumes that the startup time for a GPS is well under five seconds. I don’t know how true that is. If not, you might need to set a different timer, but that’s the idea, at least. Let the GPS tell you it’s a GPS before you go asking it stuff. If only a couple of GPSes need poking to act like a GPS, then you’re not likely to lose by waiting.

  58. OK, so how is this going to work, exactly?

    If you unconditionally spawn a popup for every class FF device,

    Please re-read what I said. I was talking about “local blacklisting”, (as in the local admin. gets to decide what’s blacklisted; this is not a decision made by you on behalf of all users of gpsd) and then you asked how that would adapt to a situation where a specific machine had two devices that IDed the same but one was a GPS and the other would be borked by treating it like a GPS.

    I then proposed a variant on local blacklisting: local “yellowlisting”, which is the very specific situation in which someone insists on using these non-identical devices that ID themselves identically. Since your current design will unilaterally bork devices, we’re giving that person the OPTION to specify “these devices ID the same, so ASK ME what to do when one is plugged in”.

    It is not a general statement on how to deal with these devices for the overwhelming majority of cases where nothing gets borked. But I do believe that it’s good for you go give users the ability to tell gpsd to leave something the hell alone.

  59. > I do believe that it’s good for you go give users the ability to tell gpsd to leave something the hell alone.

    Yes. It’s always a good idea to let people turn off things which have a possibility of not working – which is everything.

    Yours,
    Tom

  60. True, Morgan. This method will likely lose if the modem is connected, but that’s not likely at plug or startup time.

    It’s possible to configure NetworkManager/ModemManager to connect upon insert, but I think if GPSD is getting the device handed to it by udev that NetworkManager/ModemManager isn’t going to get a chance to see it before GPSD does. But I can’t be sure about that.

    All in all, I like your idea of waiting 5 seconds for a NMEA 0183 sentence before you try to probe anything. I think what I just said about ModemManager, though, may be a sticky point that requires testing.

  61. Personally, I’m against the idea of waiting 5 seconds, or more.
    If I connect an USB/serial adapter and want to start screen or minicom or any other program that opens the serial port device I have to wait:
    - the 5 seconds of gpsd understanding it’s not a GPS
    - the ~10 seconds of modem-manager understanding it’s not a modem
    Because for all that time the serial port is open by some other program. That sucks.

    I’m on the other hand in favour of local blacklisting, or global greylisting + popup, or whatever else.

    For the case where an user has both a GPS, a modem and something else both with the same vid and pid:
    If it’s a pl2303, probably there’s no way to fix it, but if it’s an FT232 there’s a way: every FT232 has an unique serial number, see it in /var/log/messages:

    Jan 7 16:21:35 fede-tux kernel: [11737.823230] usb 5-1: new full speed USB device using uhci_hcd and address 2
    Jan 7 16:21:35 fede-tux kernel: [11738.007457] usb 5-1: New USB device found, idVendor=0403, idProduct=6001
    Jan 7 16:21:35 fede-tux kernel: [11738.007467] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
    Jan 7 16:21:35 fede-tux kernel: [11738.007474] usb 5-1: Product: FT232R USB UART
    Jan 7 16:21:35 fede-tux kernel: [11738.007479] usb 5-1: Manufacturer: FTDI
    Jan 7 16:21:35 fede-tux kernel: [11738.007484] usb 5-1: SerialNumber: A1003Ws6 <—The serial #
    Jan 7 16:21:35 fede-tux kernel: [11738.015566] ftdi_sio 5-1:1.0: FTDI USB Serial Device converter detected
    Jan 7 16:21:35 fede-tux kernel: [11738.015651] usb 5-1: Detected FT232RL
    Jan 7 16:21:35 fede-tux kernel: [11738.015656] usb 5-1: Number of endpoints 2
    Jan 7 16:21:35 fede-tux kernel: [11738.015662] usb 5-1: Endpoint 1 MaxPacketSize 64
    Jan 7 16:21:35 fede-tux kernel: [11738.015667] usb 5-1: Endpoint 2 MaxPacketSize 64
    Jan 7 16:21:35 fede-tux kernel: [11738.015672] usb 5-1: Setting MaxPacketSize 64
    Jan 7 16:21:35 fede-tux kernel: [11738.016517] usb 5-1: FTDI USB Serial Device converter now attached to ttyUSB3

    So that would allow to distinguish the three devices, assigning the first to gpsd, the second to modem-manager and leaving the third spam free.
    That would require user intervention, either as a popup *the first time only* (since the serial # can be added to a local whitelist), or using the blacklist strategy in case the user has a device that can't handle being spammed. Too bad the most common is the pl2303.

  62. Morgan, that was Jay “Tron Guy” Maynard quoting the new Tron movie.

  63. Russell,

    Go fuck a duck or something. You don’t hear the windows guys complaining about GPS.

  64. Grrr…who said that?

    I thought it was from the “6 dumbest ideas in computer security” but a couple of quick searches later and it turns up it’s a Tron: evolution thing.

    Which shouldn’t be that surprising given the source.

  65. Er. No. No, there isn’t. If there were, we wouldn’t be throwing random probe strings at devices.

    In my defense, i wasn’t assuming you’re totally retarded. I figure probing for a finite set of responses is going to be generally a better plan than keeping a list of every GPS device out there(which as you said earlier, isn’t really feasible). I only asked because a product code would be a useful differentiating feature in the 0xFF case.

    Let’s be clear here that the FAIL is in the USB committee.

    I assumed the problem wasn’t the USB committee itself but rather the hardware implementers doing their USB implementations on the cheap by half attaching it to a stock RS-232 to USB chip.

  66. On tuesday, 1/11/11 @ 11am (EST) esr will eat his words

    http://online.wsj.com/article/SB10001424052748704739504576068170230339348.html

    The iPhone commeth to VZW. This should make the iOS / Android drag race more fun to watch.

    I was at CES. Must’ve seen 30 companies hawking Android tablets. Some looked all but identical to the iPad. Sincere flattery there.

    Samsung has an iPod Touch clone that runs Android. No pricing announced, but 12 demo units out for the crowd to play with. LG had 8 different Android handsets, and 3 Windows phones displayed. Sony Ericson has 3 new Android phones.

  67. Grrr…who said that?

    I thought it was from the “6 dumbest ideas in computer security” but a couple of quick searches later and it turns up it’s a Tron: evolution thing.

    Which shouldn’t be that surprising given the source.

    Come now…could I have resisted an opportunity like that?

    I assumed the problem wasn’t the USB committee itself but rather the hardware implementers doing their USB implementations on the cheap by half attaching it to a stock RS-232 to USB chip.

    …and not using the chip’s facilities to assign a custom vendor/product ID.

  68. >Go fuck a duck or something. You don’t hear the windows guys complaining about GPS

    You get a warning for pointless nastiness to another commenter. Don’t do it again; I’ve been losing patience with your jejune trolling anyway.

    Windows guys don’t complain about GPS because Windows doesn’t try to do hotplug autoconfig.

  69. >On tuesday, 1/11/11 @ 11am (EST) esr will eat his words

    Yeah, sure. And how many times have we heard Verizon-iPhone-is-imminent in the last 18 months, only to have the hyperventilators change the subject when it didn’t happen?

    One of these times the rumors might be true, but I’ll believe it when I see it and not before.

  70. If it’s a pl2303, probably there’s no way to fix it, but if it’s an FT232 there’s a way: every FT232 has an unique serial number, see it in /var/log/messages:

    @fede.tft: Which makes blacklisting feasible on FT232, which is great. OTOH, as esr has pointed out, the pl2303 is used in something like 70% of USB GPS mice, which means that the FT232 would have to be far less common.

    I don’t like having to wait 5 additional seconds, either, but I agree with esr that a popup would have to be out of the question. Users shouldn’t have to explicitly specify that the device they just plugged in is a GPS if there is any possibility that the system can determine that for them automagically.

  71. I’m going to experiment with a simpler and more elegant variant of the wait-five-seconds idea.

    Currently gpsd starts by assuming that an unknown device is NMEA and immediately sending it probe strings. What I’m going to try is splitting the NMEA driver into true NMEA and Unknown, defaulting to Unknown, and sending probe strings only after gpsd has seen one NMEA packet and switched to the NMEA driver. The Unknown driver will be read-only.

    This means we will never ship probe strings to, e.g. USB modems, unless they just happen to write something at us that looks exactly like a properly checksummed NMEA packet. Which seems unlikely.

  72. >Eric, wanna bet?

    When you wrestle with a pig, both you and the pig get covered in mud, but only the pig enjoys it.

    That’s a “no”, because I don’t care to be in the definitional flamewar you will doubtless gin up about what constitutes autoconfig and how much Windows braindamage you expect us to ignore so you can score some silly debating point.

    I suggest you be very, very polite and respectful for a while. Your threadjacking and pointless rudeness to Russ Nelson comes on top of a bad prior history. You are sailing close to the ban threshold.

  73. Eric, your split seems reasonable…but how do you propose to not lose with the devices that have to be poked to send that first NMEA0183 sentence?

    As for the VZW iPhone, I’ll cite Fast Company columnist Austin Carr, who wrote “The point here is only to recommend that you take these rumors with a grain of salt. If you trusted every analyst prediction and release confirmation, then you’d likely end up with more years of angst and longing than the Tron Guy.” While I didn’t really have years of angst and longing, the overall point is valid.

  74. >Eric, your split seems reasonable…but how do you propose to not lose with the devices that have to be poked to send that first NMEA0183 sentence?

    I’m looking at that problem now. I’m pretty sure they’re all RS232 devices, so one (admittedly klugey) way to attack the problem would be to default to NMEA on an RS232 source and Unknown on USB and others.

  75. @Morgan Greywolf
    I don’t like having to wait 5 additional seconds, either, but I agree with esr that a popup would have to be out of the question.

    Ok, one more refinement for the FT232 case, completely without popups:
    You could keep a list of unique IDs for FT232 devices, marking if they are modems or not. Every time a *new* FT232 device is connected (one whose ID is not in the list) gpsd could open it and listen for say 10 seconds if it spits out NMEA. If yes, it is added to the list as a GPS device, otherwise it is added as a non GPS device. The next time the device is plugged in, gpsd can know *for sure* what to do: open it up and send probes immediately if it is in the list marked as GPS, or don’t even try to open it otherwise.
    It may be complicated, but it would be *optimal* for FT232 devices (that include Arduinos for example).

    @esr
    I’m going to experiment with a simpler and more elegant variant of the wait-five-seconds idea.
    [...] The Unknown driver will be read-only.

    Like this idea too. That would work well for pl2303. Assuming the “open the port and listen” time is kept short I guess most users won’t even notice.

  76. Jay Maynard Says: Personally, I’d be inclined, on a plugin event, to wait five seconds before doing anything. If a NMEA0183 sentence is received during that time, it’s a GPS, and you can probe it to your heart’s content. If not, probe with “AT\r” and wait a second for an “OK\r\n” or “0\r\n” response; if received, it’s a modem, ignore it; it not, send a probe string to get it to send a NMEA0183 sentence if it’s one of the GPSes that needs such, and act accordingly.

    ESR says: Ah, now there’s some creative thinking!

    OK. So now combine it with the popup idea and pop up a query while you are waiting 5 seconds. This lets the user know what the pause is for, and lets him or her short-circuit it if desired. Otherwise, you just go on after the 5 seconds. (5 seconds might even be too short, if the popup displays a countdown.)

  77. >You could keep a list of unique IDs for FT232 devices, marking if they are modems or not. Every time a *new* FT232 device is connected (one whose ID is not in the list) gpsd could open it and listen for say 10 seconds if it spits out NMEA. If yes, it is added to the list as a GPS device, otherwise it is added as a non GPS device. The next time the device is plugged in, gpsd can know *for sure* what to do: open it up and send probes immediately if it is in the list marked as GPS, or don’t even try to open it otherwise.

    Actually, I’ve been considering a similar technique for all of our whitelisted adaptors. It would require significant additional code complexity, however. The way to do it would be for gpsd to shove the serial number of any USB device that doesn’t self-identify as a GPS to a blacklist file, then check the blacklist on every USB open.

  78. Users shouldn’t have to explicitly specify that the device they just plugged in is a GPS if there is any possibility that the system can determine that for them automagically.

    But users should be able to explicitly specify that a device should not be probed the moment it’s plugged in. Frankly, I’m appalled by the attitude that it’s OK to send random strings of information to a device without a mechanism to opt out of the digital groping.

    To reiterate, what I’m proposing is that a config file exist with a [blacklist] section (if .ini style; adjust accordingly if it’s XML config) in which the user can specify ID strings to not be probed, and an [asklist] section in which the user can specify ID strings for devices they want to be asked about. The latter is specifically for the situation in which a particular person has two or more devices that ID themselves identically, one of which is a GPS and the other can be borked by the probe.

    The above is to be machine-specific, local configuration OPTION for the specific users who need it, not something that will in any way affect those users for whom plug-n-pray really does Just Work.

  79. @The Monster:

    The udev “scripts” (some are actually binaries) are run at boot in addition to on an insert event. This means that what you are effectively asking for is a prompt similar to what kudzu used to do on Fedora if you plugged in a new usb device. I hated this, everyone I know who had an opinion hated this. This has specifically caused problems in production environments running RHEL 5. It’s, IMHO, a bad, bad idea.

  80. @Morgan

    How else will you reconcile ESR’s stated dilemma of a machine that will at various times have USB devices plugged into it that ID themselves identically, one of which the user wants gpsd to talk to, but he insists that the other device not be connected to gpsd in any way? The USER is deciding that he has two devices that he refuses to allow gpsd to distinguish from one another, because the act of doing so borks one of them. He WILL NOT ALLOW the computer to tell them apart. No one in the world BUT THE USER is permitted to disambiguate them from one another in this instance.

    I am not saying it’s a good idea to do this for EVERY FSCKING THING that people plug into USB. I am limiting it to the VERY SPECIFIC instance outlined above. Before you call it a “bad, bad idea”, what better idea do you have to solve this problem?

  81. Facebook is Evil @6:51 Windows has an ongoing problem with USB serial ports being misidentified. Hook up an ADCP (which streams binary data once per second) to a Windows machine, and boot it up. Half the time it will identify the ADCP as a mouse, and start running your cursor all over the screen.

    Next time you’re wrong, be more politely wrong.

  82. ….

    Meanwhile, in Tuscon, AZ, a 10 year old girl was killed while a nutjob attempted to carry our Sarah Palin’s orders, at point blank range, (4 feet) and failed to inflict a fatal wound on the primary target, a US Representstive, and wife of an astronaut.

    Despite AZ having concealed carry (and a long period of legal open carry.)
    There was a carry permit holder at the incident, who returned fire, but the perp wasn’t hit.
    Instead, he was *tackled*.

    I think you should have to prove that you’re actually useful in a combat situation in order to legally carry.

  83. Russell,

    Seems to me Windows correctly identified a GPS ‘mouse’, and tried to make it a … Mouse!

  84. >I think you should have to prove that you’re actually useful in a combat situation in order to legally carry.

    Don’t police on the average mis 10 out of every 11 shots they fire?

  85. tmoney, I don’t know the exact figure but that sounds approximately right. However, civs (who tend to be better trained) only fare a little bit better. Go to the range, sprint 100 yards to the firing line, fire off a few double-taps at moving target while I scream in your ear, and see how well your carefully-honed aim holds up. Then you’ll have sampled a tiny fraction of the adrenaline that would be pumping in actual combat situation.

  86. Last I checked KMarble the KDE google earth clone depended on gpsd, but nothing else.

    Uninstalling KMarble or the kedu package (with all of kde’s educational tools) should do the trick.

    Of course it all kind of depends on which distro you’re using.

  87. Oh, I have no illusions over how poorly I would do, I just have objections to implications that just because this guy wasn’t brought down in a hail of bullets that it’s some indictment against carry in general, or the person there with a CC.

  88. @esr / Jay / The Monster:

    How else will you reconcile ESR’s stated dilemma of a machine that will at various times have USB devices plugged into it that ID themselves identically, one of which the user wants gpsd to talk to, but he insists that the other device not be connected to gpsd in any way?

    I’ve given this a little additional thought; suppose gpsd.hotplug does wait for an NMEA string, as esr is suggesting. Once udev has passed the device gpsd.hotplug, is there a way to tell udev “No, this is not the device I am looking for, please give it to someone else”?

  89. Once udev has passed the device gpsd.hotplug, is there a way to tell udev “No, this is not the device I am looking for, please give it to someone else”?

    Since udev can call any arbitrary program based on its rules, there’s no reason why that program couldn’t include a special disambiguation popup (smart enough to know if X is running, and work with the text console if it isn’t) for the rare case of USB devices that identically identify AND both need to be used on the same machine AND the user wants gpsd to talk to one of them BUT not the other.

    Of course, if the program udev calls sends “AT\r” to the device and looks for “0″ or “OK” as disqualifiers, before trying the probes, then no popup is required.

    I’m not saying that the popup is the optimal solution; far from it. I AM saying that a well-behaved daemon that distros will respect well enough to enable by default has to have SOME mechanism by which the user can excuse devices from probing. The popup is the explicit acknowledgement that It Just Works does not work with these Frankenserial devices. But the fault is not in the popup itself; it is the insane situation that requires it.

  90. OK, news from the front. I have succeeded in reducing wakeups fired at unknown devices to two rare cases (the Ashtech/Thales AC12 GPS and the TNT magnetic compass) both of which are R232 devices – and disabled wakeups on USB sources. So unidentified USB devices no longer get spammed.

  91. @esr
    So unidentified USB devices no longer get spammed.

    Good job, thanks.

  92. >So unidentified USB devices no longer get spammed.

    That’s right. They might still get grabbed and read from later, however. The losing scenario looks like this:

    1. You plug in a USB GSM modem with a PL2303 on it.

    2. Hotplug notifies gpsd, which stashes the path in its device list.

    3. You start using the modem.

    4. You fire up a GPSD client app, which emits ?WATCH.

    5. gpsd opens the modem device and attempts to sniff packets from it.

    gpsd’s attempts to sync-lock will fail, but not before it reads some data that should have gone elsewhere.

    Anybody know a fast, cheap way to find out how many processes have opened a given device?

  93. @esr:

    > Anybody know a fast, cheap way to find out how many processes have opened a given device?

    Um…. Perhaps this olde-skool method – GPSD tries to open the device for “exclusive read” (OR in O_EXCL to the 3rd arg to “open()”, IIRC) – if it fails, some other proc has already gotten to it.

    Just my $0.02. Probably I’ve overlooked some tiny, essential, nagging detail.

  94. Anybody know a fast, cheap way to find out how many processes have opened a given device?

    Look in /proc/**/fd for symlinks to it. But if “fast” means O(1) then I’m not sure.

  95. >Um…. Perhaps this olde-skool method – GPSD tries to open the device for “exclusive read” (OR in O_EXCL to the 3rd arg to “open()”, IIRC) – if it fails, some other proc has already gotten to it.

    That’s good enough. I’ll try it and see if it breaks.

    UPDATE: You forgot the same thing I did. O_EXCL only has defined behavior in conjunction with O_CREAT; it fails if the file exists.

  96. Look in /proc/**/fd for symlinks to it. But if “fast” means O(1) then I’m not sure.

    Yep. This is exactly what fuser(1) does. On Ubuntu/Debian, you can look at the scan_procs function in src/fuser.c in the psmisc package.

  97. > UPDATE: You forgot the same thing I did. O_EXCL only has defined behavior in conjunction with O_CREAT; it fails if the file exists.

    Why do you need O_CREAT? Surely the device node exists…

  98. @Last Word:

    He doesn’t. Duh. That’s why esr said “O_EXCL only has defined behavior in conjunction with O_CREAT”. That’s why he can’t use it.

  99. @esr, Morgan

    > UPDATE: You forgot the same thing I did. O_EXCL only has defined behavior in conjunction with O_CREAT; it fails if the file exists.

    Yah, I RTFM *after* posting, and was reminded of this behavior. I had tried to post again to point this out, and the system said the new post “failed a pre-condition”.

    Post in haste, repent at leisure. :-P

  100. >They might still get grabbed and read from later

    I’ve fixed this, under Linux at least. The serial-device open now grovels through /proc looking to see if any other process already has the device open; if so, the open fails.

    I’ve also used TIOCEXCL to lock each serial device after open. This will tell other (non-root) processes to back off – open(2) will return EBUSY.

  101. Monster:
    > Since udev can call any arbitrary program based on its rules, there’s no reason why that program couldn’t include a special disambiguation popup (smart enough to know if X is running, and work with the text console if it isn’t)

    udev is so *far* in the background, it’s almost inside the damn kernel; there is no practical way to decide on a good place to pop up a dialog box asking that, so far as i can tell; you *certainly* can’t do it on a non-X machine.

  102. Forgive my burying the lede: “…and even if you can decide on a good place, you probably can’t *get* a dialog box there.

    Modifying how udev *works* seems wildly outside gpsd’s project scope, any road.

  103. I’ve also used TIOCEXCL to lock each serial device after open. This will tell other (non-root) processes to back off – open(2) will return EBUSY.

    Your code implementing this change contains a race.

  104. >Your code implementing this change contains a race.

    I know. The only way that race would have been avoidable is if the O_EXCL mode flag for open(2) had the effect of TIOCEXCL when used without O_CREAT.

    As a recent Secretary of Defense might have put it, you go to war with the API you have, not the API you wish you had.

  105. Forgive my burying the lede: “…and even if you can decide on a good place, you probably can’t *get* a dialog box there.

    All you have to do is drop a note to gpsd to tell it what you have. Then gpsd can for the very specific case ESR outlined, of two or more devices that ID themselves identically, one of which gpsd is forbidden to probe, but the other it is desired that gpsd talk to, produce an X popup dialog or talk to the tty the user is currently on if X isn’t running.

    Again, this is a very specific situation that apparently no longer even applies, as ESR has figured out another way to disambiguate the devices that (at least for the purposes of his scenario) could not be disambiguated by the computer (the steps necessary to do so are forbidden).

  106. I know. The only way that race would have been avoidable is if the O_EXCL mode flag for open(2) had the effect of TIOCEXCL when used without O_CREAT.

    That depends upon which side-effects you’re worried about. You can obtain an exclusive file descriptor (amongst non-root fd’s) by setting TIOCEXCL and then checking for other open file descriptors, failing or retrying if they are present. The only race then is that applications may get a spurious EBUSY in the case where there turns out to be one or more pre-existing file descriptors; they will have to be able to deal with EBUSY in the non-spurious case anyway.

    I haven’t looked closely enough to know, but I assume you’re retrying in the event that you get either EBUSY or a pre-existing file descriptor. After all, other developers do not want to have gpsd fail upon losing a race for the device just because they wanted to probe it.

  107. >That depends upon which side-effects you’re worried about.

    Quite. In this case, it’s more important to keep gpsd from snarfing data meant for other processes than it is to keep other processes from snarfing data meant for gpsd. The worst effect of the latter case, after all, probably will be that an incoming packet or two gets torn up; one cycle later (usually within a second) another sample will come in.

  108. Quite. In this case, it’s more important to keep gpsd from snarfing data meant for other processes than it is to keep other processes from snarfing data meant for gpsd.

    Perhaps I’m confused, but neither of these cases will eventuate if you check for fd’s after locking. You will be the only process with an fd on the desired device. If you check before locking gpsd can read data that is sent in response to a request from another process, which you’re saying you want to avoid. The only downside is a possibly spurious period during with ‘open’ will return EBUSY to other applications. If you’re really worried about this, you can reduce that possibility to a race by checking for file descriptors both before and after locking.

  109. >Perhaps I’m confused, but neither of these cases will eventuate if you check for fd’s after locking.

    Ah, I see. You’re suggesting I open first, then TIOCEXCL, then count opens of the device and abort if > 1?

    That might work, yes – it’s harmless to have the device open if there haven’t been any reads. I need to stare at the code some more and think about failure cases.

  110. @esr/Roger:

    It’s worth noting that setting TIOCEXCL will only prevent processes not owned by root from opening the serial TTY device. root can always open it.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>