I’ve been pounding on GPSD with the Coverity static analyzer’s self-build procedure for several days. It is my great pleasure to report that we have just reached zero defect reports in 72.8KLOC. Coverity says this code is clean. And because I think this should be an example unto others, I shall explain how I think others can do likewise.
I got emailed summaries from a Coverity scan of the repo head version of GPSD today.
In a recent Google+ comment, H. Peter Anvin grumped about GPSD using “braindead heuristics” to determine which USB devices it should sniff as possible GPses when it gets a hotplug notification saying that one has connected. I was going to reply in a comment there, but the explanation ran too long for that.
Short version: yes, GPSD will very occasionally sniff at a device that is none of its business. We’re stuck in a bad place because of deficiencies in the USB standard, But it doesn’t happen often, and all the alternative behaviors I’ve been able to imagine would be worse in very obvious ways. Detailed explanation follows.
Hacking on the C code of giflib after an absence of nearly two decades has been an interesting experience, a little like doing an archeological dig. And not one that could readily be had elsewhere; nowhere other than under Unix is code that old still genuinely useful under any but carefully sandboxed conditions. Our reward for getting enough things about API design right the first time, one might rightly say. But what’s most interesting is what has changed, and giflib provides almost ideal conditions for noticing the changes in practice that have become second nature to me while that code has stood still.
In 1994 I handed off the maintainership of giflib, the open-source library used by pretty much everything in the universe that displays images for the single most widely used icon and image format on the World Wide Web, because patent issues made it unwise for the project to be run by someone in the U.S. Now, eighteen years later, Toshio Kuratomi (the hacker who took it over then) has asked me to resume the lead. I have accepted his request.
I guess it’s paleo-game theme week. For your retrocomputing pleasure, here’s my Python forward-port of the 1973 University of Texas FORTRAN Trek game: Super Star Trek.
Anybody old enough to remember TTYs probably played this on one. While it has accreted some features over time, it’s still functionally pretty close to the original FORTRAN Star Trek. You kids should
get off my lawn try it, too – it retains considerable play value despite the primitive interface.
Recent discussion of the 4X game Eclipse reminded me of a responsibility. I’ve just shipped VMS Empire 1.9. This is a close descendent of the original solitaire Empire computer game that was the ur-ancestor of all 4X computer games, including Civilization and Master of Orion.
Five weeks ago I wrote that direct Subversion support in reposurgeon is coming soon. I’m waiting on one final acceptance test before I ship an official 2.0; in the meantime, for those of you kinky enough to find the details exciting, description follows of why this feature has required such a protracted and epic struggle. With (perhaps entertaining) rambling through the ontology of version control systems, and at least one lesson about good software engineering practice.
An interesting question showed up in my mailbox today. So interesting that I think it’s worth a public answer and discussion:
In chapter 7 of The Art of Unix Programming, you classified threads under the section “Problems and Methods to Avoid”. You also wrote that with the increased emphasis on thread-local storage, threads are looking more like a controlled use of shared memory. This trend has certainly continued; recent programming languages like D, Scala, and Go encourage the use of threads as mostly isolated lightweight processes with message passing. Observing this trend, I have often wondered, why not go all the way and use multiple OS processes? I can think of two reasons to use threads in this newer, controlled way rather than using full processes:
1. Portability to Windows, which doesn’t have an equivalent of fork(2)
2. Performance, particularly because message passing between real processes requires serialization and deserialization, whereas message passing within a process can be done with shared memory and (maybe) locks
So what do you think? Are threads still a menace to be avoided in favor of full OS processes? Or has the situation improved since 2003?
I think it has, and I think you’ve very nearly answered your own question as to why. Bare threads were dangerously prone to deadlocks, livelocks, context-trashing, and various other sorts of synchronization screwups – so language designers set out to encapsulate them in ways that gave better invariants and locality guarantees without sacrificing their performance advantages. I think Scala’s transactional memory stands out as a particularly elegant stab at the problem.
I don’t develop for Windows or communicate much with people who do, so I’m not equipped to judge how important Windows portability is in motivating these features. But the performance issue you called out is real and quite alive on Unix systems.
UPDATE: Matt Campbell, who has materialized in the comments here, send the original question and has given me permission to cite him. Thanks for a good question!
For those of you who have been following the development of reposurgeon, a pre-announcement: the next version, probably to be numbered 2.0, will directly read Subversion dumpfiles and repositories.
I’ve got this feature working now – it’s why my blogging has been scant recently – but I intend to have a really good regression-test suite in place and at least one large repo conversion done before I ship it for general use.
Note an important limitation: it will not write Subversion repos. So it will be useful as a conversion tool but not directly as an editor.
I’ve spent a lot of time and bandwidth on this blog thinking out loud about version-control systems and software forges. In my last post, I announced that I was going to try to sneak up on the problem of designing a better software forge by enhancing Roundup.
Over the last three years I’ve gotten a couple different versions of the following response to my thinking-out-loud: “Centralized forges and bugtracking are old-school thinking, as hoary as centralized VCSes. Why shouldn’t all that metadata live in the project repo and be peer-merged on demand the way code is?”
This is a good question, but I think the people advocating systems like Bugs Everywhere, scmbug, and ticgit have invested a lot of cleverness in the wrong answer.
I’ve written before about the problems with today’s software-forge sites – how they’re craptacular piles of PHP driving direct SQL queries with almost zero scriptability that become data jails for open-source projects. I’ve hinted that I think there’s a potential solution based on Roundup, a brilliantly simple and powerful message queue manager disguised as a mere issue tracker.
Well, I thought I was done hacking on this for a while. Then one of the projects I did a conversion for disclosed the existence of a second repo for their website, which I had to merge into the code repo. As a subdirectory. Which meant pushing all the file paths into a subdirectory. Which meant the new “paths sub” command; I wrote “paths sup” as its natural dual.
Also in this release: automatic preservation of untracked files under git and hg.
reposurgeon 1.8 is out, and with this release it has all the conversion features I’ve been able to think up while doing the last couple of conversions. This version creates real tags from the lightweight tags generated by git-svn, and also consolidates matched D/A pairs from Subversion into renames.
An “edit multiline” variant of the “edit” command zeroes in on commit comments that need to be tweaked into the approved form for hg and git (summary line, plus optional blank line, plus optional details).
The selection-set syntax has a new element: =H selects tip (or H for head) commits.
A new ‘sort’ command can make the DAG after a graft or merge display better in tools such as gitk.
With this release, I think I’m done for a while – barring bug reports, of course. I’m shipped a new version of my DVCS Migration Guide to go with it.
reposurgeon 1.7 is out. Fewer obvious changes this time; the big feature is that it knows how to read and use the CVS revision maps generated by the -R option of git-cvsimport. This means that it can patch CVS revision references into an action-stamp form that makes sense in a VCS-independent way.
freshmeat.net abruptly changed its name to freecode.com a couple of days ago. As a consequence, the little program I wrote to submit release announcements to it is now renamed freecode-submit.
People who ship releases frequently enough to find freecode-submit essential might also want to look at shipper, which I wrote to automate other aspects of release shipping as well.
shipper is how, when I want to ship a release of one of my projects, I can normally just type “make release” and the right things will happen – webpage updates, freecode release notification, SourceForge release, and release-tagging in the project repository.
This is a request for comment on a convention for uniquely identifying user actions on the Internet. The motivating context was identifying commit changesets in version-control systems in a way independent of the specific VCS. It is anticipated that this format will have uses in recording many other similar sorts of transactions, including actions on web interfaces, where we want a simple cookie identifying “who did this and when”.
I’ve been doing a lot of repository conversions recently, lifting ancient project histories from Subversion or even CVS into modern distributed version control systems. I’ve written about the technical problems with these conversions elsewhere but they also raise issues that are almost philosophical – and not unlike, actually, the challenges natural-language translators face moving a literary work between human languages.