I’ve been blog-silent the last couple of days because I’ve been chasing down the bug I mentioned in Request for help – I need a statistician.
I have since found and fixed it. Thereby hangs a tale, and a cautionary lesson.
GPSD has a serious bug somewhere in its error modeling. What it effects is position-error estimates GPSD computes for GPSes that don’t compute them internally themselves and report them on the wire. The code produces plausible-looking error estimates, but they lack a symmetry property that they should have to be correct.
I need a couple of hours of help from an applied statistician who can read C and has experience using covariance-matrix methods for error estimation. Direct interest in GPS and geodesy would be a plus.
I don’t think this is a large problem, but it’s just a little beyond my competence. I probably know enough statistics and matrix algebra to understand the fix, but I don’t know enough to find it myself.
Hundreds of millions of Google Maps users might have reason to grateful to anyone who helps out here.
UPDATE: Problem solved, see next post.
In a blog post on Computational Knowledge and the Future of Pure Mathematics Stephen Wolfram lays out a vision that is in many ways exciting and challenging. What if all of mathematics could be expressed in a common formal notation, stored in computers so it is searchable and amenable to computer-assisted discovery and proof of new theorems?
Infinite Science Fiction One (edited by Dany G. Zuwen and Joanna Jacksonl Infinite Acacia) starts out rather oddly, with Zuwen’s introducton in which, though he says he’s not religious, he connects his love of SF with having read the Bible as a child. The leap from faith narratives to a literature that celebrates rational knowability seems jarring and a bit implausible.
That said, the selection of stories here is not bad. Higher-profile editors have done worse, sometimes in anthologies I’ve reviewed.
Collision of Empires (Prit Buttar; Osprey Publishing) is a clear and accessible history that attempts to address a common lack in accounts of the Great War that began a century ago this year: they tend to be centered on the Western Front and the staggering meat-grinder that static trench warfare became as outmoded tactics collided with the reality of machine guns and indirect-fire artillery.
Concentration on the Western Front is understandable in the U.S. and England; the successor states of the Western Front’s victors have maintained good records, and nationals of the English-speaking countries were directly involved there. But in many ways the Eastern Front story is more interesting, especially in the first year that Buttar chooses to cover – less static, and with a sometimes bewilderingly varied cast. And, arguably, larger consequences. The war in the east eventually destroyed three empires and put Lenin’s Communists in power in Russia.
A Call To Duty (David Weber, Timothy Zahn; Baen Books) is a passable extension of Baen Book’s tent-pole Honorverse franchise. Though billed as by David Weber, it resembled almost all of Baen’s double-billed “collaborations” in that most of the actual writing was clearly done by the guy on the second line, with the first line there as a marketing hook.
The Abyss Beyond Dreams (Peter F. Hamilton, Random House/Del Rey) is a sequel set in the author’s Commonwealth universe, which earlier included one duology (Pandora’s Star, Judas Unchained) and a trilogy (The Dreaming Void, The The Temporal Void, The Evolutionary Void). It brings back one of the major characters (the scientist/leader Nigel Sheldon) on a mission to discover the true nature of the Void at the heart of the Galaxy.
The Void is a pocket universe which threatens to enter an expansion phase that would destroy everything. It is a gigantic artifact of some kind, but neither its builders nor purpose are known. Castaway cultures of humans live inside it, gifted with psionic powers in life and harvested by the enigmatic Skylords in death. And Nigel Sheldon wants to know why.
Sometimes reading code is really difficult, even when it’s good code. I have a challenge for all you hackers out there…
Better Identification of Viking Corpses Reveals: Half of the Warriors Were Female insists an article at tor.com. It’s complete bullshit.
What you find when you read the linked article is an obvious, though as it turns out a superficial problem. The linked research doesn’t say what the article claims. What it establishes is that a hair less than half of Viking migrants were female, which is no surprise to anyone who’s been paying attention. The leap from that to “half the warriors were female” is unjustified and quite large.
There’s a deeper problem the article is trying to ignore or gaslight out of existence: reality is, at least where pre-gunpowder weapons are involved, viciously sexist.
Yesterday I shipped cvs-fast-export 1.15, with a significant performance improvement produced by replacing a naive O(n**3) sort with a properly tuned O(n log n) version.
In ensuing discussion on G+, one of my followers there asked if I thought this was likely to produce a real performance improvement, as in small inputs the constant setup time of a cleverly tuned algorithm often dominates the nominal savings.
This is one of those cases where an intelligent question elicits knowledge you didn’t know you had. I discovered that I do believe strongly that cvs-fast-export’s workload is dominated by large repositories. The reason is a kind of adverse selection phenomenon that I think is very general to old technologies with high exit costs.
The rest of this blog post will use CVS as an example of the phenomenon, and may thus be of interest even to people who don’t specifically care about high version control systems.
Our new cat Zola, it appears, has a mysterious past. The computer that knows about the ID chip embedded under his skin thinks he’s a dog.
There’s more to the story. And it makes us think we may have misread Zola’s initial behavior. I’m torn between wishing he could tell us what he’d been through, and maybe being thankful that he can’t. Because if he could, I suspect I might experience an urge to go punch someone’s lights out that would be bad for my karma.
I just had a rather hair-raising experience with a phase-of-moon-dependent bug.
I released GPSD 3.11 this last Saturday (three days ago) to meet a deadline for a Debian freeze. Code tested ninety-six different ways, run through four different static analyzers, the whole works. Because it was a hurried release I deliberately deferred a bunch of cleanups and feature additions in my queue. Got it out on time and it’s pretty much all good – we’ve since turned up two minor build failures in two unusual feature-switch cases, and one problem with the NTP interface code that won’t affect reasonable hardware.
I’ve been having an extremely productive time since chewing through all the stuff I had deferred. New features for gpsmon, improvements for GPSes watching GLONASS birds, a nice space optimization for embedded systems, some code to prevent certain false-match cases in structured AIS Type 6 and Type 8 messages, merging some Android port tweaks, a righteous featurectomy or two. Good clean fun – and of course I was running my regression tests frequently and noting when I’d done so in my change comments.
Everything was going swimmingly until about two hours ago. Then, as I was verifying a perfectly innocent-appearing tweak to the SiRF-binary driver, the regression tests went horribly, horribly wrong. Not just the SiRF binary testloads, all of them.
Yes, I’m aware of the spam on the blog front page. The management does not hawk dubious drugs.
Daniel Franke and I just did an audit and re-secure of the blog last night, so this is a new attack. Looks like a different vector; previously the spam was edited into the posts and invisible, this time it’s only in the front-page display and visible.
It’s a fresh instance of WordPress verified against pristine sources less than 24 hours ago, all permissions checked. Accordingly, this may be a zero-day attack.
Daniel and I will tackle it later tonight after his dinner and my kung-fu class. I’ll update this post with news.
UPDATE: The initial spam has been removed. We don’t know where the hole is, though, so more may appear.
UPDATE2: It’s now about 6 hours later and spam has not reappeared. I changed my blog password for a stronger one, so one theory is that the bad guys were running a really good dictionary cracker.
Once Dead (Richard Phillips; Amazon Publishing) is a passable airport thriller with some SF elements.
Jack Gregory should have died in that alley in Calcutta. Assigned by the CIA to kill the renegade reponsible for his brother’s death, he was nearly succeeding – until local knifemen take a hand. Bleeding, stabbed and near death, he is offered a choice: die, or become host to Ananchu – an extradimensional being who has ridden the limbic systems of history’s greatest slayers.
Before you read any further, go look at the drawing accompanying the New Tork Times article on the autopsy of Michael Brown,
There’s a story in that picture. To read it, you have to be familiar with pistol shooting and the kind of pistol self-defense training that cops and amateur sheepdogs like me engage in.
In the remainder of this post I’m going to walk you through the process of extracting the story from the picture.
One of the reasons I like cats is because I find it enjoyable to try to model their thought processes by observing their behavior. They’re like furry aliens, just enough like us that a limited degree of communication (mostly emotional) is possible.
Just now I’m contemplating a recent change in the behavior of our new cat, Zola. Recent as in the last couple of days. Some kind of switch has flipped.
I join my voice to those of Rand Paul and other prominent libertarians who are reacting to the violence in Ferguson, Mo. by calling for the demilitarization of the U.S.’s police. Beyond question, the local civil police in the U.S. are too heavily armed and in many places have developed an adversarial attitude towards the civilians they serve, one that makes police overreactions and civil violence almost inevitable.
But I publish this blog in part because I think it is my duty to speak taboo and unspeakable truths. And there’s another injustice being done here: the specific assumption, common among civil libertarians, that police overreactions are being driven by institutional racism. I believe this is dangerously untrue and actually impedes effective thinking about how to prevent future outrages.
In the Kivila language of the Trobriand Islands there is a lovely word, “mokita”, which means “truth we all know but agree not to talk about”. I am about to speak some mokitas.
Unexpected Stories (Octavia Butler; Open Road Integrated Media) is a slight but revealing work; a novelette and a short story, one set in an alien ecology among photophore-skinned not-quite humans, another set in a near future barely distinguishable from her own time. The second piece (Childfinder) was originally intended for publication in Harlan Ellison’s never-completed New Wave anthology The Last Dangerous Visions; this is its first appearance.
These stories do not show Butler at her best. They are fairly transparent allegories about race and revenge of the kind that causes writers to be much caressed by the people who like political message fiction more than science fiction. The first, A Necessary Being almost manages to rise above its allegorical content into being interesting SF; the second, Childfinder, is merely angry and trite.
I shipped point releases of cvs-fast-export and reposurgeon today. Both of them are intended to fix some issues around the translation of ignore patterns in CVS and Subversion repositories. Both releases illustrate, I think, a general point about software engineering: sometimes, it’s better to punt tricky edge cases to a human than to write code that is doomed to be messy, over-complex, and a defect attractor.