So, I announce ForgePlucker, and within a day I’ve got some guy
from Y Combinator sneering at me for using regular expressions to parse HTML. Says it’s “crappy code”. The poor fool…he has fallen victim to a conceptual trap which I, fortunately, learned to avoid decades ago. I could spout a freshet of theory about it, but instead I’m just going to utter a maxim: Never confuse structure with meaning.
I’ve been strongly hinting in recent blog entries that I planned to do something concrete about the data-jail problems of present open-source hosting sites. Because I believe in underpromising and overperforming, I decided at the outset not to announce a project until I could not only show working code, but code with wide enough coverage to make it crystal-clear that the project goals are achievable with a relatively modest amount of effort.
That time has arrived. I am very pleased to announce ForgePlucker, a project aimed at developing project-state extractor software for backup, offline analysis, and (eventually) re-importation. The proof-of-concept code can extract complete issue-tracker state from Berlios, Gna!, or Savane — and issue trackers are probably the hardest part of the job. I expect extraction of repository histories and developer permissions tables to be easier. Extraction of mailing-list state is probably a bit trickier than either of those, but doable.
I’m still not going to talk about my attack on the forge infrastructure problems quite yet; the software is coming along nicely, but I intend to announce only after it handles its fourth forge type (yes, that was a tease). But I will say this: I now think I know what the future of forges looks like. It’s called Roundup, and it is astonishingly elegant and potentially more powerful than anything out there. Anything, not excluding the clever decentralized systems like Fossil or Bugs Everywhere.
Here are the big wins:
1. Mailing lists, issue trackers, and online forums unify into *one* message queue that can be filtered in various ways.
2. Scriptable via XML-RPC or an email responder ‘bot.
3. Small base system with good extensibility – just three base classes (User. Msg, File) and the ability to define new classes. ‘Issue’ is a class built on top of these.
4. Arbitrary attributes per issue is basically free, with baked-in support for defining controlled vocabularies.
5. There’s a uniform way, called “designators”, for messages and other objects to refer to each other in text.
6. Small, clean implementation written in Python.
There are some things it needs, though… (Read the Roundup design document before continuing.)
Hackers have a lot of odd superstitions about software licensing. I was reminded of this recently when a project maintainer asked me whether he needed to get a sign-off from each and every one of his contributors before switching from Apache v1 to Apache v2. Here’s what I told him:
My opinion is this. Under U.S. law — and I believe European codes are not different in this respect, because both are controlled by the Berne convention — a license change on a collection is grounds for protest or legal action only if the rights of the contributors are materially affected by the change. That is, a court would have to be persuaded that the change caused a monetary loss or at least damage to a contributor’s public reputation. If there is no such possibility, then there is no harm and no grounds for complaint.
I am not generally unhappy with my model of how the universe works. Yes, it would be pleasant if there really were a beneficient creator-god and an afterlife; it would be nice if “good government” were actually a sustainable possibility rather than a fond but deluded hope in the minds of statists; it would be just peachy if wishes were horses and everybody could be happy and rich. But I’m reconciled to these things being not true because it seems to me they are necessarily not true – that is, for example, I cannot imagine a universe in which the actual existence of an intervening creator-god is actually compatible with the observed regularity of natural law, nor a universe in which scarcity and agency problems do not imply that governments are subject to the iron laws of political economics.
Other people can imagine theism or statism to be true, and so could I at one time – before I understood enough to grok the contradictions they would entail. But I’m not actually writing to argue for anarchism or atheism today; I brought those up as examples to make a more subtle point. It seems to me it’s actually more difficult to deal with unpleasant possible truths that are only contingent, because you can imagine how things could be different if you lived in a universe that was slightly different but recognizable.
I’ll give two examples of this that I’m still struggling with. One: I fear feminism may be dangerously wrong. Two: I fear that the crowd calling for the abolition of all forms of intellectual property may be dangerously right.
Following my post Condemning Censorship, Even of Werewolves, a great many people took me for task for trying to make a principled distinction between public space and private space, and tangled that up with a lot of confusion about the distinction between “public” and “state-owned” space. Here I’ll attempt to shed some light on the matter. In doing so, I will not actually be retailing a private theory of mine, but touching on principles with a long history in ethics and Anglo-American common law.
Thomas Paine once wrote: “He that would make his own liberty secure must guard even his enemy from oppression; for if he violates this duty he establishes a precedent that will reach to himself.” Paine’s truth is not idealistic handwaving, it is brutal pragmatism. Justifications for censorship, even the best-intentioned kind, have a way of expanding until they become instruments of abuse. Therefore, if we truly care about freedom of speech, it is not sufficient to defend that freedom when it is comfortable to do so — when the censors are ugly and the victim is appealing. It is necessary, sometimes, to speak up in defense of ugly victims of censorship.
I have found myself placed under that necessity in the last week. A member in good standing of the open-source community, one Beth Lynn Eicher, had sought and achieved the suppression of public speech by one Mikhail Kvaratskhelia, aka ‘mikeeeUSA’, aka serveral other aliases. When I first approached her privately on the matter, she refused to apologize or retract. In my judgment, she was committing a crime against our community’s future by setting a precedent which might one day reach to all of us.
This put me in a difficult position. I had received an explicit appeal for redress in my capacity as one of our tribal elders, and I felt the appeal was in the right and Ms. Eicher in the wrong. But I knew — for various reasons which will become very clear — that making that case would involve me in a batter and divisive wrangle. I was prepared to do it anyway, because my conscience would not permit otherwise, but I knew it was going to be hell.
Fortunately, after several days of debate among myself and some friends of mine who leapt to Ms. Eicher’s defense, Ms. Eicher proved to be cleverer than either them or me. While Ms. Eicher’s defenders were still flaming me for intransigence on the free-speech issue, she designed a solution which I consider totally appropriate, and which I actually hope will set a precedent.
In my previous two posts I have diagnosed a significant weakness in the open-source infrastructure. The architecture of the code behind the major SourceForge-descended hosting sites is rotten, with all kinds of nasty consequences — data seriously jailed, poor or completely absent capabilities near scripting and project migration. I said I was going to do something about it, and I’m working the problem now — actually writing code.
The rest of this post is not an announcement, because it will be mostly about things I’ have figured out I should not try to do. Yet. But it is a teaser. I see a path forward, and shortly I expect to have some working code to exhibit that shows the way. Actually, I have working code that attacks the problem in an interesting way now, but I’m still adding capabilities to make it a more impressive demonstration.
Here are some approaches I’ve considered, or had suggested to me by others, and rejected:
In my previous post, Three Systemic Problems With Open-Source Hosting Sites I identified some missing features that create serious brittleness in or project-hosting infrastructure. The question naturally arises, why don’t existing hosting systems already have these facilities? I have looked into this question, actually examining the codebases of Savane and GForge/FusionForge, and the answer appears to go back to the original SourceForge. It offered such exciting, cutting-edge capabilities that nobody noticed its internal architecture was a tar-pit full of nasty kluges. The descendants — Savane, GForge, and FusionForge — inherited that bad architecture.
I’ve been off the air for several days due to a hosting-site failure last Friday. After several months of deteriorating performance and various services being sporadically inaccessible, Berlios’s webspace went 404 and the Subversion repositories stopped working…taking my GPSD project down with them. I had every reason to fear this might be permanent, and spent the next two days reconstructing as much as possible of the project state so we could migrate to another site.
Berlios came back up on Monday. But I don’t trust it will stay that way. This weekend rubbed my nose in some systemic vulnerabilities in the open-source development infrastructure that we need to fix. Rant follows.
In my previous post, Why Artists Defend Roman Polanki, I analyzed the flap over the Roman Polanski arrest as a case of artists arguing for a privilege to behave like shitheels without being held to account for it. I advanced this as an explanation because I think it covers the facts better than some of the culture-war political narratives being bandied about, especially by conservatives, but I deliberately did not take a position on the rights and wrongs of the arrest or whether I think Polanski should be prosecuted at this late date.
Now I’ll do so. I expect it will startle almost all of my regulars and offend a good many of them, but I think Polanki should be let go.
In 1977, Roman Polanski drugged, raped, and sodomized a 13-year-old girl. When he believed a sort-of-plea-bargain was about to come unstuck, he took it on the lam. He lived the high life in this self-imposed exile for thirty years, until busted in Switzerland recently. Now various of the usual suspects on the right wing’s enemies list are campaigning to block his extradition.
There’s a good deal of perplexity being expressed about this, and some predictable chuntering from right-wingers about lefties being moral degenerates. But this flap isn’t really about politics at all — it’s much simpler than that. It’s about people who think of themselves as “artistes” reserving themselves a get-out-of-jail card when they feel like behaving like repellent scum of the earth, too.
In all the foofaraw surrounding the ACORN scandals, there is a huge important consequence of them understood – but not spoken – by everyone who follows politics as a blood sport. This story describing conditions in Michigan and tallying up some recent ACORN convictions for electoral fraud is an indicator. And, on top of Obama’s plummet in the polls, the attention now being focused on ACORN has got to have any thinking Democratic strategist deeply worried about the 2010 midterms.
No, I’m not talking about the mere aura of scandal, the prospect that some of the smell coming off of ACORN might cling to the general run of Democratic politicians. That’s not going to happen, not while most of the mainstream media seems ever more intent on operating as an unpaid auxiliary for the Democratic National Committee. No stench will be allowed to adhere, not even if the stalwart partisans of the Fourth Estate have to lick it off with their own tongues. No, the real problem is this: ACORN was the linchpin of the Democratic electoral-fraud machine. Without it, the party’s position going into the next round of elections may be seriously weakened.
A few moments ago, I read a review of a new book, Uncivil Society: 1989 and the Implosion of the Communist Establishment, and the following sentences jumped out at me:
This is less a story of dissidents, so-called civil society, than of the bankruptcy of a ruling classâ€“communismâ€™s establishment, or â€œuncivil society.â€ The Communists borrowed from the West like drunken sailors to buy mass consumer goods, then were unable to pay back the hard-currency debts and so borrowed even more. In Eastern Europe, communism came to resemble a Ponzi scheme, one whose implosion carries enduring lessons.
I found myself wondering “And this differs from our political class…how?”
In California Dreamin’ I wrote:
The U.S. as a whole will almost certainly face [California’s structural deficit] problem before the end of Barack Obamaâ€™s administration in 2012. Social Security obligations were due to exceed collections in 2013; even before Obama quadrupled the federal deficit this meant a giant blazing meteorite was already hurtling straight at the heart of the Fedsâ€™ dinosaurian finances.
It turns out the end is coming even sooner than I thought. The Congressional Budget Office now says Social Security will start running a deficit in 2010. And there is no money in Social Security’s so-called “trust fund” just bonds issued by the Federal Government itself. This means that in 2010, the Obama administration is going to face tough choices. It can (a) cut SS benefits, (b) raise taxes, or (c) kick the can down the road by borrowing more money to keep paying them.
This blog is one of many that got hit late last week by a particularly nasty and invasive worm targeting WordPress sites. (No, it wasn’t a botched upgrade, as I saw at least one commenter speculate.) The first symptoms showed up either late Thursday or early Friday of last week, when links from the main page became garbled. What was going on was an attempt to insert pharmaceutical-spam malware into the site permalinks.
This injection attack actually corrupted the mysql database behind the blog, and some fairly serious surgery (which the ibiblio site admins were reluctant to try on a Friday) was required to fix it. All posts and coments from Thursday evening or previous should now be restored. The blog is now running WordPress 2.8.4, the very latest version which was rush-released to foil the worm.
A big hand to Ken Chestnutt, the ibiblio site administrator who did the actual repair. Regular posting will resume shortly.
In The Book of Lies, the diabolically brilliant occultist Alesteir Crowley once wrote:
“Explain this happening!”
“It must have a natural cause!”
“It must have a supernatural cause!”
Let these two asses be set to grind corn!
In the original, there is a sort of grouping bracket connecting the second and third lines lines and pointing at the fourth. Crowley was asserting, in both lucid and poetic terms, that to the understanding mind the distinction between “natural” and “supernatural” is meaningless, an argument conducted about language categories with no predictive value.
There just isn’t any better marker for the fundamental narcissism of today’s left-wing politics that this:
That’s Che Guevara’s granddaughter, Lydia, We are told that she posed semi-nude to promote vegetarianism and People for the Ethical Treatment of Animals.
There are so many levels of wrongness and unintended irony here that it’s hard to know where to begin…
Today, Wired magazine gives us an article, Clive Thompson on the New Literacy, busting the supposedly conventional wisdom that cellphones, social networking, and the Internet in general have accelerated the decline of writing skills. The author says we’re actually in an age of rising literacy unparalleled since classical Greece. Er, what?
This morning I had the delightful “I was right all along!” experience of learning that the one grammatical bugaboo in my life is probably bogus. The next time a copy-editor invokes it on me I shall gleefully kick him or her in the snout…