Mar 09

shellcheck: boosting the signal

I like code-validation tools, because I hate defects in my software and I know that there are lots of kinds of defects that are difficult for an unaided human brain to notice.

On my projects, I throw every code validater I can find at my code. Standbys are cppcheck for C code, pylint for Python, and go lint for Go code. I run these frequently – usually they’re either part of the “make check” I use to run regression tests, or part of the hook script run when I push changes to the public repository.

A few days ago I found another validator that I now really like: shellcheck Yes, it’s a lint/validator for shell scripts – and in retrospect shell, as spiky and irregular and suffused with multilevel quoting as it is, has needed something like this for a long time.

I haven’t done a lot of shell scripting in the last couple of decades. It’s not a good language for programming at larger orders of magnitude than 10 lines or so – too many tool dependencies, too difficult to track what’s going on. These problems are why Perl and later scripting language became important; if shell had scaled up better the space they occupy would have remained shell code as far as the eye can see.

But sometimes you write a small script, and then it starts to grow, and you can end up in an awkward size range where it isn’t quite unmanageable enough to drive you to port it to (say) Python yet. I have some cases like this in the reposurgeon suite.

For this sort of thing a shell validater/linter can be a real boon, enabling you to have much more confidence that you’ll catch silly errors when you modify the script, and actually increasing the upper limit of source-line count at which shell remains a viable programming language.

So it is an excellent thing that shellcheck is a solid and carefully-thought-out piece of work. It does catch lot of nits and potential errors, hardening your script against cases you probably haven’t tested yet. For example. it’s especially good at flagging constructs that will break if a shell variable like $1 gets set to a value with embedded whitspace.

It has other features you want in a code validator, too. You can do line-by-line suppression of specific spellcheck warnings with magic comments, telling the tool “Yes, I really meant to do that” so it will shut up. This means when you get new warnings they are really obvious.

Also, it’s fast. Fast enough that you can run it on all your shellscripts up front of all your regular regression tests and probably barely ever notice the time cost.

It’s standard practice for me to have a “make check” that runs code validators and then the regression tests. I’m going back and adding shellcheck validation to those check productions on all my projects that ship shell scripts. I recommend this as a good habit to everybody.

Feb 27

The right to be rude

The historian Robert Conquest once wrote: “The behavior of any bureaucratic organization can best be understood by assuming that it is controlled by a secret cabal of its enemies.”

Today I learned that the Open Source Initiative has reached that point of bureaucratization. I – OSI’s co-founder and its president for its first six years – was kicked off their lists for being too rhetorically forceful in opposing certain recent attempts to subvert OSD clauses 5 and 6. This despite the fact that I had vocal support from multiple list members who thanked me for being willing to speak out.

It shouldn’t be news to anyone that there is an effort afoot to change – I would say corrupt – the fundamental premises of the open-source culture. Instead of meritocracy and “show me the code”, we are now urged to behave so that no-one will ever feel uncomfortable.

The effect – the intended effect – is to diminish the prestige and autonomy of people who do the work – write the code – in favor of self-appointed tone-policers. In the process, the freedom to speak necessary truths even when the manner in which they are expressed is unpleasant is being gradually strangled.

And that is bad for us. Very bad. Both directly – it damages our self-correction process – and in its second-order effects. The habit of institutional tone policing, even when well-intentioned, too easily slides into the active censorship of disfavored views.

The cost of a culture in which avoiding offense trumps the liberty to speak is that crybullies control the discourse. To our great shame, people who should know better – such as the OSI list moderators and BOD – have internalized anticipatory surrender to crybullying. They no longer even wait for the soi-disant victims to complain before wielding the ban-hammer.

We are being social-hacked from being a culture in which freedom is the highest value to one in which it is trumped by the suppression of wrongthink and wrongspeak. Our enemies – people like Coraline Ada-Ehmke – do not even really bother to hide this objective.

Our culture is not fatally damaged yet, but the trend is not good. OSI has been suborned and is betraying its founding commitment to freedom. “Codes of Conduct” that purport to regulate even off-project speech have become all too common.

Wake up and speak out. Embrace the right to be rude – not because “rude” in itself is a good thing, but because the degenerative slide into suppression of disfavored opinions has to be stopped right where it starts, at the tone policing.

The OSI membership page is here.

Feb 21

Reposurgeon defeats all monsters!

On January 12th 2020, reposurgeon performed a successful conversion of its biggest repository ever – the entire history of the GNU Compiler Collection, 280K commits with a history stretching back through 1987. Not only were some parts CVS, the earliest portions predated CVS and had been stored in RCS.

I waited this long to talk about it to give the dust time to settle on the conversion. But it’s been 5 weeks now and I’ve heard nary a peep from the GCC developers about any problems, so I think we can score this as reposurgeon’s biggest victory yet.

Continue reading

Feb 15

The reality of the lizard people

One of the loonier and more wonderful conspiracy theories floating around the Internet is it many of the world’s elite are shapeshifting lizardoid extraterrestrials. This explainer seems to sum it up pretty well.

When I first encountered this idea I was gobsmacked. How could anyone actually believe such a thing? And yet, apparently, many people do – millions of them, if polls on the topic are to be believed.

How humans form and maintain insane beliefs when there are plenty of objective reasons to know better is, I fear, a topic of continuing fascination to me. If only because when contagious and totalizing forms of insanity like Marxism or supernaturalist religions motivate the behavior of mobs they pose a significant threat to my survival.

The lizard-people theory isn’t in that class of danger, but I think cases like it and (for example) flat-Earthism are worth analysis precisely because they’re so implausible and still manage to attract adherents. Extremes like this can be revealing about mechanisms that are harder to see closer to the ordinary.

And indeed when I was mulling over lizard-people theory a few years ago I think I really did get a significant insight about the psychology of belief and what lizard-people conspiracy theory actually means.

Continue reading

Feb 07

Chinese bioweapon II: Electric Boogaloo

Yikes. Despite the withdrawal of the Indian paper arguing that the Wuhan virus showed signs of engineering, the hypothesis that that it’s an escaped bioweapon looks stronger than ever.

Why do I say this? Because it looks like my previous inclination to believe the rough correctness of the official statistics – as conveyed by the Johns Hopkins tracker – was wrong. I now think the Chinese are in way deeper shit than they’re admitting.

Continue reading

Jan 31

Head-voice vs. quiet-mind

I’m utterly boggled. Yesterday, out of nowhere, I learned of a fundamental divide in how peoples’ mental lives work about which I had had no previous idea at all.

From this: Today I Learned That Not Everyone Has An Internal Monologue And It Has Ruined My Day.

My reaction to that title can be rendered in language as – “Wait. People actually have internal monologues? Those aren’t just a cheesy artistic convention used to concretize the welter of pre-verbal feelings and images and maps bubbling in peoples’ brains?”

Apparently not. I’m what I have now learned to call a quiet-mind. I don’t have an internal narrator constantly expressing my thinking in language; in shorthand, I’m not a head-voice person. So much not so that when I follow the usual convention of rendering quotes from my thinking as though they were spoken to myself, I always feel somewhat as though I’m lying, fabulating to my readers. It’s not like that at all! I justify writing as though there had been a voice in my head only because the full multiordinality of my actual thought-forms won’t fit through my typing fingers.

But, apparently, for others it often is like that. Yesterday I learned that the world is full of head-voice people who report that they don’t know what they’re thinking until the narratizer says it. Judging by the reaction to the article it seems us quiet-minds are a minority, one in five or fewer. And that completely messes with my head.

What’s the point? Why do you head-voice people need a narrator to tell you what your own mind is doing? I fully realize this question could be be reflected with “Why don’t you need one, Eric?” but it is quite disturbing in either direction.

So now I’m going to report some interesting detail. There are exactly two circumstances under which I have head-voice. One is when I’m writing or consciously framing spoken communication. Then, my compositional output does indeed manifest as narratizing head-voice. The other circumstances is the kind of hypnogogic experience I reported in Sometimes I hear voices.

Outside of those two circumstances, no head-voice. Instead, my thought forms are a jumble of words, images, and things like diagrams (a commenter on Instapundit spoke of “concept maps” and yeah, a lot of it is like that). To speak or write I have to down-sample this flood of pre-verbal stuff into language, a process I am not normally aware of except as an occasional vague and uneasy sense of how much I have thrown away.

(A friend reports Richard Feynman observing that ‘You don’t describe the shape of a camshaft to yourself.” No; you visualize a camshaft, then work with that visualization in your head. Well, if you can – some people can’t. I therefore dub the pre-verbal level “camshaft thinking.”)

To be fully aware of that pre-verbal, camshaft-thinking level I have to go into a meditative or hypnogogic state. Then I can observe that underneath my normal mental life is a vast roar of constant free associations, apparently random memory retrievals, and weird spurts of logic connecting things, only some of which passes filters to present to my conscious attention.

I don’t think much or any of this roar is language. What it probably is, is the shock-front described in the predictive-processing model of how the brain works – where the constant inrush of sense-data meets the brain’s attempt to fit it to prior predictive models.

So for me there are actually three levels: (1) the roaring flood of free association, which I normally don’t observe; (2) the filtered pre-verbal stream of consciousness, mostly camshaft thinking, that is my normal experience of self, and (3) narratized head-voice when I’m writing or thinking about what to say to other people.

I certainly do not head-voice when I program. No, that’s all camshaft thinking – concept maps of data structures, chains of logic. processing that is like mathematical reasoning though not identical to it. After the fact I can sometimes describe parts of this process in language, but it doesn’t happen in language.

Learning that other people mostly hang out at (3), with a constant internal monologue…this is to me unutterably bizarre. A day later I’m still having trouble actually believing it. But I’ve been talking with wife and friends, and the evidence is overwhelming that it’s true.

Language…it’s so small. And linear. Of course camshaft thinking is intrinsically limited by the capabilities of the brain and senses, but less so. So why do most people further limit themselves by being in head-voice thinking most of the time? What’s the advantage to this? Why are quiet-minds a minority?

I think the answers to these questions might be really important.

UPDATE: My friend, Jason Azze, found the Feynman quote. It’s from “It’s As Simple As One, Two, Three…” from the second book of anecdotes, What Do You Care What Other People Think?:

When I was a kid growing up in Far Rockaway, I had a friend named Bernie Walker. We both had “labs” at home, and we would do various “experiments.” One time, we were discussing something — we must have been eleven or twelve at the time — and I said, “But thinking is nothing but talking to yourself inside.”

“Oh yeah?” Bernie said. “Do you know the crazy shape of the crankshaft in a car?”

“Yeah, what of it?”

“Good. Now, tell me: how did you describe it when you were talking to yourself?”

So I learned from Bernie that thoughts can be visual as well as verbal.

Jan 26

Missing documentation and the reproduction problem

I recently took some criticism over the fact that reposurgeon has no documentation that is an easy introduction for beginners.

After contemplating the undeniable truth of this criticism for a while, I realized that I might have something useful to say about the process and problems of documentation in general – something I didn’t already bring out in How to write narrative documentation. If you haven’t read that yet, doing so before you read the rest of this mini-essay would be a good idea.

“Why doesn’t reposurgeon have easy introductory documentation” would normally have a simple answer: because the author, like all too many programmers, hates writing documentation, has never gotten very good at it, and will evade frantically when under pressure to try. But in my case none of that description is even slightly true. Like Donald Knuth, I consider writing good documentation an integral and enjoyable part of the art of software engineering. If you don’t learn to do it well you are short-changing not just your users but yourself.

So, with all that said, “Why doesn’t reposurgeon have easy introductory documentation” actually becomes a much more interesting question. I knew there was some good reason I’d never tried to write any, but until I read Elijah Newren’s critique I never bothered to analyze for the reason. He incidentally said something very useful by mentioning gdb (the GNU symbolic debugger), and that started me thinking, and now think I understand something general.

Continue reading

Jan 24

30 Days in the Hole

Yes, it’s been a month since I posted here. To be more precise, 30 Days in the Hole – I’ve been heads-down on a project with a deadline which I just barely met. and then preoccupied with cleanup from that effort.

The project was reposurgeon’s biggest conversion yet, the 280K-commit history of the Gnu Compiler Collection. As of Jan 11 it is officially lifted from Subversion to Git. The effort required to get that done was immense, and involved one hair-raising close call.

Continue reading

Dec 23

The Great Inversion

There’s a political trend I have been privately thinking of as “the Great Inversion”. It has been visible since about the end of World War II in the U.S., Great Britain, and much of Western Europe, gradually gaining steam and going into high gear in the late 1970s.

The Great Inversion reached a kind of culmination in the British elections of 2019. That makes this a good time, and the British elections a good frame, for explaining the Great Inversion to an American audience. It’s a thing that is easier to see without the distraction of transient American political issues.

(And maybe I have an easier time seeing the pattern because I lived in Great Britain as a child. British politics is more intelligible to me than to most Americans because of that early experience.)

To understand the Great Inversion, we have to start by remembering what the Marxism of the pre-WWII Old Left was like — not ideologically, but sociologically. It was an ideology of, by, and for the working class.

Continue reading

Nov 17

Some places I won’t go

A few minutes ago I received a request by email from a conference organizer who wants me to speak at an event in a foreign country. Unfortunately, the particular country has become a place I won’t go.

Having decided that I want my policy and my reasoning to be publicly known, I reproduce here the request and my reply. I withhold the requester’s name for his protection.

Continue reading

Nov 10

Grasping Bloomberg’s nettle

Michael Bloomberg, the former Mayor of New York perhaps best known for taking fizzy drinks, and now a Democratic presidential aspirant, has just caused a bit of a kerfuffle by suggesting that minorities be disarmed to keep them alive.

I think the real problem with Bloomberg’s remark is not that it reads as shockingly racist, it’s that reading it that way leaves us unable to deal with the truth he is telling. Because he’s right; close to 95% of all gun murders are committed by minority males between 15 and 25, and most of the victims are minorities themselves. That is a fact. What should we do with it?

Continue reading

Sep 28

The dream is real

Elon Musk’s new Starship is not the tall skinny pressurized-aluminum cylinder we’re used to thinking of a real rocket, but a fat cigar-shaped thing made of stainless steel, with tail fins.

I just listened to an elaborate economic and engineering rationale for this. And I don’t believe a word of it.

It had to be that way because Elon Musk grew up on the same Golden Age science fiction magazine cover illustrations I did, and it looks exactly like those.

Has tailfins. Freaking tailfins. And lands on a pillar of fire just like God and Robert Heinlein (PBUH) intended.

The dream is real.

Sep 14

Gratitude for Beto

Beto O’Rourke is a pretty risible character even among the clown show that is the 2020 cycle’s Democratic candidate-aspirants. A faux-populist with a history of burglary and DUI, he married the heiress of a billionaire and money-bombed his way to a seat in the House of Representatives, only to fail when he ran for the Senate six years later because Texas had had enough of his bullshit. Beneath the boyish good looks on which he trades so heavily, his track record reveals him to be a rather dimwitted and ineffectual manchild with a severe case of Dunning-Kruger effect.

Beto’s Presidential aspirations are doomed, though he and the uncontacted aborigines of the Andaman Islands are possibly the only inhabitants of planet Earth who do not yet grasp this. Before flaming out of the 2020 race to a life of well-deserved obscurity, however, Beto has done the American polity one great service for which I must express my most sincere and enduring gratitude.

Continue reading

Sep 04

Be the America Hong Kong thinks you are

I think this is my favorite Internet meme ever.

Yeah, Hong Kong, we actually have a problem with Communist oppression here, too. Notably in our universities, but metastasizing through pop culture and social media censorship too. They haven’t totally captured the machinery of state yet, but they’re working on that Long March all too effectively.

And you are absolutely right when you say you need a Second-Amendment-equivalent civil rights guarantee. Our Communists hate that liberty as much as yours do – actually, noticing who is gung-ho for gun confiscation is one of the more reliable ways to unmask Communist tools.

We need to be the America you think we are, too. Some of us are still trying.

Aug 24

The Order of Defenders

Officiant: “One comes before us today who wishes to become a Sworn Brother. Let him approach.”

Officiant: “Are there two Brothers present who will affirm that the candidate is of sound mind and good character, being aware that the penalty for error in this judgment is expulsion and disgrace?”

Upon hearing affirmations, the officiant continues:

“Very well. Aspirant. Take your firearm in your dominant hand. Raise it in the posture I now demonstrate, and repeat after me. After each line, the assembled Brothers will affirm with one voice.”

My gun protects the weak.

SO MOTE IT BE!

My gun speaks for liberty.

SO MOTE IT BE!

My gun defends myself, my neighbors and my nation.

SO MOTE IT BE!

My gun guards civilization.

SO MOTE IT BE!

With this oath I become a Sworn Brother of the Order of Defenders.

SO MOTE IT BE!

I will defend, and teach others to defend themselves.

SO MOTE IT BE!

I will shoot neither in anger nor haste nor from any sort of intoxication, but in grave knowledge of the consequences.

SO MOTE IT BE!

When a Sworn Brother calls for aid in defending, I will answer.

SO MOTE IT BE!

These things I swear by all I hold sacred.

SO MOTE IT BE!

Following the initiation, all repair to a shooting range for convivial practice.


I wrote the above after thinking about Rudyard Kipling’s Ritual of the Iron Ring for newly-graduated engineers.

Rituals like this exist to express and formalize what is best in us.

The Order of Defenders does not exist. Perhaps it should.

Aug 17

How the D candidates would introduce themselves at the next debate if they were honest

Hi, I’m Joe Biden. I’m the perfect apparatchik – no principles, no convictions, and no plan. I’m senile, and I have a problem with groping children. But vote for me anyway because orange man bad.

Hi, I’m Kamala Harris. My white ancestors owned slaves, but I use the melanin I got from my Indian ancestors to pretend to be black. My own father has publicly rebuked me for the pandering lies I tell. I fellated my way into politics; put me into the White house so I can suck even more!

Hi, I’m Elizabeth Warren. Even though I’m as white as library paste, I pretended to be an American Indian to get preferment. My research on medical bankruptcies was as fraudulent as the way I gamed the racial spoils system. So you should totally trust me when I say I’m “capitalist to my bones”!

Hi, I’m Bernie Sanders. I honeymooned in the Soviet Union. I’m an unreconstructed, hammer-and-sickle-worshiping Communist.

Hi, I’m Kirsten Gillibrand. I used to be what passes for a moderate among Democrats – I even supported gun rights. Now I’ve swung hard left, and will let you just guess whether I ever had any issue convictions or it was just pandering all the way down. Tee-hee!

Hi, I’m Amy Klobuchar, and I’ve demonstrated my grasp on the leadership skills necessarily for the leader of the Free World by being notoriously abusive towards my staff.

Hi, I’m Robert Francis O’Rourke. I’m occupying the “imitate the Kennedy” lane in this race, and my credentials for it include DUI and fleeing an accident scene. The rumors that I’m a furry are false; the rumors that I’m a dimwitted child of privilege are true. But vote for me anyway, crucial white-suburban-female demographic, because I have such a winning smile!

Hi, I’m Pete Buttigieg. I was such a failure as the mayor of South Bend that my own constituents criticize me for having entered this race, but the Acela Corridor press loves me because I’m fashionably gay. And how right they are; any candidate you choose is going to bugger you up the ass eventually, but I’ll do it like an expert!

Hi, I’m Bill de Blasio. I’m as Communist as Bernie, but I hide it better. And if Pete thinks his constituents don’t want him in this race? Hold…my…beer!

Hi, I’m Cory Booker, and I’m totally not gay. OK, maybe I’m just a little gay. My city was a shithole when I was elected and I’ve done nothing to change that; I’m really just an empty suit with a plausible line of patter, especially the “I am Spartacus” part. But you should totally vote for me because I’m…what was the phrase? Oh, yeah. “Clean and articulate.”

Hi, I’m Marianne Williamson. If elected, I will redecorate the White House so it has proper feng shui. I am the sanest and least pretentious person on this stage.