May 13

A Taxonomy of Cognitive Stress

I have been thinking about UI design lately. With some help from my
friend Rob Landley, I’ve come up with a classification schema for the
levels at which users are willing to invest effort to build
competence.

The base assumption is that for any given user there is a maximum
cognitive load any given user is willing to accept to use an
interface. I think that there are levels, analogous to Piagetian
developmental thresholds and possibly related to them, in the
trajectory of learning to use software interfaces.

Level 0: I’ll only push one button.

Level 1: I’ll push a sequence of buttons, as long as they’re all visible
and I don’t have to remember anything between presses. These people
can do checklists.

Level 2: I’m willing to push as sequence of buttons in which later ones may
not be visible until earlier ones have been pressed. These people
will follow pull-down menus; it’s OK for the display to change as long
as they can memorize the steps.

Level 3: I’m willing to use folders if they never change while I’m not looking.
There can be hidden unchanging state, but nothing must ever
happen out of sight. These people can handle an incremental replace
with confirmation. They can use macros, but have no capability to
cope with surprises other than by yelling for help.

Level 4: I’m willing to use metaphors to describe magic actions. A folder
can be described by “These are all my local machines” or “these
are all my print jobs” and is allowed to change out of sight in an
unsurprising way. These people can handle global replace, but must
examine the result to maintain confidence. These people will begin
customizing their environment.

Level 5: I’m willing to use categories (generalize about nouns). I’m
willing
to recognize that all .doc files are alike, or all .jpg files are
alike, and I have confidence there are sets of actions I can apply
to a file I have never seen that will work because I know its type.
(Late in this level knowledge begins to become articulate; these
people are willing to give simple instructions over the phone or
by email.)

Level 6: I’m willing to unpack metaphors into procedural steps. People at
this level begin to be able to cope with surprises when the
metaphor breaks, because they have a representation of process.
People at this level are ready to cope with the fact that HTML
documents are made up of tags, and more generally with
simple document markup.

Level 7: I’m willing to move between different representations of
a document or piece of data. People at this level know that
any one view of the data is not the same as the data, and lossless
transformations no longer scare them. Multiple representations
become more useful than confusing. At this level the idea of
structural rather than presentation markup begins to make sense.

Level 8: I’m willing to package simple procedures I already understand.
These people are willing to record a sequence of actions which
they understand into a macro, as long as no decisions (conditionals)
are involved. They begin to get comfortable with report generators.
At advanced level 8 they may start to be willing to deal with
simple SQL.

Level 9: I am willing to package procedures that make decisions, as long
as I already understand them. At his level, people begin to cope
with conditionals and loops, and also to deal with the idea of
programming languages.

Level 10: I am willing to problem-solve at the procedural level, writing
programs for tasks I don’t completely understand before
developing them.

I’m thinking this scale might be useful in classifying interfaces and
developing guidelines for not exceeding the pain threshold of an
audience if we have some model of what their notion of acceptable
cognitive load is.

(This is a spinoff from my book-in-progress, “The Art of Unix
Programming”, but I don’t plan to put it in the book.)

Comments, reactions, and refinements welcome.

Blogspot comments

May 05

The Delusion of Expertise

I learned something this weekend about the high cost of the subtle delusion that creative technical problem-solving is the preserve of a priesthood of experts, using powers and perceptions beyond the ken of ordinary human beings.

Terry Pratchett is the author of the Discworld series of satirical fantasies. He is — and I don’t say this lightly, or without having given the matter thought and study — quite probably the most consistently excellent writer of intelligent humor in the last century in English. One has to go back as far as P.G. Wodehouse or Mark Twain to find an obvious equal in consistent quality, volume, and sly wisdom.

I’ve been a fan of Terry’s since before his first Discworld novel; I’m one of the few people who remembers Strata, his 1981 first experiment with the disc-world concept. The man has been something like a long-term acquaintance of mine for ten years — one of those people you’d like to call a friend, and who you think would like to call you a friend, if the two of you ever arranged enough concentrated hang time to get that close. But we’re both damn busy people, and live five thousand miles apart.

This weekend, Terry and I were both guests of honor at a hybrid SF convention and Linux conference called Penguicon held in Warren, Michigan. We finally got our hang time. Among other things, I taught Terry how to shoot pistols. He loves shooter games, but as a British resident his opportunities to play with real firearms are strictly limited. (I can report that Terry handled my .45 semi with remarkable competence and steadiness for a first-timer. I can also report that this surprised me not at all.)

During Terry’s Guest-of-Honor speech, he revealed his past as (he thought) a failed hacker. It turns out that back in the 1970s Terry used to wire up elaborate computerized gadgets from Timex Sinclair computers. One of his projects used a primitive memory chip that had light-sensitive gates to build a sort of perceptron that could actually see the difference between a circle and a cross. His magnum opus was a weather station that would log readings of temperature and barometric pressure overnight and deliver weather reports through a voice synthesizer.

But the most astonishing part of the speech was the followup in which Terry told us that despite his keen interest and elaborate homebrewing, he didn’t become a programmer or a hardware tech because he thought techies had to know mathematics, which he thought he had no talent for. He then revealed that he thought of his projects as a sort of bad imitation of programming, because his hardware and software designs were total lash-ups and he never really knew what he was doing.

I couldn’t stand it. “And you think it was any different for us?” I called out. The audience laughed and Terry passed off the remark with a quip. But I was just boggled. Because I know that almost all really bright techies start out that way, as compulsive tinkerers who blundered around learning by experience before they acquired systematic knowledge. “Oh ye gods and little fishes”, I thought to myself, “Terry is a hacker!”

Yes, I thought ‘is’ — even if Terry hasn’t actually tinkered any computer software or hardware in a quarter-century. Being a hacker is expressed through skills and projects, but it’s really a kind of attitude or mental stance that, once acquired, is never really lost. It’s a kind of intense, omnivorous playfulness that tends to color everything a person does.

So it burst upon me that Terry Pratchett has the hacker nature. Which, actually, explains something that has mildly puzzled me for years. Terry has a huge following in the hacker community — knowing his books is something close to basic cultural literacy for Internet geeks. One is actually hard-put to think of any other writer for whom this is as true. The question this has aways raised for me is: why Terry, rather than some hard-SF writer whose work explicitly celebrates the technologies we play with?

The answer now seems clear. Terry’s hackerness has leaked into his writing somehow, modulating the quality of the humor. Behind the drollery, I and my peers worldwide have accurately scented a mind like our own.

I said some of this the following day, when I ran into Terry surrounded by about fifty eager fans in a hallway. The nature of the conference was such that about three-quarters of them were hackers, many faces I recognized. I brought up the topic again, emphasizing that the sort of playful improvisation he’d been describing was very normal for us, and that I thought it was kind of sad he’d been blocked by the belief that hackers need to know mathematics, because about all we ever use is some pieces of set theory, graph theory, combinatorics, and Boolean algebra. No calculus at all.

Terry then admitted that he had at one point independently re-invented Boolean algebra. I didn’t find this surprising — I did that myself when I was about fifteen; I didn’t mention this, though, because the moment was about Terry’s mind and not mine. I think reinventing Boolean algebra is probably something a lot of bright proto-hackers do.

“Terry,” I said, fully conscious of the peculiar authority I wield on this point as the custodian of the Jargon File, the how-to on How To Become A Hacker and several other related documents, “you are a hacker!

The crowd agreed enthusiastically. Somebody handed Terry one of the “Geek” badge ribbons the convention had made for attendees who wanted to identify themselves as coming from the Linux/programming side. Much laughter ensued when it was discovered that the stickum on the ribbon had lost its virtue, and a nearby hacker had to ceremonially affix the thing to Terry’s badge holder with a piece of duct tape.

Terry actually choked up a little while this was going on, and I don’t think there was anyone there who didn’t understand why. To the kind of teenager and young man he must have been — bright, curious, creative, proud of his own ability — it must have been very painful to conclude that he would never cut it as the techie he so obviously wanted to be. He ended up doing public-relations work for the British nuclear-power industry instead.

The whole sequence of events left me feeling delighted that I and my friends could deliver the affirmation Terry had deserved so long ago. But also — and here we come to the real point of this essay — I felt very angry at the system that had fed the young Terry such a huge load of cobblers about the nature of what programmers and hardware designers do.

I’m not referring to the obvious garbage about needing a brain-bending amount of mathematics. No; they fed Terry something much subtler and more crippling, a belief that real techies actually know what they’re doing. The delusion of expertise.

The truth is that programmers only know what they’re doing when the job is not very interesting. When you’re breaking new ground in any technical field, exploration and improvisation is the nature of the game. Your designs are going to be lash-ups because you don’t yet know any better and neither does anyone else. Systematization comes later, with the second system, during the re-write and the re-think. Einstein had it right; imagination is more valuable than knowledge, and people like Terry with a demonstrated ability to creatively wing it make far better hackers than analytically smart but unimaginative people who can only follow procedures.

The thought that Terry may have spent thirty years of working days grinding out press releases for the Central Electricity Generating Board because he didn’t know this, rather than following his dreams into astronomy or programming or hardware design, bothers the crap out of me. If Terry was bright enough to invent Boolean algebra, he was bright enough to cut it in any of these fields. The educational system failed him by putting artificial requirements in his way and making him believe they were natural ones. It failed him even more fundamentally by teaching him a falsehood about the nature of expertise.

In doing this, it failed all of us. How many bright kids with first-class minds, I wonder, end up under-employed because of crap like this? How much creative potential are we losing?

OK, some might answer, so we got the Discworld fantasies instead…that ain’t exactly chopped liver. The thing is, I’m not sure that was actually a trade-off. I’m enough of a writer myself to believe that you can’t block a writing talent like Terry’s merely by dropping him into a more demanding day job. It will come out.

On the other hand, one thing I am sure of is that you don’t need intelligence or talents like Terry’s just to do PR. One way or another, this man was going to do something with more lasting effects than soothing British farmers about radiation leaks. Inventing one of the funniest alternate worlds of the last hundred years during your free time is nice, and I devoutly hope he will get to keep doing it for decades to come — but in a society that valued and nurtured genius properly, I think Terry might have helped re-imagine the real world just as radically during his day job.

But he didn’t. Tot it up to the cost of taking creativity too seriously, of undervaluing improvisation and play and imagination. And wonder how much else that error has cost us.

Blogspot comments