Tools generate culture: a trivial example

If I were the kind of person who grumbles about feeling ancient, I’d have been doing it today.

I got reminded that younger hackers don’t know the bit structure of ASCII like their tongues know the back of their teeth. Man, we all grokked that back when I was new at this.

Nowadays not so much. I’ve actually seen younger hackers be confused about, say, how to generate a NUL from the keyboard. And I’m all, like, “How can you not know this?”

I’m bothering to post because I think I’ve figured out why this changed. The kids are OK, it’s conditions around them that have shifted.

I think it was the death of RS-232 hardware terminals in the early 1990s that did it.

Back before software terminal emulators ruled the world, there was just enough functional pressure to use all manner of odd nonprintable ASCII characters daily – and learn what control-foobar key combinations generated them – that newbie hackers tended to upload the ASCII code table into their heads pretty quickly.

This doesn’t happen reliably any more. Yes, hackers still learn individual magic keystrokes in various interfaces like vim or Emacs, and ^C as interrupt we will probably always have with us.

But, for example, who types ^L to clear a screen anymore? Or ^W to delete a word? Or, even rarer, ^S and ^Q as pause-resume. OK, I’m sure some people do – but I was actually surprised when ^L and ^W just worked in the software terminal emulator I use under i3. Because they don’t everywhere, and as a result I lost those finger habits – oh, about twenty years ago, I’d guess. And, of course, younger hackers probably never learned them.

The kids are all right. It’s the world that changed around them, and tools generate culture. Sometimes, when a tool goes away, a bit of cultural commonality – like everyone knowing ASCII down to the bits – silently evaporates with it.

I wonder if, a quarter century from now, one of today’s young hackers will find himself saying “What? You don’t know HTML tags?” And if I’m there, I’ll chuckle.

76 comments

  1. Someday someone will write a blog post like this and the examples will be ‘git rebase’ and “pull request”.

    1. >Someday someone will write a blog post like this and the examples will be ‘git rebase’ and “pull request”.

      That’s a funny line, but on reflection I disagree. Version control is too far up the stack; I remember using RCS commands 25 years ago and not feeling like they were an immutable structure of reality the way ASCII sort of seemed to be.

  2. @esr: “But, for example, who types ^L to clear a screen anymore? Or ^W to delete a word? Or, even rarer, ^S and Q as pause-resume. OK, I’m sure some people do – but I was actually surprised when they just worked in the software terminal emulator I use under i3.”

    I still type ^W to delete a word in some instances. And I still know what ^L, ^Q, and ^S do. But I haven’t had to use them for those purposes in decades.

    But I’d put “the death of RS-232 hardware terminals in the early 1990s” rather later. I was still supporting a shop full of them in the mid-2000’s – Wyse 75s configured to do Wyse 50 emulation, hardwired to a Unix host as terminals. They used software flow control with ^S/^Q because the RS-232 connector had three pins connected, RX, TX and ground, and couldn’t *do* hardware flow control. The whole world wasn’t (and still isn’t) connected to hosts via TCP-IP over a WAN.

    I also spent the odd hour back when hacking TERMCAP to try to better support terminals in use. These days, I don’t care, because terminal emulators all default to a superset of VT-100, and the stuff they talk to all groks that.

    “I wonder if, a quarter century from now, one of today’s young hackers will find himself saying “What? You don’t know HTML tags?””

    Try now, not 25 years hence. It’s like the reason next to no one writes Postscript code by hand. Easier to specify what you want with a WYSIWIG tool and let the tool generate the PS code the printer will grok. I suspect the majority of web design these days is done with WYSIWIG tools that generate the appropriate HTML tags, precisely so web developers *don’t* have to keep that stuff in their heads.
    ______
    Dennis

  3. Sure, but I feel this is also partly due to the scarcity of good solid articles like, “c structure packing” : http://www.catb.org/esr/structure-packing/
    I am neither young nor smart enough to be a hacker; I feel frustrated when I google terms such as “structure padding”, “structure alignment” I get a bunch of copy-paste, on the surface, click bait, ad entrenched pages by self appointed tech ninjas/gurus; and we need to filter the wheat from the chaff; For a person who is just beginning an interest in programming and who genuinely wants to understand things, all this sometimes can be very daunting.
    Actually I started following esr after reading above article – before that didnt know esr even existed. So the onus is really on people who know the stuff to actually contribute with indepth technical articles like above (C and algorithms would suit me fine :-)) – for, what might be trivial to hackers may not be so for lesser minds.

  4. It’s interesting you mention this because I’ve noticed a divide between developers who are GUI centric and those who rely on keyboard shortcuts. Those who value the need to improve their productivity build up their keyboard shortcut and command line skills. The command line is still ops centric but I was amazed by how a senior engineer from AWS was able to use the command line at the speed of talking when he demonstrated Chalice, a Python microframework AWS developed.

    Personally I’ve found upping my command line shortcut game has been enormously productive. I’m at the point where I feel a noticable slowdown when my muscle memory or lack of shortcut makes me use the mouse. What really turned me on to keyboard shortcuts on the command line is someone who has an efficiency hobby have a talk on the number of hours wasted on reaching for the mouse.

  5. I would guess that any decent terminal emulator will support ^S and ^Q for flow control; sensible ones (e.g., Konsole) actually coordinate that with Scroll Lock. ^L and ^W are, at least on the modern CRT-based terminals that you fogeys can’t get adjusted to, implemented by the shell (^W is in readline).

    And I’m solidly in The Next Generation for you, but I’ve had to XMODEM a firmware update to a Cisco router over a 9600bps serial line. While I took a long lunch.

  6. Heh. I used DOS for a couple of years in my early childhood. My father told me about the Alt + numpad combinations, though I don’t remember how many I actually learned (I’ve forgotten them anyway). I think I even saw a table on paper. (DOSBox supports those combinations.)

    We had an American-layout keyboard at the time, so we needed those combos in order to type the diacritics so common in Spanish.

    And yes, I got to use floppy disks; first the 5¼-inch ones and then, under Windows 3.1, the 3½-inch ones. Ah, those were the days… but I’m dating myself, am I not? :-)

    But, for example, who types ^L to clear a screen anymore? Or ^W to delete a word?

    I do, and I’m not even a programmer. And C-i instead of TAB and C-m instead of RET, among other chords. But my favorite one in Bash (and Emacs) is C-t, which is incredibly handy given how many typos are mere transpositions.

    Or, even rarer, ^S and ^Q as pause-resume.

    Didn’t know those two. Thanks. I just tried them on htop and on Moon-Buggy; they work on the former, but not on the latter.

    And I’m there, I’ll chuckle.

    Did you mean “And if I’m there…”?

    By the way, this thread provides an opportunity to give you two tips:
    · Remember our discussion about typing the eth (“ð”)? I’ve since learned that in Emacs you can do that by pressing C-x 8 ~ d. For the uppercase version (“Д), it’s – quite logically – C-x 8 ~ D.
    · You mentioned i3 in the OP. You once said that the one thing you miss in i3 is a category-based app menu; well, the morc_menu utility provides just that. I haven’t used it, though, so download and install it at your own risk. :P

    HTH.

  7. I think not hackers not memorizing bit structure can be more easily explained as a consequence of a) more compute power pushing bit-level operations lower in the stack than where most hackers operate, and b) the far lower friction cost of “just look it up” today vs. even 20 years ago.

  8. I didn’t start in Linux until the mid-2000s, and I’ve used all those fairly routinely. (If in the case of C-s/C-q only in the hair-pullingly common “search for a word/wait, I’m in less, not emacs/argh, where are all my keystrokes going?” loop.)

    It’s true, though, that they’re now Obscure Knowledge. (I’ve had to teach co-workers about them who are ten years older than me.) I think a big part is the rise of IDEs instead of living in curses apps; all those keystrokes have worked in every software terminal emulator I’ve used.

  9. @esr:
    > I think it was the death of RS-232 hardware terminals in the early 1990s that did it.

    I think another huge factor was the environment in which the young hackers and hackeroids of today learned to use computers: When we learned to use computers hardware terminals were still around, but they weren’t where we were (I *think* our local library when I was growing up used something like a VT-100 hooked up to a minicomputer of some sort for the library catalogue, but I didn’t know the questions to ask or have the interest to ask them before I was in my 20s and had left that library far behind). We didn’t learn to use computers in college on a university mini, or even a single-user lab workstation. We *grew up* on DOS and DOS kernel Windows, and every once in a while would encounter an Apple II or a Mac or a Commodore 64. We used computers primarily for games and for writing school papers in MS Word. By the time we decided to pursue a computer related education and/or career we had already been immersed for years, starting not long after we learned to speak..

    In that environment, Ctrl- and Alt- Foobar weren’t associated with characters, they were associated directly with actions. And different developers with different applications used different key combinations (often mnemonic in some way), so we developed a mental model of “environment+key combo=action”. By the time we were exposed to Unix or the detailed structure of ASCII, this mental model was solidified, and Unix terminal key combos were absorbed into it: Ctrl-C at a Unix terminal is interrupt, Ctrl-D is end of input, etc, and the fact that it sends an ASCII character is just an implementation detail that we only worry about if we’re writing code that has to send or interpret those characters.

    > immutable structure of reality the way ASCII sort of seemed to be.

    In before Jay Maynard chimes in with “don’t you mean EBCDIC?”

    1. >And different developers with different applications used different key combinations (often mnemonic in some way), so we developed a mental model of “environment+key combo=action”. By the time we were exposed to Unix or the detailed structure of ASCII, this mental model was solidified, and Unix terminal key combos were absorbed into it:

      I think this is broadly correct. I was focusing on the variability/unreliability of response to these combinations as a change that reduced the value of old finger reflexes and old knowledge, but the complementary thing you point out also happened.

  10. On that Wyse75 farm, fun fact:

    In the late 90s, I had a client expand their collection with some Link MC5’s, and we had a problem with their 3270 emulator (they were a moving company, with a multiseat SNA emulator) locking up their terminals.

    Turns out the Wyse 50 code freezes *hard* — requiring a power cycle of the terminal — when it *receives* a Ctrl-S in the main serial port.

    And the Link MC5 *emulation* of it does *too*… which weirded me out, until I found out that Link had been bought by Wyse (or vice versa) the previous year.

    I mean, bug-for-bug compatibility, yeah, but *there*?

  11. Reminder: ^s and ^q had nothing to do with the *emulator*; they were handled by the tty line discipline in the driver of the device you were talking to; this is *usually* also true of ^h, though ^w and ^l were usually one layer higher in the stack.

    If you’re seeing these work in a graphical terminal emulator connected to a local shell or telnet port, they are themselves being emulated somewhere, but I *wouldn’t* bet on it being in the terminal emulator, rather than whatever it’s connected to.

  12. I think the underlying issue is the meaning in context of hacker. Tools do indeed generate culture but tools don’t come as naturally as language for Whorf-Sapir

    Time was a power user could and would by force slide into much lower level things than today’s user. The first version of Word I used we had to patch it to enable landscape printing. Given the offset use a hex-editor and so it goes. Norton Utilities and something that shipped with the 6-Pack hardware/software combination led power users toward taking control. Some would go on to become hackers, some programmers and some stuck at power user. With enough practice came understanding and memory of things not consciously studied.

    Those gateways are long long gone. Windows desk tops with hex editors are few and far between.

    One generation before me folks were optimizing memory calls to drum rotation. My exposure was don’t do that the hardware changes are coming too fast.

    There is an opening between truly low level things and the user level for higher level hacking today.

    When the browser defaults to gray screen and setting the screen to white is the first step a few commands will be in muscle memory. HTML tags were an easy transition for somebody accustomed to careful formatting in a dos version of Word Perfect – markup is close to markup from proof reader’s marks on paper to on-screen formatting until WYSIWYG hits.

    I might as well feel ancient. I remember the lesson of Forgetfulness by Campbell (as by Don Stuart). It’s not the ancient knowledge I have that they don’t that matters it’s the new things they know that I need to learn that define the culture today.

  13. “> immutable structure of reality the way ASCII sort of seemed to be.

    In before Jay Maynard chimes in with “don’t you mean EBCDIC?””

    Naw…although I can switch back and forth easily, it does, in a way, feel like switching universes. Things are just different on a 3270.

  14. Interesting. At first I thought I was a counterexample: I’m about as young a whippersnapper as you’ll find ’round these parts, and I certainly know at least the obvious internal structure of ASCII, though I admit I couldn’t tell you what a Data Link Escape or a Group Separator is actually for. But then it occurred to me: many of my formative programming experiences had to do with retrocomputing — I was vaguely involved in the ZX Spectrum ‘scene’, which is surprisingly lively for a machine that’s been obsolete for two decades — and you can’t really get anything done in machine code without bit-twiddling, and at least knowing (say) what XOR #20 does to a letter.

    I don’t know if there’s something particularly didaktik about Sinclair machines, but I doubt it’s entirely an accident that Linus started out on a QL ;)

    1. > I admit I couldn’t tell you what a Data Link Escape or a Group Separator is actually for.

      I’m actually thinking now about writing “A Young Hacker’s Guide to ASCII”, which would explain some of this lore for people who didn’t grow up immersed in it.

      DLE, for example, had significant use on serial links as a delimiter for the start of a binary data packet. This survives in some of the odder GPS protocols like Trimble TSIP, where the general form of a packet starts with a DLE, ands with a DLE, and any DLE contained in the data of the packet itself is doubled.

      1. >I’m actually thinking now about writing “A Young Hacker’s Guide to ASCII”

        I’m no longer just thinking about it. Watch this space.

  15. Jon Brase is on to something:

    We *grew up* on DOS and DOS kernel Windows […]

    In that environment, Ctrl- and Alt- Foobar weren’t associated with characters, they were associated directly with actions.

    I certainly do reason in this way. For the user inside me, Ctrl and Alt are just modifier bits which must be faithfully passed from my fingers to the application, which must then decide which command I meant to invoke. It bothers me to no end that in Emacs, even the Gtk+ build, they get squashed into control characters and ESC sequences on input and decoded back, imperfectly, before being interpreted.

    As for this:

    I wonder if, a quarter century from now, one of today’s young hackers will find himself saying “What? You don’t know HTML tags?” And if I’m there, I’ll chuckle.

    Chuckle right now, because that’s exactly what I say to LiveJournal users who routinely post text in #333333 on #ffffff 13px Proxima Nova, in a moderated community that explicitly disallows excessive markup.

  16. This survives in some of the odder GPS protocols like Trimble TSIP, where the general form of a packet starts with a DLE, ands with a DLE, and any DLE contained in the data of the packet itself is doubled.

    Reminds me of how you escape a single quote in SQL: by doubling it. It’s a convention I wish bash adopted because I really hate the shell sequence of ‘\” to have a single quote inside a single quote-wrapped string.

  17. I think ^W probably has something to do with it closing your web browser. I actually use ^W quite a lot in Vim because it’s the shortest way to remove a word as I’m typing in insert mode. And I often close my browser as I’m typing a reply to a post. Oops.

    ^L I’ve been using a lot lately. Something seems borked with the instance of tmux that I have – it’s not properly drawing things on the screen, like irssi. Vim is mostly fine, but irssi and aspell -c are really having problems. But ^L doesn’t work in aspell -c!

  18. I keep using ^W when it does not work for the fun of it, ^L I’m surprised it works even on my current terminals (and extra happy you told about it)

  19. Even now, I actually find myself running across decimal sequences such as “119 104 97 116 32″… and thinking, hmm, that’s plaintext…

  20. > EBCDIC

    …which is a marvel of clarity and simplicity compared to Unicode.

    “You are in a maze of twisty little passages, all alike.”

  21. @esr:
    > I was focusing on the variability/unreliability of response to these combinations as a change that reduced the value of old finger reflexes and old knowledge

    I don’t think that young hacker(oid)s that actually heavily use applications that make use of those old key combos are any less likely to have them in muscle memory: Ctrl-C is reflex at a terminal for me by now, despite conflicting with “copy to clipboard” in other contexts. Ctrl-D is deeply ingrained enough that I try using it with Windows cmd and powershell, which don’t understand it. I don’t use vim as often as graphical editors, so I haven’t used it sufficiently heavily to have invested in learning its full functionality, but the parts I have learned are muscle memory. If I used emacs heavily I would probably have a good fraction of its command set committed to memory.

  22. > this is *usually* also true of ^h, though ^w and ^l were usually one layer higher in the stack.

    ^W is in the line discipline. ^L isn’t (there are a few otherwise-VT100-ish emulators that implement it for some unknown reason though).

    And of course most of these things are ’emulated’ in modern readline-using shells.

    > I’m no longer just thinking about it. Watch this space.

    One thing that might be worth mentioning is the fact that some characters (symbols associated with number keys, mainly) are arranged so that a bit-paired keyboard will (somewhat) match a manual typewriter’s layout, vs the current US keyboard layout being based on electric typewriters.

  23. It’s worth mentioning that for all that you’re equating them, unix terminal keys only have a loose relationship to the actual ASCII/ECMA-48 meanings of the control characters:

    C “interrupt”, or perhaps “interactive attention”, the character is “end of text”. (In the old old days, interrupt was ^? instead, as a poor man’s BREAK.)
    D “end of transmission” – this one fits pretty well – the exception that proves the rule.
    H Backspace… except the actual backspace key sends ^? delete.
    I Tab… which only has a loose relationship to the modern purpose of completion.
    J line feed, okay, so why does enter send ^M but the process reads it as ^J? quirk of which serial terminals became dominant on the market, I suppose. There’s half a dozen line discipline flags involved in squaring this circle.
    L “form feed” vs “clear screen”, okay. I like it better when it’s used as a section delimiter in text files. I don’t think this is actually a function that the VT100 series implemented, though maybe some lesser-known CRT serial terminals did.
    M carriage return (see J)
    N O shift out and shift in. Okay, this is valid, in the sense that they’re in anyone’s minds at all it’s for the VT100 line drawing character mode.
    Q R S T – these are “device control 1-4”. the use of DC1/DC3 for XON/XOFF is a higher layer than ASCII. Not sure why DC2 wasn’t used, or if it ever had an associated meaning, but the “reprint” meaning used today is clearly chosen for its initial “R”.
    U kill line vs negative acknowledge. You could stretch things to connect the two, but you’d have to explain why ^X (cancel) wasn’t chosen for this function instead.
    V clearly chosen as the initial of “verbatim”, nothing to do with “synchronous idle”. Any actual synchronous idling on unix serial terminals is done with NUL.
    W initial of “word”, not “end of transmission block”
    Y delayed suspend, a far cry from “end of medium”, mainly chosen because it’s next to Z. (see also emacs below)
    Z suspend… even further removed from “substitute”
    [ escape is fine
    \ sigquit. “file separator”?
    ] group separator. Only place I’ve seen this as a key is as an escape from telnet, mainly chosen because nothing else uses it for anything. I’m vaguely disappointed that ssh switched the escape to an awkward newline-tilde.
    ^? (0x7F), DEL. Along with NUL, one of the only control characters *in* ASCII whose assignment has anything to do with bit structure. All bits one makes it a good candidate for “scratching out” a character on paper tape.

    A B E F G* K N P R T X Y – Emacs casts a long shadow on bash and other similar shells’ default key bindings. These seem to be mostly chosen for initialism mnemonics (back end forward kill next previous reverse-search transpose yank), or A as the first letter of the alphabet to the beginning of the line. *Everyone knows about the bell, but bash uses it to interrupt miniprompts such as C-r or M-x, just like emacs. Used because C-c is a prefix key for some reason in emacs, and ^G as a key isn’t used by much of anything.

    Other than the ones that are so ingrained that they’re sent by physical keys such as tab/enter/backspace/escape, the main consequence of which is that you can’t make a separate keybinding for them, most of these keys’ meanings have little to do with the bit structure of ASCII, and everything to do with being a mnemonic command.

  24. @Jay Maynard: Things are just different on a 3270.

    Beginning with block mode versus async char at a time mode.

    The first computer I dealt with was an IBM mainfame host with 3270 terminals. Move the cursor around the screen, make changes, press Enter (or Submit, IIRC), and the entire changed screen got sent to the host.

    Full screen editing on a mainframe through a 3270 was a very different animal form what we are now accustomed to.
    _____
    Dennis

  25. “H Backspace… except the actual backspace key sends ^? delete.”

    This is very much variable from one serial terminal to the next. The ADM-3A and lots of others sent ^H.

  26. Like, I get what you’re saying about the bit structure – the fact that $x is $ctl and 0x40+$x and 0x60+$x are upper/lower $letter lets you generate a $ctl character by ctrl-$letter. But there are very few $ctl where that’s the actual meaning of why someone is pressing ctrl-$letter.

  27. “Full screen editing on a mainframe through a 3270 was a very different animal form what we are now accustomed to.”

    Very much so. The whole 3270 design was such that the only time the CPU got an interrupt was when the user hit one of the action keys (ENTER, PFn, PAn, a few SNA-defined keys). Everything else only affected local storage. (Effectively, in the terminal, but a regular 3270 was astoundingly st00pid even by the standards of a PC keyboard/screen. The actual backing storage was mirrored in the 3[12]7[46] controller, and all editing operations happened there. Later DFT (distributed function terminal) terminals were smarter.) The idea was that the host would send a form with defined fields of text with attributes such as highlighting, input allowed, hidden, and some others. The user could move around the input-enabled fields and change their contents, and when he pressed an action key, the modified fields, or the entire screen, could be read by the host.

    This greatly diminished the I/O load on the host, and made it possible to service hundreds of terminals with what by today’s standards is impossibly slow hardware.

    The programmers’ editors XEDIT (on VM) and ISPF/PDF (on TSO) were very much alike, but just different enough to be annoying. The similarities were largely dictated by the design forced on them by the 3270; the differences because they were developed independently by two different groups and by the time it was desired to have both be roughly the same, the incompatibilities had become too entrenched.

  28. > This is very much variable from one serial terminal to the next. The ADM-3A and lots of others sent ^H.

    The ADM-3A doesn’t, in fact, have a key labeled backspace. It has a key labeled “H”, with a left arrow on it, and ctrl-H obviously sends ^H. It has a key labeled “RUB”, which (with the shift key held down) sends ^?. Of course, it’s anybody’s guess which of these were more popular as the “stty erase” setting. Though, the default on 4.2BSD seems to have been ^? (in 4.1BSD it’s still #)

    The VT100 has “Back space” and “Delete” keys, right next to each other, which send the respective characters. Again, anybody’s guess which was more popular for stty erase. Though, in this case, we can observe that “stty dec” in 4.2BSD sets it to ^?.

    Sure, a key labeled “backspace”, when present, sent ^H. I’m just not convinced of how popular it ever was to bind “delete the character before the cursor” to that key, particularly given that it collides with help in Emacs.

  29. @Jay Maynard: “H Backspace… except the actual backspace key sends ^? delete.”

    This is very much variable from one serial terminal to the next. The ADM-3A and lots of others sent ^H.

    The Wyse 50 (and the Wyse 75 configured to emulate it) sent Ctrl H. This was annoying, as the 50 had a Delete key that also sent ^H, with no way to distinguish on the Unix host side between them. The 50 was emulating the DEC VT52.

    The Wyse 75, which effectively emulated a DEC VT100 natively, was much better about that.

    The VT52 and Wyse 50 were ASCII terminals. The VT100 and Wyse 75 used ANSI escape sequences, and were a lot easier to deal with.
    _______
    Dennis

  30. @Jay Maynard: “H Backspace… except the actual backspace key sends ^? delete.”

    This is very much variable from one serial terminal to the next. The ADM-3A and lots of others sent ^H.

    The Wyse 50 (and the Wyse 75 configured to emulate it) sent Ctrl H. This was annoying, as the 50 had a Delete key that also sent ^H, with no way to distinguish on the Unix host side between them. The 50 was emulating the DEC VT52.

    The Wyse 75, which effectively emulated a DEC VT100 natively, was much better about that.

    The VT52 and Wyse 50 were ASCII terminals. The VT100 and Wyse 75 used ANSI escape sequences, and were a lot easier to deal with.
    _______
    Dennis

  31. aargh!

    And aside from duplicate posting in trying to fix a typo, I was wrong in what I stated. The issue on the Wyse 50 wasn’t that Backspace and Delete sent the same thing – it was that Backspace and Left Arrow did.

    What I wanted was for Backspace to erase the char to the left of the cursor, but not when the Left Arrow didn’t.

    Apologies for unintended noise.
    _______
    Dennis

  32. Random832: Hrm. I do remember having to always be setting ^H as the erase key…might have been on the SOROC IQ-120 I used as my main CRT for several years. But ^H as backspace was quite common.

    And no, EMACS didn’t affect the prevalence of that, since not all CRTs were connected to Unix boxes.

  33. esr:

    >I’m actually thinking now about writing “A Young Hacker’s Guide to ASCII”
    I’m no longer just thinking about it. Watch this space.

    Eric S. Raymond,
    writing the words of a how-to that hackers will read.
    Zola comes near.
    Look at him sleeping
    next to the keyboard at night when there’s nobody there.
    Sometimes he purrs.

    All the ASCII combos,
    what purpose do they serve?
    All the ASCII combos,
    they will get on your nerves.

    1. >Eric S. Raymond,
      >writing the words of a how-to that hackers will read.

      Well done! HOWTO coming soon; I’m wrapping it up now.

  34. I expect the main influence on using ‘backspace’ for ‘delete previous character’ is actually the PC keyboard, and Unixes of that era. The Sun Type 4 keyboard, the first to give backspace a position on the keyboard where it makes sense than delete, is from 1988, half a decade after the PC and clearly heavily influenced by it compared to the Type 3.

  35. Though I might be underestimating the effect of local editing and “move the cursor, then type over it”. (The SOROC IQ-120 seems to have had the same situation as the ADM-3a – no “backspace”, yes “RUB”, and of course you’re free to use ctrl-H if you feel like it)

  36. @Jay Maynard: This greatly diminished the I/O load on the host, and made it possible to service hundreds of terminals with what by today’s standards is impossibly slow hardware.

    When I first arrived, the bank where I did this had a ITEL IBM 370 plug compatible box, supporting 500 remote 3270s via CICS over TCAM. The box had 2MB RAM and 16MB virtual storage. Downtime was frequent.

    (I saw the RFP for the datacenter, They managed to bring in a full IBM mainframe data center for just under $1 million, which was a very good price for that back then, but to do it had to go third party and plug compatible. Since it wasn’t genuine IBM kit, it also wasn’t up do date on OS and utilities.)

    They later upgraded to two genuine IBM 4341s loosely coupled through JES 2, and reliability soared.

    The programmers’ editors XEDIT (on VM) and ISPF/PDF (on TSO) were very much alike, but just different enough to be annoying. The similarities were largely dictated by the design forced on them by the 3270; the differences because they were developed independently by two different groups and by the time it was desired to have both be roughly the same, the incompatibilities had become too entrenched.

    The bank had a third party TSO/SPF replacement called ACEP when I first came on board. It was developed under and took idea from Unix. When they upgraded to real TSO/ISPF, I thought it a step backward.

    (And ACEP seems to have vanished with no trace. I’ve been unable to find information about them.)

    I almost never dealt with a VM/CMS installation, so didn’t get to use Xedit.

    I have PC emulations of both here, and find them very different indeed.
    _______
    Dennis

  37. @Random832 –

    > The ADM-3A doesn’t, in fact, have a key labeled backspace. It has a key labeled “H”, with
    > a left arrow on it, and ctrl-H obviously sends ^H.

    Fun fact – the original implementation of vi was for ADM-3A terminals; the use of “h” “j” “k” and “l” as motion keys was because that way you could use them either shifted or un-shifted to move your cursor.

    > It has a key labeled “RUB”, which (with the shift key held down) sends ^?.

    And this dates back to an even more ancient artifact – a paper-tape-equipped teletypewriter. “RUB” is an abbreviation for “rubout”, the ASCII code for which is octal 177, i.e., all bits set. (The ASCII name for this character is “DEL” or delete.) On the corresponding paper tape, that meant all the holes punched out across the width of the tape – which neatly masked whatever incorrect character you had typed. You literally pushed the ‘back up the paper tape’ button on top of the punch enough times to get to the (start of the) bad character(s), and overpunched them with rubouts, which the receiving end ignored. Then you typed the correct stuff.

    /me is old enough to have used both a papertape teletype, and an ADM-3a :P

  38. I used text terminals wired to a SCO Unix PC host every day for work until 2000. I only gave away my own pair of VT 320’s (in excellent working condition) in 2005.

    Though I am really surprised in one way… by the belated nature of the realization. Must not get out much – of course tools shape culture.

    The wider culture is replete with mix tape / VCR / pay phone / 8 track / rotary phone / etc jokes, even floppy disk references.

  39. Part of it is that ASCII is no longer the preferred way to handle text. If I see anyone handling text as ASCII in a production system I’m rejecting that diff. Whatever the efficiency gains, the i18n bugs are more expensive in the long term. There’s really no need to know the ASCII table. It’s a nice trick to pull out in an interview, but even then I usually push back on the candidate and ask for how they would handle it with full Unicode support.

  40. Once every couple of years I tell someone about the difference between \r and \n and dos2unix.

    In fact I gave a talk at work called “Old School Computer Stuff That is Still Relevant Today”, like UNIX vs. BSD vs. OSX vs. Linux, signals, exit codes, etc.

  41. @Kevin:

    > There’s really no need to know the ASCII table.

    Bollocks.

    > I usually push back on the candidate and ask for how…

    Back at you and then some — someone who really doesn’t see any value in the ASCII table would fail interviews at a lot of places I know of.

  42. @Kevin:

    I take it you mostly work with UTF-16?

    In UTF-8 environments (which is what Eric and most of his audience will be working with), handling Unicode implies handling ASCII.

  43. @DMcCunney:
    > The box had 2MB RAM and 16MB virtual storage.

    You try telling the yoong people of today that, and they won’t believe you!

    When I consider that the maximal configuration of the PDP-8 had less RAM than my current desktop (which I built a couple years ago for ~$1000) by a factor of a million, the assertions of yorkshiremen that they grew up in a rolled up newspaper in a septic tank suddenly look plausible.

    “Well it had 32k of core memory, no paging, and an ‘ardcopy terminal, but it were a computer to oos!”

  44. >> I’m actually thinking now about writing “A Young Hacker’s Guide to ASCII”
    >> ASCII is no longer the preferred way to handle text.

    Knowing about the ASCII table still has utility in understanding how sorting / indexing of database etc table text fields will turn out and why. In addition, they aid in understanding how poorly filtered imported data can trash a table in any number of unexpected ways.

  45. In UTF-8 environments (which is what Eric and most of his audience will be working with), handling Unicode implies handling ASCII.

    No, no, no. UTF-8 is not magic pixie dust; nor, for that matter, is Unicode. The right thing is to assume that textual data may be in an arbitrary encoding, and have all text objects such as strings, buffers, and files(!)[0] carry and preserve encoding info with them. Operations such as concatenation on textual objects of different encodings would be forbidden without an explicit conversion.

    The reason for this is because Unicode has been subject to political maneuvers such as Han unification that make it inherently lossy. You cannot UTF-8 all the things — or even Unicode all the things — without introducing what may be perceived in some countries as errors in the data.

    [0] It’s time to reintroduce the intrinsic text/binary distinction in filesystems, and for all text files to have an encoding attribute. The status quo is to “guess” encodings, or build assumptions about encoding into the system, both of which are error-prone.

    1. >0] It’s time to reintroduce the intrinsic text/binary distinction in filesystems, and for all text files to have an encoding attribute.

      That is a horrifyingly bad idea. “Solution” worse than disease – out-of-band data like that inevitably falls out of sync, causing hard-to-diagnose bugs. Better to live with UTF-8’s imperfections than that; somebody’s ass was going to get chapped no matter what decisions they made, so the existence of a minority of objectors is not interesting.

  46. > esr on 2017-01-26 at 06:07:00 said: I’m actually thinking now about writing “A Young Hacker’s Guide to ASCII”, which would explain some of this lore for people who didn’t grow up immersed in it.
    >
    > esr on 2017-01-26 at 08:13:11 said: I’m no longer just thinking about it. Watch this space.
    >
    >esr on 2017-01-26 at 17:08:39 said: Well done! HOWTO coming soon; I’m wrapping it up now.
    >

    I am convinced that these time points illustrate one of the personality traits of the very successful compared to the “dreamers”. The thought and action are perhaps a day apart, and completion nine hours later. It’s something that Elon Musk does, (I’ve been reading about him, lately) and I suspect most of your industry leaders do as well.

    I’d still be “thinking about” it! I *will* learn to emulate this trait!

  47. > The reason for this is because Unicode has been subject to political maneuvers such as Han unification that make it inherently lossy. You cannot UTF-8 all the things — or even Unicode all the things — without introducing what may be perceived in some countries as errors in the data.

    Han unification isn’t itself “lossy” for textual data as it preserves round-trip conversion to all of the legacy East Asian codings.

    To my understanding, the issue East Asian users have with Unicode is related to two things: Glyphs being displayed with the wrong shape when there is no font or language data (this is, broadly speaking, “the” Han unification issue), and, I’ve heard, a more minor issue that the use of mixed-coding systems (ISO 2022) was used to implicitly define the width of ambiguous-width (e.g. Greek and Cyrillic) characters (along with acting as language data for the unification problem).

    There has been an effort to solve this in Unicode itself with variant selectors, but transcoding tools and display mechanisms would need to support them.

  48. @Jeff Read:

    I did not read Jon’s comment as asserting that either UTF-8 or Unicode was magical pixie dust; he merely (correctly) asserted that (7-bit) ASCII is a proper subset of the UTF-8 encoding of Unicode.

  49. What really taught me the bit structures was learning to write assembly/machine code for 6502/6800/6809 around 1983. This was reinforced by many of the soft switches on early home computers being PEEKs, POKEs, and CALLs.

    Who still remembers that CALL -151 would put you into the monitor mode on the Apple II? Or that CALL -2458 put you in the mini-assembler? Anybody remember what POKE 50,128 and POKE 214,0 did on early Apples?

    There was a whole other set of these for Commodor 64 and TRS-80 CoCos. It made you think in terms of fiddling individual bits and bytes to get the machine into the state you wanted. And since assembly/ML makes more sense in hex coding, you learned “0” not as ASCII 48, which looks arbitrary and hard to remember, but “0” as $30, “1” as $31, etc. you suddenly get it.

    Same thing for the alphabet. “A” is 65 because it’s $41, where 1 is the first letter of the alphabet. “E” is the fifth letter of the alphabet, so it’s $45. If you always worked in BASIC, you didn’t see this as readily.

    Then when you moved to Unix and found yourself using vi for your editor, you really went to town with control codes. :-)

  50. Don Greer and esr:

    Thanks.

    Just esr:

    I now think it should have been “…for hackers to read” rather than “…that hackers will read”.

    Initially, though, it was going to be “…that no one will read”, which is closer to the original; I changed it because it sounded insulting and discouraging.

    Anyway, how ’bout this as a chapter-header “quote”:

    Here is the best part:
    you have a head start
    if you learn ASCII ’65
    by heart.

    1. >Initially, though, it was going to be “…that no one will read”, which is closer to the original; I changed it because it sounded insulting and discouraging.

      *snrk* Rob Pike just tweeted about the article. I think people are reading it.

  51. esr:

    Rob Pike just tweeted about the article. I think people are reading it.

    I didn’t say, or even imply, that they wouldn’t. I was just describing my creative process (if you can call it that): first I came up with a line that was almost like the original (probably because that’s easier for my brain), then I changed it to avoid offending you. At no point did I actually think that
    no one would read the article.

    1. >It no point did I actually think that no one would read the article.

      And I didn’t read you that way. I just thought it was funny to hear that line proposed just after I’d seen Pike’s tweet.

  52. And I didn’t read you that way. I just thought it was funny to hear that line proposed just after I’d seen Pike’s tweet.

    Oh, I get it now: you were amused by the “cosmic” irony. Thanks for clarifying. :-)

    While we’re at it, I want to offer you a belated apology for that time I drove you nuts. At the time, I decided to simply shut up; but now I think I should have said this before shutting up: “Sorry, I got carried away. We’ll talk some other time if you want. Good luck with NTPsec.”

    I’ve since strived to clean up my act, and I feel I’ve largely succeeded; but you deserve a formal apology. So I apologize.

    But enough about me; I don’t want to ruin this thread like I ruined that one.
    Here’s a video that seems (faintly) relevant to the present thread. I’d like to know what you lads and lasses think of it. :-)

  53. Cathy,

    I remember CALL -151, but I never knew the direct entry to the mini-assembler, I always used F666G from the monitor. I had a ][+, so this was after loading the Programmers Aid #1 image (I don’t remember if it was in INTBASIC on the System Master or if I scrounged if from somewhere else) with the right sequence of C081 and C083 and BLOAD… and the wrong sequence would lock the machine. Fun times.

    Then there was D5 AA AD and D5 AA 96, which relates to my handle.

  54. > but I *wouldn’t* bet on it being in the terminal emulator, rather than whatever it’s connected to.

    Yeah, I didn’t expect Terminal (on OSX) to handle ^L and ^W outside the shell. Turns out it does, which I discovered by running cat. You learn something new every day.

  55. > Yeah, I didn’t expect Terminal (on OSX) to handle ^L and ^W outside the shell. Turns out it does, which I discovered by running cat. You learn something new every day.

    I always think it’s weird when terminal emulators interpret ^L as clear screen, because the VT100 didn’t and neither does xterm. I didn’t know that the OSX terminal did. When I said “some” terminal emulators do, it was actually putty that I had in mind.

    I’m not sure why exactly anyone is surprised when ^W works since it’s another line discipline thing, though it’s one that didn’t exist in System V and isn’t defined in POSIX for that reason. The IEXTEN flag is required for it to work, for this reason (Anyone here who has had it not work on a system they use might consider looking into this, you might be surprised)

  56. > HTML tags were an easy transition for somebody accustomed to careful formatting in a dos version of Word Perfect

    I’m glad to see I’m not the only person to think this. The first time I saw HTML tags, I immediately thought of WP Reveal Codes’ “[BOLD]this section is bolded[bold]” approach, and the transition to the use of / rather than case to differentiate begin/end was nothing at all.

  57. Random832:

    I always think it’s weird when terminal emulators interpret ^L as clear screen, because the VT100 didn’t and neither does xterm.

    I just tried it in xterm and it did work. (If it matters, I’m using Ubuntu MATE 16.10.)

  58. @Jorge Dujan – the terminal itself, or bash? Yes, bash clears the screen when you type ctrl-L, no matter what terminal you’re in. The thing that some terminals do and others don’t is clearing when the form feed character echoed to the terminal. The question is, does the screen clear if you run the command “printf ‘\f'” , or if you type ^L then enter in cat?

  59. Random832:

    …the terminal itself, or bash?

    Oops. You’re right: I just tried three alternative shells (rc, zsh, and dash) inside xterm, and one of them (dash) didn’t clear the screen when I pressed ^L. Thanks for the correction.

    By the way, I just repeated the experiment with the tty that’s accessed by pressing Ctrl+Alt+F1. The result is the same: ^L works under bash, rc, or zsh, but not under dash.

    The question is, does the screen clear if you run the command “printf ‘\f’”…

    In all four shells I tried (in both xterm and the tty), that command simply printed a blank line.

    …or if you type ^L then enter in cat?

    How does that work? In my experiments, I typed ^L and then entered cat, but that didn't affect the result; bash, rc, and zsh cleared the screen as soon as I pressed ^L, and dash refused to clear the screen with or without "cat" (it always inserted a literal "^L"!).

    Am I missing some necessary step? I know how to use cat for seeing the contents of text files, but nothing more.

  60. My brain farted: only the word “cat” was meant to be in code format; but in the closing tag, where I was supposed to write “/code”, I wrote “/cat” instead.

    Meow.

  61. >In all four shells I tried (in both xterm and the tty), that command simply printed a blank line.

    In PuTTY, printf '\f' clears my window, apparently regardless of shell, which makes sense; printf '\f' >testfile produces a file with ^L in it.

  62. @Jorge I meant “type ^L and then enter in cat” as “go into cat, type ^L, then type enter”. Basically to get the same result as printf ‘\f’ would, but involving cat as EMF described.

    Using cat is a quick and dirty way to see which features are supported by the terminal-plus-line-discipline and what are layered on top of it by shells like bash/tcsh/zsh.

  63. And just to clarify it’s cat by itself with no files or redirections, which copies input from the terminal to the terminal. You press ^D at the beginning of a line (or twice in a row elsewhere) to end input when it is being read from the terminal.

Leave a Reply to DMcCunney Cancel reply

Your email address will not be published. Required fields are marked *