The craft of programming is not a thing easily taught. It’s not so much that the low level details like language syntaxes are difficult to convey, it’s more that (as I’ve written before) “the way of the hacker is a posture of mind”.
The posture of mind is more essential than the details. I only know one way to teach that, and it looks like this…
19:51:23 esr | You know, at some point you should build Open Adventure and play it. For that geek heritage experience, like admiring Classical temple friezes.
ianbruene | note to self: play advent
19:53:56 esr | I actually think this should count as (a very minor) part of your training, though I’m not sure I can fully articulate why. Mumble mumble something mimesis and mindset. It was written by two guys with the mindset of great hackers. If playing that game gets you inside their heads even a little bit, you’ll have gained value…
19:55:08 ianbruene | I already had a flag set to fix the non-human-readable save problem someday, if no one else got to it first. Kind of hard to do that without playing at least *some* of the game.
19:55:15 esr | :-)
19:55:29 ianbruene | (non human readable saves irk me)
19:55:35 esr | An excellently chosen exercise, apprentice!
19:57:57 esr | Since you’ve brought it up, let’s think through some of the tradeoffs.
19:58:16 ianbruene |
19:59:25 esr | You are right, in general, to dislike non-eyeball-friendly save formats, especially for small-scale applications.
19:59:54 esr | But: There are two reasons to like the way it’s done now.
20:00:27 esr | Care to guess at them? You can ask me questions about the code context to develop your answer.
20:00:12 ianbruene | anti-cheat being the obvious one
20:00:48 * | ianbruene pulls up advent page
20:00:51 esr | Right – it’s a low barrier, but not a completely useless one.
20:02:02 esr | That is, it hits a nice sweet spot between “too easily editable” and “putting in extra work for something that in principle can always be circumvented no matter what you do”.
20:02:19 esr | Speed bump.
20:02:31 esr | That’s reason #1. What’s #2?
20:02:49 esr | (Don’t overthink it.)
20:03:16 ianbruene | ok, just seeing the first couple functions the comments mention twiddling a few things to prevent savescumming. falls under anti-cheat I assume?
20:03:24 esr | Yes.
20:03:44 esr | No, there’s something else much more basic.
20:04:29 ianbruene | um…. well restore() is just a memcopy……. that seems almost too simple
20:04:41 esr | You’re warm.
20:04:43 ianbruene | also incredibly fragile
20:05:20 esr | I guess you’re close enough.
20:05:36 esr | Reason #2: IT’S *SIMPLE*.
20:06:17 esr | Not a consideration to be despised. Smart programmers do as little work as possible on unimportant things, so they can save brainpower and effort for important things.
20:06:43 ianbruene | am I right in thinking that this is considered ok because 1. very early: the field hadn’t really developed yet
20:07:07 ianbruene | and 2. no need for interop, or devastating if data lost
20:07:21 ianbruene | unlike a certain *cough* MS Word *cough*
20:07:32 esr | Pretty much correct.
20:07:45 esr | However, note 2 subtle points:
20:08:09 ianbruene | ok, *readjusts utility function for memdump fileformat tradeoff*
20:08:20 esr | 1. Format is rather less fragile than you think (I’ll explain that).
20:08:33 ianbruene | I only knew of the ludicrous example of the MS formats previously
20:09:27 esr | 2. The FORTRAN save/restore code was really nasty and complicated. It doesn’t get as simple as it now is unless you have fread/fwrite and a language with structs.
20:10:07 esr | Now, why this isn’t as bad a choice as you think:
20:10:31 ianbruene | (pre-guess: everything is pre-swizzled)
20:11:56 esr | No. It’s because any processor this is likely to run on uses the same set of struct-member aligment rules, what’s called “self-aligned” members. So padding won’t bite you, just endianness and word size.
20:12:54 ianbruene | *blink* oh…. another win from the intervening steamroller of standardization
20:13:17 esr | Precisely, another steamroller win.
(Editor’s note: ianbruene is aware that at the time the original ADVENT was written, greater diversity in processor architectures meant that the structure member alignment rules were more variable and more difficult to predict. So you took a harder portability hit from using a structure dump as a save format then.)
20:12:19 esr | Wait, I want to give you a pointer
20:12:58 esr | Read this, if you haven’t: http://www.catb.org/esr/structure-packing/
20:13:26 ianbruene | read it before, been quite a while (and didn’t have anything to put it into practice)
20:14:22 esr | So, reread with this save format in mind. Go through the reasoning to satisfy yourself about what I just claimed.
20:14:32 esr | It won’t take long.
20:16:06 esr | Now, this does *not* mean a memory dump would be a good format for anything much more complex than this game state. We’re sort of just below the max-complexity threshold here.
20:16:44 esr | And we do get screwed by endianness and word-size differences.
20:17:39 esr | But…let’s get real, how often are these save files going to move between machines? This is not data with a long service lifetime!
20:19:49 esr | OK. Continuation of exercise:
20:20:28 esr | What’s the simplest way you can think of to design an eyeballable save format?
20:21:14 ianbruene | *thinks* (given that I know nothing of the internal structure of advent)
20:22:45 esr | You don’t need to. Look at the structure definition.
20:23:47 * | ianbruene looking up struct
20:28:34 ianbruene | well *one* simple way of doing it would be to do a (I forget what the format is called) var=value\n format, with the save function being a giant printf of doom and the load function being a giant switch of doom.
20:28:44 ianbruene | I don’t think that is *the* simple one though
20:30:06 esr | That sort of thing is generally called a “keyword/value” format. It is the most obvious choice here. Can you think of a simpler one?
20:30:42 esr | (I’m not sure I can.)
20:31:16 ianbruene | ok, we know the struct “shape”, could arrange for all of type X, all of type Y, etc. to be in contiguous spans. Sequence either hardcoded or using a build time generator for the var names. Hmmmmmmmm….. while it has a certain elegance it seems brittle and complex
20:31:25 esr | Yes. It would be that.
20:31:38 esr | Pretty classic example of “too clever by half”, there.
20:33:35 ianbruene | ok, ignore assuming a shape, it would be *possible* for a code generator to simply look at the struct and create a pair of load/save functions from it, using either the internal names, or special comments in the definition
20:33:59 ianbruene | I don’t think the format itself can get any simpler than key=value though
20:34:12 ianbruene | there isn’t much complex structure in the save.
20:34:21 esr | I think you’re right.
20:34:33 ianbruene | it isn’t a bunch of logical blocks of different rooms and characters
20:35:09 ianbruene | if there were there would be useful tradeoffs in how you grouped things
20:35:45 esr | Good! That was a sound insight, there.
20:34:59 esr | Now, you’ve correctly described a way to implement dump as a giant printf.
20:35:27 esr | Do you as yet know enough C to sketch restore?
20:35:39 ianbruene | *thinks*
20:36:24 ianbruene | ok, the template I’m thinking is similar to the packet handling code for mode 6 (python side). but it is more complicated due to C
20:37:29 ianbruene | read until get a token, slice off the token, feed the token into the Switch of Doom
20:37:47 ianbruene | the SoD sets any vars it gets tokens for
20:38:07 ianbruene | if the file is well formed you get all the data you need
20:39:49 esr | Alas, you’ll find actually doing restore in C is a PITA for a couple reasons. One is that you can’t switch on a string’s content, only its start address – C switch only accepts scalars.
20:40:31 ianbruene | grrrr, so you have a big ugly set of str compares in if statements
20:40:37 esr | Indeed, you’re going to write a big fscking if () with a whole bunch of strcmp() guards.
(Editor’s note: There’s another way to do it, driven from iterating through a table of struct initializers, that would be slightly more elegant but no simpler.)
20:40:38 ianbruene | I forgot about that
20:41:29 ianbruene | this is something where your code style becomes *very* important or it will be an ugly, incomprehensible mess
20:41:45 esr | Yes. Now you begin to see why I went to stupid fread/fwrite and stayed there.
20:42:31 ianbruene | and the obvious way to do it in something like python (magic introspection to class elements or dict keys) doesn’t work here
20:43:03 esr | Right. Replacing this binary dump with something clean and textual is not a terrible idea, but really only justifies itself as a finger exercise for a trainee, like playing scales to learn an instrument.
20:43:17 ianbruene | I see
20:43:39 ianbruene | hence why you mentioned in the blog that it was very low priority
20:43:46 esr | Right. The absence of introspection is the other lack in C that makes it a PITA.
20:44:17 esr | And you’ve extracted most of the value of the finger exercise by thinking through the design issues.
20:45:02 ianbruene | when you get to do it introspection is a gigantic win, makes it difficult to remember how bad it is when you don’t get it
20:45:08 esr | Yes.
20:46:05 esr | Those of us who started in LISP learned this early. Took forty years for the rest of the world to catch up, and they’re only getting there now.
20:46:40 ianbruene | I have done only the barest toying with lisp, barely even hello world level
20:47:00 ianbruene | but even that (coupled with some reading of a lisp book) changed the way I thought
20:47:35 ianbruene | plenty of times I’ve hit a snag of “this would be *so much easier* if I could do a lisp macro in python”
20:47:52 esr | Indeed.
20:48:33 ianbruene | incidentally, has GvR used lisp at all? the impression I’ve heard is that he doesn’t like the lispy features?
20:49:11 esr | He doesn’t. Back in the late ’90s I practically had to arm-wrestle him into not killing lambdas.
20:49:45 ianbruene | [insert rant against anyone who thinks lambdas are useless]
20:51:37 esr | I think I might edit this dialog into a blog post. Start it with the Heinlein quote about the ideal university: a log with a teacher on one end and a student on the other.
20:51:51 * | ianbruene grins
The foregoing was transcribed from IRC and lightly edited to fix typos, fill out sentence fragments, and complete 80%-articulated ideas we mutually glarked from context. A few exchanges have been slightly reordered.