Missing documentation and the reproduction problem

I recently took some criticism over the fact that reposurgeon has no documentation that is an easy introduction for beginners.

After contemplating the undeniable truth of this criticism for a while, I realized that I might have something useful to say about the process and problems of documentation in general – something I didn’t already bring out in How to write narrative documentation. If you haven’t read that yet, doing so before you read the rest of this mini-essay would be a good idea.

“Why doesn’t reposurgeon have easy introductory documentation” would normally have a simple answer: because the author, like all too many programmers, hates writing documentation, has never gotten very good at it, and will evade frantically when under pressure to try. But in my case none of that description is even slightly true. Like Donald Knuth, I consider writing good documentation an integral and enjoyable part of the art of software engineering. If you don’t learn to do it well you are short-changing not just your users but yourself.

So, with all that said, “Why doesn’t reposurgeon have easy introductory documentation” actually becomes a much more interesting question. I knew there was some good reason I’d never tried to write any, but until I read Elijah Newren’s critique I never bothered to analyze for the reason. He incidentally said something very useful by mentioning gdb (the GNU symbolic debugger), and that started me thinking, and now think I understand something general.

If you go looking for gdb intro documentation, you’ll find it’s also pretty terrible. Examples of a few basic commands is all they can do; you never get an entire worked example of using gdb to identify and fix a failure point. And why is this?

The gdb maintainers probably aren’t very self-aware about this, but I think at bottom it’s because the attempt would be futile. Yes, you could include a session capture of someone diagnosing and debugging a simple problem with gdb, but the reader couldn’t reliably reproduce it. How would you the user go about generating a binary on which the replicating the same commands produced the same results?

For an extremely opposite example, consider the documentation for an image editor such as GIMP. It can have excellent documentation precisely because including worked examples that the reader can easily understand and reproduce is almost trivial to arrange.

What’s my implicit premise here? This: High-quality introductory software documentation depends on worked examples that are understandable and reproducible. If your software’s problem domain features serious technical barriers to mounting and stuffing a gallery of reproducible examples, you have a problem that even great willingness and excellent writing skills can’t fix.

Of course my punchline is that reposurgeon has this problem, and arguably an even worse example of it than gdb’s. How would you make a worked example of a repository conversion that is both nontrivial and reproducible? What would that even look like?

In the gdb documentation, you could in theory write a buggy variant of “Hello, World!” with a crash due to null pointer dereference and walk the reader through locating it with gdb. It would be a ritual gesture in the right direction, but essentially useless because the example is too trivial. It would read as a pointless tease.

Similarly, the reposurgeon documentation could include a worked conversion example on a tiny synthetic repository and be no better off than before. In both problem domains reproducibility implies triviality!

Having identified the deep problem, I’d love to be able to say something revelatory and upbeat about how to solve it.

The obvious inversion would be something like this: to improve the quality of your introductory documentation, design your software so that user reproduction of instructive examples is as easy as its problem domain allows.

I understand that this only pushes the boundaries of the problem. It doesn’t tell you what to do when you’re in a problem domain as intrinsically hostile to reproduction of examples as gdb and reposurgeon are.

Unfortunately, at this point I am out of answers. Perhaps the regulars on my blog will come up with some interesting angle.

42 comments

  1. This sounds like a situation that greatly benefits from (interactive) teaching. Find a non-trivial problem and convince an experienced user to help you through the process the first time.

    This won’t scale. But the next best thing may be good enough: screencasts, with (perhaps edited) narration of what features of the tool you’re using, as you use it to solve some real problem. I did not understand the appeal of i3 until I saw a video of it in action; particularly cool was a onscreen, real-time display of what keys were being pressed to control opening/moving containers and focus.

  2. It might be worthwhile to build a series of brief tutorials around the cases in reposurgeon’s test suite, especially if they’re fairly minimal and the *.tst files contain the commands a user would actually want to issue when dealing with the associated repo.

    E.g. “Extract foo.svn from examples.tar.gz. It suffers from pathology X which is often encountered if a user has performed action Y at some point in the repo’s history. Here’s how to test for X, and here’s the recipe to fix it using reposurgeon.”

    That’s a step up from just handing the user the individual commands and expecting him to understand how to put them together, but modular enough he can recognise and skip the lessons that won’t be useful to him (e.g. he knows nobody’s done Y to his repo).

  3. It’s kind of a second-order problem, really. The usage of the program is incidental to the discoverability of the problem space itself. Every task is different, and so tools designed to aid with navigation through those spaces require teaching of methodologies outside the scope of those tools.

    An analog would be sitting in an airplane, learning what all the controls and switches and knobs do, but never learning how to actually fly or how all those pieces work together to solve in-flight problems. Or being given use of an MRI machine without having any medical knowledge.

    The point being, it seems like any documentation for such as reposurgeon/gdb should necessarily begin with methodology for identifying the problem being wrestled with, determining if the tool is actually the right one for the job, and showing how/if the tool helps map the extents of potential problem spaces, but not giving cookbook style example solutions to pre-defined problems since those require too many artificial and thus less useful cases.

    Once the user actually understands what they’re dealing with, it should be much easier to determine how to get to a desired point B, and the basic docs stuff ought to suffice.

    I might just be throwing an exception up the stack, though.

  4. Eric, I think I understand your problem description in the abstract. But it would help me to have a more concrete picture what a failure to reproduce would look like to the user, and what the chief causes of it would be.

    Suppose your reposurgeon documentation included a real repository you once converted. (Not gcc obviously, but one that’s just complex enough to be instructive.) Suppose you also supplied narrative documentation to tell the story of how you went about that conversion. As your readers follow along, typing what you typed, what problem would they run into, and why? Is it that the reposurgeon output is so specific to your example project that it would tell them nothing useful about the project they want to tackle in their own line of work eventually? Or is it that changes between your computer’s environment and their computers’ environment would cause them to see very different output?

    If it’s the latter, that would suggest to me that the solution lies in finding a clever way to standardize environments. (For example, put the example repo on a web server, tailor your documentation to its environment, and invite users to type along in some shell running inside their browsers.)

    But I don’t want to assume. Hence my question.

    1. >[I]s it that changes between your computer’s environment and their computers’ environment would cause them to see very different output?

      That’s not it.

      >Is it that the reposurgeon output is so specific to your example project that it would tell them nothing useful about the project they want to tackle in their own line of work eventually?

      That’s it.

      Every repository is its own self-contained universe of glitches and pain. There’s not much in any lift script that can be generalized, because if there were that regularity would have been pushed downward into the DSL as a reusable primitive.

      As an illustration, here’s the final lift script for GCC:

      # The GCC conversion recipe
      #

      set echo

      authors read <gcc.map

      # Deal with the one ambiguous username.
      (1.. & /master/b) | /premerge-fsf-branch/b assign gcc2
      authors read <<EOF
      dje = Doug Evans
      EOF
      ~ authors read <<EOF
      dje = David Edelsohn America/New_York
      dje-logonly1 = David Edelsohn America/New_York
      EOF

      # Process the ChangeLog files, including ChangeLog., for
      # better commit attribution. However, don’t do this for merges as
      # those may contain many different ChangeLog entries for changes being
      # merged. It would probably be better to do this for branch formation
      # as well, but forming a selection set for this is not as simple.
      ~=M changelogs /^ChangeLog(\..*)?$/

      # If anything after this fails, it’s better to get a diagnostic
      # message than have the script abort
      set relax

      # This commit was made by Julian Brown with his own email, but Paul’s name.
      # (Reviewing the email logs suggest this is the correct attribution.)
      attribution =A set paul@codesourcery.com

      # Commit made by Dale Johannesen with his own email, but Geoff Keating’s name.
      attribution =A set geoffk@apple.com

      # Commit made by Aldy Hernandez with his own email, but Richard
      # Henderson’s name.
      attribution =A set rth@redhat.com

      # Correct parents of branch-creation commits where cvs2svn messed up
      # identifying the right parent commit.

      # /branches/GC_5_0_ALPHA_1
      | reparent –use-order
      # /branches/apple-200511-release-branch
      | reparent –use-order
      # /branches/apple-gcc_os_35-branch
      | reparent –use-order
      # /branches/apple-tiger-release-branch
      | reparent –use-order
      # /branches/bje-unsw-branch
      | reparent –use-order
      # /branches/bounded-pointers-branch
      | reparent –use-order
      # /branches/cfg-branch
      | reparent –use-order
      # /branches/csl-3_3_1-branch
      | reparent –use-order
      # /branches/csl-3_4-linux-branch
      | reparent –use-order
      # /branches/csl-3_4_0-hp-branch
      | reparent –use-order
      # /branches/csl-3_4_3-linux-branch
      | reparent –use-order
      # /branches/csl-arm-2004-q3-branch
      | reparent –use-order
      # /branches/csl-gxxpro-3_4-branch
      | reparent –use-order
      # /branches/csl-sol210-3_4-branch
      | reparent –use-order
      # /branches/cygming331
      | reparent –use-order
      # /branches/cygming332
      | reparent –use-order
      # /branches/cygwin-mingw-gcc-3_1-branch
      | reparent –use-order
      # /branches/cygwin-mingw-gcc-3_2-branch
      | reparent –use-order
      # /branches/cygwin-mingw-gcc-3_2_1-branch
      | reparent –use-order
      # /branches/cygwin-mingw-v2-branch
      | reparent –use-order
      # /branches/egcs_1_00_branch
      | reparent –use-order
      # /branches/gcc-2_95_2_1-branch
      | reparent –use-order
      # /branches/gcc-3_2-branch
      | reparent –use-order
      # /branches/gcc-3_3-e500-branch
      | reparent –use-order
      # /branches/gcc-3_3-rhl-branch
      | reparent –use-order
      # /branches/gcc-3_4-e500-branch
      | reparent –use-order
      # /branches/gcc-3_4-rhl-branch
      | reparent –use-order
      # /branches/gcc-4_0-rhl-branch
      | reparent –use-order
      # /branches/gomp-01-branch
      | reparent –use-order
      # /branches/libgcj-2_95-branch
      | reparent –use-order
      # /branches/struct-reorg-branch
      | reparent –use-order
      # /branches/tree-cleanup-branch
      | reparent –use-order
      # /branches/apple-200508-beta-branch
      | reparent –use-order
      # /branches/bnw-simple-branch
      | reparent –use-order
      # /branches/egcs_gc_branch
      | reparent –use-order
      # /branches/ffixinc-branch
      | reparent –use-order
      # /branches/gcc-3_2-rhl8-branch
      | reparent –use-order
      # /branches/gnu-win32-b20-branch
      | reparent –use-order
      # /branches/structure-aliasing-branch
      | reparent –use-order

      # Likewise, for tag-creation commits.

      # /tags/egcs_1_0_1_prerelease
      | reparent –use-order
      # /tags/egcs_1_0_1_release
      | reparent –use-order
      # /tags/egcs_1_0_2_980309_prerelease
      | reparent –use-order
      # /tags/egcs_1_0_2_prerelease
      | reparent –use-order
      # /tags/egcs_1_0_2_release
      | reparent –use-order
      # /tags/egcs_1_0_3_prerelease
      | reparent –use-order
      # /tags/egcs_1_0_3_release
      | reparent –use-order
      # /tags/egcs_1_0_release
      | reparent –use-order
      # /tags/egcs_1_1_1_pre
      | reparent –use-order
      # /tags/egcs_1_1_1_prerelease
      | reparent –use-order
      # /tags/egcs_1_1_1_prerelease_2
      | reparent –use-order
      # /tags/egcs_1_1_1_prerelease_3
      | reparent –use-order
      # /tags/egcs_1_1_1_release
      | reparent –use-order
      # /tags/egcs_1_1_2_prerelease_1
      | reparent –use-order
      # /tags/egcs_1_1_2_prerelease_2
      | reparent –use-order
      # /tags/egcs_1_1_2_prerelease_3
      | reparent –use-order
      # /tags/egcs_1_1_2_release
      | reparent –use-order
      # /tags/egcs_1_1_prerelease
      | reparent –use-order
      # /tags/egcs_1_1_release
      | reparent –use-order
      # /tags/gcc-2_95-release
      | reparent –use-order
      # /tags/gcc-2_95_1-release
      | reparent –use-order
      # /tags/gcc-2_95_2-release
      | reparent –use-order
      # /tags/gcc-2_95_2_1-release
      | reparent –use-order
      # /tags/gcc-2_95_3
      | reparent –use-order
      # /tags/gcc-2_95_3-test1
      | reparent –use-order
      # /tags/gcc-2_95_3-test2
      | reparent –use-order
      # /tags/gcc-2_95_3-test3
      | reparent –use-order
      # /tags/gcc-2_95_3-test4
      | reparent –use-order
      # /tags/gcc-2_95_3-test5
      | reparent –use-order
      # /tags/gcc-2_95_test
      | reparent –use-order
      # /tags/gcc_3_0_1_release
      | reparent –use-order
      # /tags/gcc_3_0_2_release
      | reparent –use-order
      # /tags/gcc_3_0_3_release
      | reparent –use-order
      # /tags/gcc_3_0_4_release
      | reparent –use-order
      # /tags/gcc_3_0_release
      | reparent –use-order
      # /tags/gcc_3_1_1_release
      | reparent –use-order
      # /tags/gcc_3_2_1_release
      | reparent –use-order
      # /tags/gcc_3_2_2_release
      | reparent –use-order
      # /tags/gcc_3_2_3_release
      | reparent –use-order
      # /tags/gcc_3_2_release
      | reparent –use-order
      # /tags/gcc_3_3_1_release
      | reparent –use-order
      # /tags/gcc_3_3_2_release
      | reparent –use-order
      # /tags/gcc_3_3_3_release
      | reparent –use-order
      # /tags/gcc_3_3_4_release
      | reparent –use-order
      # /tags/gcc_3_3_5_release
      | reparent –use-order
      # /tags/gcc_3_3_6_release
      | reparent –use-order
      # /tags/gcc_3_3_release
      | reparent –use-order
      # /tags/gcc_3_4_0_release
      | reparent –use-order
      # /tags/gcc_3_4_1_release
      | reparent –use-order
      # /tags/gcc_3_4_2_release
      | reparent –use-order
      # /tags/gcc_3_4_3_release
      | reparent –use-order
      # /tags/gcc_3_4_4_release
      | reparent –use-order
      # /tags/gcc_4_0_0_release
      | reparent –use-order
      # /tags/gcc_4_0_1_release
      | reparent –use-order
      # /tags/gcc-2_8_1-RELEASE
      | reparent –use-order
      # /tags/gcc_3_1_release
      | reparent –use-order
      # /tags/gcc_4_0_2_release
      | reparent –use-order
      # /tags/libgcj-2_95-release
      | reparent –use-order
      # /tags/libgcj-2_95_1-release
      | reparent –use-order

      # Avoid merge commits on master.
      # cxx0x-lambdas-branch
      unmerge
      # lto
      unmerge
      # wide-int
      unmerge

      # Ensure libstdcxx_so_7-2-branch (created by creating directory then
      # copying subdirectory) is parented on relevant master commit.
      | reparent –use-order

      # Deletion of spurious commits.

      # Commit 243505 wrongly created a branch as a copy of / not /trunk; it
      # was then deleted.
      , delete

      # Try to generate better commit summary lines.
      msgout >gcc-commitlog
      shell ./fixbugmessages
      msgin <gcc-fixedprs

      # Strata tags.
      tag misc/cutover-git create
      msgin <<EOF
      Tag-Name: misc/cutover-git
      Tagger: Joseph Myers

      This tag marks the last Subversion commit before the move to Git.
      EOF

      tag misc/cutover-egcs-0 create
      msgin <<EOF
      Tag-Name: misc/cutover-egcs-0
      Tagger: Joseph Myers

      This is the snapshot of the gcc2 CVS repository on which EGCS was
      based. Subsequent changes in the gcc2 CVS repository are on
      premerge-fsf-branch. That repository started out as a collection of
      RCS files, and initially not all files were version-controlled, so the
      gcc2 history does not contain a complete buildable set of source
      files; in particular, documentation and ChangeLogs were not
      version-controlled until late in gcc2 history.
      EOF

      tag misc/cutover-egcs-1 create
      msgin <<EOF
      Tag-Name: misc/cutover-egcs-1
      Tagger: Joseph Myers

      This was the initial commit of the EGCS fork. Later revisions follow
      that fork, which became GCC 3.

      This is the first point at which the repository has complete coverage
      of the code tree.
      EOF

      tag misc/cutover-cvs2svn create
      msgin <<EOF
      Tag-Name: misc/cutover-cvs2svn
      Tagger: Joseph Myers

      This is the point at which cvs2svn was applied to move the history
      from CVS to Subversion. A custom version of CVS was used to combine
      the files from the gcc2 CVS repository with those from the EGCS /
      GCC 3 CVS repository before the result was converted with cvs2svn.
      EOF

      # Tags for ancient releases. See https://gcc.gnu.org/releases.html
      # for a list of ancient releases without tarballs; this one
      # is certainly not complete. These are from ssb.
      #define release {0} reset refs/tags/{1} create
      #do release 1.36
      #do release 2.0
      #do release 2.1
      #do release 2.2
      #do release 2.3.1
      # r4489 + r4492 + r4493 = 2.4.0
      #do release 2.5.0
      #do release 2.6.0
      #do release 2.7.0
      #do release 2.7.2

      # To be done: incorporate tarballs from ancient releases
      # Source tarballs are at:
      # ftp://sourceware.org/pub/gcc/old-releases/gcc-1/
      # ftp://sourceware.org/pub/gcc/old-releases/gcc-2/

      lint >gcc-lint

      # Massage comments into gittish form with separator lines where it can be
      # done automatically.
      gitify

      # Add the version of SVN that this commit was copied from
      =C append –rstrip –legacy “\n\nFrom-SVN: r%LEGACY%\n”

      # Statistics on read and processing times
      timing

    2. I’d note that the case report is commonly used in medicine as a way of exchanging experience of rare incidents.

      Providing a series of documents along those lines might be the best approach practical.

  5. I don’t think it’s impossible any more. I agree that toy examples might be too trivial to be a huge use, but that’s not insurmountable. Brendan Gregg wrote a great GDB tutorial using a real-life example of a specific bug found in a specific piece of software (http://www.brendangregg.com/blog/2016-08-09/gdb-example-ncurses.html). The user could, in principle, reproduce this exact error on their own machine simply by using the same versions of the libraries involved. Robert O’Callahan used the same bug to demo Pernosco (https://robert.ocallahan.org/2019/10/pernosco-demo-video.html). I recommend examining both; they show the great contrast between these two debuggers.

    The same could be done for GCC. It’s been used to convert a lot of repositories, so there must be stories like this that we could record. Of course we didn’t think to keep a copy of every old crusty SVN repository that’s been converted, but I do still have the dump file from the GCC conversion. This is not ideal as a reproducible example due to its size (402GB!), but this is the simply price of our lack of omniscience.

    There are a lot of interesting things in gcc.lift, such as “<243505>,<243547> delete”, “~=M changelogs /^ChangeLog(\..*)?$/” or “msgout >gcc-commitlog” “shell ./fixbugmessages” “msgin <gcc-fixedprs”. For each of them there is a story that could be told about how some defect was discovered in git repository created by Reposurgeon, and how Reposurgeon commands can be used to remedy that defect.

  6. >Of course my punchline is that reposurgeon has this problem, and arguably an even worse example of it than gdb’s.

    Diachronically, I can’t imagine anything much worse than GDB’s problem. The user cannot be guaranteed to be using the platform that was used to write the example. The example might be written for foonix on m68k, and the user might be using Linux on z/Architecture.

    Synchronically, however, the problem is somewhat ameliorated by the fact that PC and server *nix these days is largely a Lintel monoculture, and most of the x86 install base these days is running 64 bit software on 64 bit hardware. That wasn’t always the case, and might not be in the future, but it is the current situation.

  7. If the problem is “documentation that is an easy introduction for beginners”, then toy examples _are_ a good solution. You have always disliked toy programs and contrived examples because, as you say, they are “useless because the example is too trivial.” I disagree. Simple examples for _beginners_ can help answer questions so basic you aren’t even considering them, because I’m not sure you experience being a beginner the way many people do. Yes, even the kinds of people who would reach for Reposurgeon as a tool.

    Oh, this is a command line program? It takes some input and generates output . . . how do I tell it what and where? What does it look like in general when it is working? What do error messages look like? How do I know when it’s done? Ooo, I might have to do some prep work on my source material before I can begin. etc. A simple toy example can be worth one thousand man-page words.

    I learned the word “generative” from you many years ago. Toy examples may not help a beginner with their specific problem, but than can contain general guidance that give the user useful intuition about how they can use the tool to do the job at hand. And . . . it’s better than nothing.

    1. And I completely agree with Jason Azze. I supply and attempt to document a complex software system; I have plenty of experience in this area, going back decades. Something is always better than nothing. A simple worked example gets the user started. It can often be gradually mutated into a complicated and useful solution to the problem.

  8. I came here to make pretty much the point Jason Azze has.

    > In the gdb documentation, you could in theory write …
    > but essentially useless because the example is too trivial.

    This is simply incorrect. It is easier to vary from success from failure; a working example is essential to the introduction to any project. The more a developer believes that he is the exception, the more important it is for him to provide a working example. (An analogous example from the old O’Reilly pocket guides: A few of them were so well-organized that they didn’t need an index. These were all written by writers so obsessive that they provided indexes as well. The guides written by those who didn’t think they needed an index, all badly needed an index. I ended up memorizing the page number for the Perl guide’s regular expression subsection.)

  9. Describe the problem domain.?Describe how reposurgeon attacks the problem; what methods it uses internally, what external tools it provides.?Show how those can be combined for one or two examples.?The user now has the knowledge to concoct more solutions because they understand the underlying mechanisms, not just the surface veneer from the examples.

    The man page is a reference for running the command.?The accompanying paper covers the rest, just as the Unix editions came with papers for more complex commands and topics.

  10. A user can’t understand a solution without understanding the problem.

    To make worthwhile documentation for a beginner, you’d have to clearly introduce (or be able to expect that the user already understands) the problem space while showing how the program assists in interacting with that problem space.

    I think the primary issue here is that you have a tool whose predominant utility involves difficult tasks. A debugger will be avoided if the code in question is short and/or simple enough to evaluate mentally or with the assistance of a few printfs. As a basic debugging technique, printfs have a beautifully low barrier for entry that corresponds well with the skills necessary to write offending code that needs to fixed when you’re a beginner. It also scales (perhaps further than one would like) as the programmer becomes more skilled, knowing the more effective places to gain insight into their broken code.

    Using a debugger isn’t as natural of a transition. The skills you gained writing code don’t directly translate and scale into being able to use a debugger well. If it’s not with a toy problem, you’re using a debugger for the first time in the worst conditions — you’re frustrated, the problem is more complex than you’ve been able to figure out otherwise, and you’re thrust into an unfamiliar environment with its own control language — you had one problem, and now you have two.

  11. Emphasis mine:

    … the author, like all too many programmers, hates writing documentation, has never gotten very good at it, and will evade frantically when under pressure to try. But in my case none of that description is even slightly true. Like Donald Knuth, I consider writing good documentation an integral and enjoyable part of the art of software engineering. If you don’t learn to do it well you are short-changing not just your users but yourself.

    This is why I found Knuth’s literate programming concept impressive — the core conceit really has nothing to do with the commingling of the various categories of programming artifacts [comments are a kind of commingled documentation which existed before WEB, even if they cannot suffice as end-user manuals], but the attendant requirement that all types of output artifact be developed in coordination. Naturally, the complications of how Knuth’s WEB would disentangle code from documentation makes the concept-as-presented rather impractical, but the success of related ideas such as JavaDoc shows the essential idea has unmistakable benefits.

    The obvious inversion would be something like this: to improve the quality of your introductory documentation, design your software so that user reproduction of instructive examples is as easy as its problem domain allows. I understand that this only pushes the boundaries of the problem. It doesn’t tell you what to do when you’re in a problem domain as intrinsically hostile to reproduction of examples as gdb and reposurgeon are.

    I’ve never written code in a domain quite as hostile as that, but still my solution to this has always been: draft the end-user guides first, before any code. While I recognize this quite likely sounds ridiculous, it serves multiple purposes which — outside the structured corporate development cycle — I find difficult to achieve any other way. First and foremost among them, is that it requires me to record the “user story” / use cases for which I began the project, before being exposed to the development details which can change them. There is a good chance that these are the examples which are most likely to be instructive, since they were originally written from my most ignorant viewpoint and must be updated as my understanding changes. If there is any superior method for aiding others in mastery of a topic than this sort of forced documentation of your own journey, I certainly haven’t seen it.

  12. Isn’t the issue ‘What are you documenting?’ The operating instructions of a tool, or how create something / solve a problem?

    Consider a radial arm saw. The instructions, even tutorials, don’t include ‘How to cut crown molding, and create an end table, and cabinet repair.’ Rather, they include how to safely use and adjust the tool. The use of the tool for project is a separate discussions. In fact, the discussion becomes, given this task, what are the best tools (available to me or for acquisition) to let me complete it?

    When something is in a tool chain you can document the behavior, but configuring a tool chain to solve a problem is a complete different narrative. Context matters. In the OP example, GIMP is, in many contexts, the complete tool chain, and so the behavior may be illuminated with a narrative tutorial.

  13. For an author, it takes an effort to disconnect from the knowledge of the internals and all the details, and look from the perspective of a typical basic user. For a simle example, think of the “tar” command: it has a good deal of options but pretty much all you need to know to use it in practice are “czvf” and “xzvf”. From my experience, it takes time and effort to achieve this disconnection, and kind of look from the side.

    For gdb, the typical things to do are:

    * load a core (and nowadays this comes with the problem of how to create a core file in the first place, because the modern Linux systems discard it by default!)
    * backtrace
    * switch to a specific frame
    * print a variable
    * print registers
    * disassemble
    * dump memory in raw hex/ascii
    * find the list of threads and backtrace them
    * switch to a specific thread

    That’s it! There is no need for a repeatable example. Just show them on SOME example, and everyone can repeat it on their own program.

  14. I encountered a similar issue in the very long ChangeLogs discussions on bug-standards. RMS was fixated on any alternative to using ChangeLog format needing a canned recipe, that would work in all cases, for the specific task of mapping from a commit to a list of the named entities changed in that commit. But (a) that was the wrong problem (the practical issue in development is much more the inverse problem of mapping from an entity to the commits that changed it, which the version control history can solve much better than ChangeLog files can) and (b) while there are a few key commands such as “git blame” to learn, learning the practical process of using the version control history in the course of developing software requires actually *doing* it, for the particular software you care about, not just reading a few toy examples of someone else’s use case.

  15. This sounds similar to the problem of setting up end-to-end testing. The spec of a well-written functional test– input, operations, expected output– amounts to a reproducible worked example, right?

    1. >This sounds similar to the problem of setting up end-to-end testing. The spec of a well-written functional test– input, operations, expected output– amounts to a reproducible worked example, right?

      Yes, and I have lots of such tests. But they make terrible intro documentation. By definition they’re hyperfocused on individual point manipulations, and seldom convey much generative knowledge about the overall process.

  16. You state that you don’t want to build the documentation around toy programs, like your example buggy Hello World, lest it give the false impression that the problem space is trivial to work in. And yet, I can’t help but find myself thinking, it can’t be completely impossible to synthesize an example at a medium level of complexity — something complex enough to be instructive, but not as soul-devouring as GCC.

    Sure, each repository conversion where reposurgeon has been used has been its own problem space, each with its own quirks, traps, and gotchas. But is it really the case that every single repository conversion was its own unique set of problems, sharing absolutely no elements or sub-problems with any other repository conversion done prior or since? Having done absolutely no respoitory conversions myself, I’m assuming (or hoping?) that’s not the case. It doesn’t make sense to me that reposurgeon would even exist then; wouldn’t each repository conversion have to be laboriously done by hand?

    Rather, I’m imagining that if the kinds of problems and the number of conversions where those problems were found were plotted on a graph, it would form a skewed distribution with a (very?) long tail. There would be a cluster of problems that were present on every conversion, or at least, most of them. Then, there would be other problems that became less and less frequent, until at the far end of that tail, you had those unique problems that happened on specific individual conversions and never since. Is this, or something fairly close, the case?

    Assuming that’s the case, here’s how I might attempt to structure a tutorial:

    I would probably break the tutorial into multiple parts. The first part would be built around a synthetic repository to convert, one closer to the toy repo than not, containing the easiest of those most common problems to address. The goal of this first stage would be twofold: guide the the user into the kind of thinking/mindset needed to work in this problem space, and give him a basic familiarity with how reposurgeon is generally used. If it were practical to do so, there would be a set of tests to run against the user’s converted repository, to help determine if he had successfully tackled all the problems and gotten a fully correct conversion.

    The next part would have another, more complex synthetic repository to convert. The problems in this one would be a superset of the first one, containing the easy common problems, as well as some tougher, less common problems. And this pattern would continue until there were enough parts that I was confident that a user who had completed the tutorial could think in the problem space, and find their own way forward on their own repositories.

    It’s possible that I am failing to appreciate just how difficult it is to replicate problems in this space. Perhaps it is impossible to construct a synthetic repo complex enough to be instructive instead of a trivial toy. In that case, do you still have copies of the RCS/CVS/SVN/etc repositories you’ve converted? Could one or more of those serve as instructive examples of not just how use reposurgeon, but more generally, how to think in the problem space of repository conversion?

    1. >But is it really the case that every single repository conversion was its own unique set of problems, sharing absolutely no elements or sub-problems with any other repository conversion done prior or since?

      Yes and no. You should study the GCC lift file I posted in an earlier comment.

      The problem is that the shared problems are, in general the trivial ones that are solved inside the DSL primitives. The unshared problems are the nasty, unique, ones that would be most important to document solutions to.

      1. While the shared problems may be the most important to document solutions on, I think it would be beneficial to include the trivial cases to better understand how the application works to someone coming in totally dark who wants to migrate a repository. You could provide a disclaimer that this might not represent a real world production issue or methodology, but I still think it would address the original complaint about the documentation issue.

  17. It’s been sort of mentioned in other comments but i’m thinking perhaps a soup to nuts stream of consciousness style “diary of a reposurgeon” would be useful? Take a real world project you’ve already done of medium complexity(by medium complexity I mean something that isn’t trivial and had some surprise moments but not something that required you to rewrite parts of reposurgeon) and then talk through how you went about converting it. What pre-work did you do? Knowing what you know about repos in general, what things did you look for (because they cause issues)? At each point, what results were you expecting and what did you get? How did that change your next step?

    My goal would be to describe one journey that you took and describe the feedback the process gave and how it guided the next step. Yes every journey is fundamentally different and you’ll want to hammer that point in the first “chapter” but even the description of one journey is better than 0 descriptions. (also it might be an interesting read even for those who have no plans to convert a repo)

  18. Just skimming through the script I thought at first that “reparent” was actually “repent.”
    MOST software should come with a “repent” command.

  19. I think there exists a general case problem. Some domains are simply so complex that you have to be a domain expert before you can solve anything but the toy problems. Any tutorial thorough enough to leave the user well prepared to tackle the hard problems would be literally a textbook.

    1. I think the general problem is related to some issues in education and hiring.

      Consider the hypothetical of a highschool graduate who tests well enough to enter into an accredited Chemical Engineering program. They take and pass classes from a bunch of different stubborn people with academic backgrounds in Chemical Engineering. Then they get a job at an oil refinery with the degree, and spend the time working under PEs to qualify for the exam. They take the PE exam, pass, and get the certification from the state board of licensure. Dangerous idiots do come out of that process. Historically, it has certified a lot of PEs smart and careful enough to realize that oil refinery experience does not automatically translate to, say, a ground beef process.

      No one could write a tutorial so good that it would take a high schooler directly to the mindset of that Chemical Engineering PE with a background in some esoteric refinery minutia. In part because part of the change in mind set is the experience with many different professional minds. Definitely, even if you could write the tutorial, and test the results, there are so many little specialties that it would be impossible to set up such training and certification for all of them. Which means that hiring engineers is as an HR process impossible to automate and perhaps very unreliable.

      Programming looks weird to me because you’ve got what looks like trades, engineers, and academics all mixed together without any even marginally useful certification process. (The status quo in traditional engineering disciplines has flaws, which may well cause huge problems.) I do not know what can or should be done, if anything.

      In conclusion, there are domains more remote than a text book is sufficient for. Nested domains, remoteness of domains, and adequate paths/materials to prepare for a given domain are probably things we need better descriptions for.

  20. You are tutoring apprentice programmers. Why? Isn’t it more efficient to write a textbook? Aren’t there already plenty good textbooks? I think the answer is that problem-solving skills cannot be entirely verbalized, so you demonstrate them to the apprentice on a particular problem and hope the apprentice is going to grasp the more general non-verbal principles, methods through their usage on that particular example. Zen-master type of thing.

    But if you would write a programming textbook anyway, one that would reduce the time needed for tutoring apprentice programmers, what would in contain? Case studies, I think. With enough case studies everybody finds one that is relevant to the problem they are working on and then same grasping of the more general non-verbal principles can happen.

    And I think in this case too. But this is something you already know, I mean, if you would have documented every step of you using this tool, every problem, every thought, every failed and succesful approach, a complete brain dump, then of course that would work, the issue is that that would be enormous. So your actual problem is how to cut down that potential 1000-page brain dump to something shorter.

    I can’t really recommend anything else than to do a few shorter conversions after this and keep a journal of every step, problem, thought, failed and succesful approach. Like, use a dictaphone and just think aloud all the way through. Then a number of people will use them, and that will cut down on the time needed to tutor them.

    1. >You are tutoring apprentice programmers. Why? Isn’t it more efficient to write a textbook?

      I’m doing both, actual – tutoring and writing a book. Using my teaching experience to inform the book.

  21. The documentation is complex because you’ve really built a “reposcalpel”. A repo surgeon is still required to operate it. You’re trying to stuff all of “repo medical school” into the documentation.

    1. >The documentation is complex because you’ve really built a “reposcalpel”. A repo surgeon is still required to operate it. You’re trying to stuff all of “repo medical school” into the documentation.

      That is a fair point.

      Do you have any ideas about how to solve or reframe the general problem? It’s a difficult one.

  22. *This isn’t the 70s.*

    That state of cryptic parameters and commands, and separate man pages, is just as bad as those stupid “search” interfaces, that mimic CLIs and force you to play a losing game of Family Feud, yet are all the rage nowadays.

    Please write your software in a way that it is self-explaining. In every state it can be in. And telling you when it would lead you into situations you can’t revert. No exceptions. (Ideally, make *everything* undoable… Yeah, I’m looking at you, Android and iOS!)

    And I don’t mean explanation texts or popups. I mean you can see what you can do and what it would result in and how it is meant to be use *right from looking at it*!

    In fact, that should be written first. A la literate programming, but from a UI standpoint. Then, if done right, you got your model’s unit tests for free.

    1. >Please write your software in a way that it is self-explaining. In every state it can be in.

      I’ll tell you what. When you exhibit a functional equivalent of gdb with this property I will take your exhortations seriously. Until then I’ll just laugh.

      If you don’t understand why gdb is relevant, you don’t understand the design problem here,

        1. >Have you ever used the integrated debugger in Visual Studio?

          No. But don’t bother claiming it does the full-display-of-state thing; I looked at some videos.

          1. I’m not making that sort of strong claim — only that Visual Studio comes closer to that ideal than gdb does, and that’s why people loathe working with gdb and love working with Visual Studio (despite the latter having lots of bugs and not even being 64-bit clean).

            Closer still is the work of Bret Victor and programs based on it, like the next-gen IDE Light Table. Seriously — read Bret Victor’s essays or watch his talks to get a sense of how stone-knives-and-bearskins the traditional Unix stack of text editor, compiler, debugger, and tools truly is.

            Anyway, I don’t think BAReFOOt was demanding that programs display their entire state, only that they make all actions that can be performed — and whether there are serious irreversible consequences for such an action — obvious, at any moment in time. This is really UI 101. It’s one of the principles behind the Macintosh UI guidelines dating back to 1984.

            1. > This is really UI 101. It’s one of the principles behind the Macintosh UI guidelines dating back to 1984.

              I understand that philosophy. I’ve even designed by it. Yes, I can design pretty good UI, I did some I’m still proud of when I was working on Battle For Wesnoth. You don’t see me do this more often only because visual UIs don’t tend to be wanted on the kinds of code I find most interesting to write.

              But there are domains where the Mac philosophy can be practically applied versus domains where attempting it would either have unacceptable complexity overhead or conflict with other design objectives that are more important than an easy-to-use UI. I did at one point try to seriously think through what it would take to give reposurgeon an undo/redo queue – I’m a big fan of that UI design pattern as exemplified in Emacs and GIMP – but rapidly concluded the complexity was prohibitive.

              I even contemplated designing a GUI front end for reposurgeon using PyGTK. What stopped me? Well, if you look at reposurgeon you’ll find that it can emit graph descriptions of repository DAGs in DOT markup; it’s trivial to pipe that to viewer and I give a recipe for it.

              Cool-sounding feature, utterly useless in practice because real repository DAGS are waaaaay too huge and complex for even the three monster monitors I use. Even subsections of those visualizations large enough to be diagnostically useful are too large.

              Generalizing: Naive advice that reposurgeon should be transparent about repository state hits a scale problem and sinks. I know what could change that: VR driving an immersive interface that lets you fly through the DAG and zoom in on metadata. Still somewhat impractical, especially since I’m in the unfortunate minority vulnerable to VR nausea.

              1. The full DAG is usually too large and messy, but I wonder if simplifications would be useful: preserve the branching structure, but compress any fully linear sections to a single node.

                1. >The full DAG is usually too large and messy, but I wonder if simplifications would be useful: preserve the branching structure, but compress any fully linear sections to a single node.

                  Alas, reality says no. I have a “reduce” command to do that topological reduction that I wrote partly to try this.

                  (Its other use is for simplifying gnarly test cases in a way that preserves the diagnostically interesting parts. If you send me a repository that breaks reposurgeon, it is very likely that I will strip it, reduce it, fix the problem, and then add the stripped/reduced version to my test suite.)

  23. It sounds like the core problem is that the domain knowledge isn’t common enough to have a base level to which to speak. i.e., a database programmer can assume that the audience knows something about what a database is and what it’s for and can focus on explaining how to use *this* database. Even much of the common office software was eased by being targeted at technical users who had an analogy – there was a time where paper spreadsheets existed, so a manual for something like Lotus 1-2-3 would be a manual on how to do some physical task with *this* piece of software.

    If that’s the missing link for something like Reposurgeon, it sounds like the manual itself has to take a backseat to writing a treatment of the domain that happens to have some information on Reposurgeon. Looking at the GCC script above seems to bear this out, as the primitives to rewrite committers or preserve tags are based on general desirable end states (like correct history which was sadly omitted when the company I work for migrated from SVN to Git) and understanding how to map that over to the primitives offered by Reposurgeon.

    There is even some precedent for this. Take “Version Control with Subversion” (http://svnbook.red-bean.com/). It starts with talking about source control and introduces Subversion commands later.

  24. I would really appreciate documentation on the core algorithms of reposurgeon. The Graph Algorithms, and the “three operator-composition algebras” that you developed.

    You could probably write a nice paper on them!

Leave a Reply to ktk Cancel reply

Your email address will not be published. Required fields are marked *