git-weave, a tool for synthesizing repositories from fossil tarballs

Welcome to my first new-project release of the year, git-weave. It’s a polished and documented version of the script I used to reconstruct the early history of INTERCAL five years ago – see Risk, Verification, and the INTERCAL Reconstruction Massacree for the details on that one.

git-weave can be used to explode a git repository into a sequence of per-commit directory trees accompanied by a metadata file describing parent-child linkage, holding committer/author/timestamps/comment metadata, and carrying tags.

Going in the other direction, it can take the same sequence of trees plus metadata file and reconstruct the live repository. Round-tripping is lossless.

What it’s really useful for is reconstructing a partial but useful ancient history of a project from before it was put under version control. Find its release archives, synthesize a metadata file, apply this tool, and you get a repository that can easily be glued to the modern, more continuous history.

Yes, you only get a commit for each release tree or patch you can dig up, but this is better than nothing and often quite interesting.

Nifty detail: the project logo is the ancient Egyptian hieroglyph for a weaver’s shuttle.

20 thoughts on “git-weave, a tool for synthesizing repositories from fossil tarballs

  1. Quite excellent; I’ve had scripts a couple times (but I think I’ve lost them) to do a similar task, and done the work manually if the amount of tarballs is somewhere around 20 or less. I’m glad to see a proper release to do this sort of things.

  2. It looks useful also to import from source control systems that can’t generate a fast-import file. It’s a toss-off bash loop to get something to check out every rev into another directory. Then some simple text massaging to turn a “vc log” output into a metadata file.

  3. Well, how git-weave is different from contrib/fast-import/import-directories.perl script (well, except that import-* scripts are only in one direction)?

    • > (well, except that import-* scripts are only in one direction)?

      That’s a big difference! I didn’t know the script you point at existed; I’ll have to look at it.

  4. It’s like you’re building a toolkit for operating on software source revision histories in the general sense. One level up from a VCS, which presumes a particular database and a particular format, you’re building tools to capture, transform, and convert project history for very general values of that term, lessening the dependence on a certain database in a certain format.

    The need we’ve felt for tools of this sort is palpable, so I thank you.

  5. Am I the only one who cringed after visualizing someone attempting to weave with tar balls in his hands? ;-D

  6. Sounds cool! Now, the usual nitpicking: in the first paragraph, you wrote “massacree”. Correcting you makes me feel like an asshole, but you haven’t complained yet. :-P

    > What it’s really useful for is reconstructing a partial but useful ancient history of a project from before it was put under version control.

    Is this what you meant by “repairing the very fabric of time”? :-)

    • >Sounds cool! Now, the usual nitpicking: in the first paragraph, you wrote “massacree”. Correcting you makes me feel like an asshole, but you haven’t complained yet. :-P

      But I was not wrong, grasshopper. There is mystery here you have not penetrated. Google for “Alice’s Restaurant” and perhaps you will achieve enlightenment.

      >Is this what you meant by “repairing the very fabric of time”? :-)

      No. I’m afraid I still have to be mysterious about that for a few weeks.

  7. Jorge: “Now, the usual nitpicking: in the first paragraph, you wrote “massacree”.”

    That, as with much else about the linked post, was an intentional reference to the classic Arlo Guthrie tune Alice’s Restaurant. It’s definitely a nice way to wastespend 18 minutes. Check out the recording on YouTube.

  8. @ Jay Maynard

    > That, as with much else about the linked post, was an intentional reference to the classic Arlo Guthrie tune Alice’s Restaurant.

    Oops! Looks like this time I’ve screwed up in an epic way. :$ Thanks for the correction.

    @ esr

    > But I was not wrong, grasshopper. […] Google for “Alice’s Restaurant”

    Sorry, Master Po: Jay Maynard beat you to it. =P

    > No. I’m afraid I still have to be mysterious about that for a few weeks.

    You’re a contender for Hitchcock’s title of “Master of Suspense”. :-)

  9. Well I’m waiting for the ’28 color photos with the circles and arrows and the paragraph on the back of each one explaining what it was’….

    • >A bug report: `make install` complains because `git-weave.xml` does not exist.

      Fixed in the repo. Probably not worth shipping a point release, but it’ll go out in 1.1.

    • >Another bug report: `git weave` eats blank lines in commit messages when unraveling a repo.

      All blank lines? Leading ones? Trailing ones?

  10. > All blank lines? Leading ones? Trailing ones?

    All blank lines. If it were just leading or trailing blank lines, i wouldn’t have even noticed, since git usually strips those anyway.

    Here’s a small example showing the rawlog and log files that git-weave produces when run on a simple test repository: https://gist.github.com/magical/b171a3b9841717d2039e

  11. > Please email me a fast-import dump of that repo

    I’m afraid i deleted it. It’s simple enough to recreate.

    % mkdir test && cd test
    % git init
    % git commit –allow-empty

    Paste your favourite koan into the commit message.

Leave a Reply

Your email address will not be published. Required fields are marked *