{"id":5634,"date":"2014-03-29T19:28:25","date_gmt":"2014-03-29T23:28:25","guid":{"rendered":"http:\/\/esr.ibiblio.org\/?p=5634"},"modified":"2014-03-29T22:17:20","modified_gmt":"2014-03-30T02:17:20","slug":"ugliest-repository-conversion-ever","status":"publish","type":"post","link":"http:\/\/esr.ibiblio.org\/?p=5634","title":{"rendered":"Ugliest&#8230;repository&#8230;conversion&#8230;ever"},"content":{"rendered":"<p>Blogging has been light lately because I&#8217;ve been up to my ears in reposurgeon&#8217;s most serious challenge ever.  Read on for a description of the ugliest heap of version-control rubble you are ever likely to encounter, what I&#8217;m doing to fix it, and why you do in fact care &#8211; because I&#8217;m rescuing the history of one of the defining artifacts of the hacker culture.<\/p>\n<p><!--more--><\/p>\n<p>Imagine a version-control history going back to 1985 &#8211; yes, <em>twenty-nine<\/em> years of continuous development by no fewer than 579 people.  Imagine geologic strata left by no fewer than <em>five<\/em> version-control systems &#8211; RCS, CVS, Arch, bzr, and git.  The older portions of the history are a mess, with incomplete  changeset coalescence in the formerly-CVS parts and crap like paths prefixed with &#8220;=&#8221; to mark RCS masters of deleted files.  There are hundreds of dead tags and dozens of dead branches.  Comments and changelogs are rife with commit-reference cookies that no longer make sense in the view through more modern version-control systems.<\/p>\n<p>Your present view of the history is a sort of two-headed monster.  The official master is in bzr, but because of some strange deficiences in bzr&#8217;s export tools (which won&#8217;t be fixed because bzr is moribund) you have to work from a poor-quality read-only git mirror that gets automatically rebuilt from the bzr history every 15 minutes.  But you can&#8217;t entirely ignore the bzr master; you have to write custom code to data-mine it for bzr-related metadata that you need for fixing references in your conversion.<\/p>\n<p>Because bzr is moribund, your mission is to produce a full standalone git conversion that doesn&#8217;t suck.  Criteria for &#8220;not sucking&#8221; include (a) complete changeset coalescence in the RCS and CVS parts, (b) fixing up CVS and bzr commit references so a human being browsing through git can actually follow them, (c) making sense out of the mess that is RCS deletions in the oldest part of the history.<\/p>\n<p>Also, because the main repo is such a disaster area, there is at least one satellite repo for a Mac OS X port that really wants to be a branch of the main repo, but isn&#8217;t. (Instead it&#8217;s a two-tailed mutant clone of a nine-year old version of the main repo.) You&#8217;ve been asked to pull off a cross-repository history graft so that after conversion day it will look as though the whole nine years of OS X port history has been a branch in this repo from the beginning.<\/p>\n<p>Just to put the cherry on top, your customers &#8211; the project dev group &#8211; are a notoriously crusty lot who, on the whole, do not go out of their way to be helpful.  If not for  a perhaps surprising degree of support from the project lead the full git conversion wouldn&#8217;t be happening at all. Fortunately, the lead groks it is important in order to lower the barrier to entry for new talent.<\/p>\n<p>I have been working hard on this conversion for eight solid weeks.  Supporting it has required that I write several major new features in reposurgeon, including a macro facility, large extensions to the selection-set sublanguage, and facilities for generic search-and-replace on both metadata and blobs.<\/p>\n<p>Experiments and debugging are a pain in the ass because the repository is so big and gnarly that a single full conversion run takes around ten hours.  The lift script is over 800 lines of complex reposurgeon commands &#8211; and that&#8217;s not counting the six auxiliary scripts used to audit and generate parts of it, nor an included file of mechanically-generated commands that is over <em>two thousand<\/em> lines long.<\/p>\n<p>You might very well wonder what could make a repository conversion worth that kind of investment of time and effort. That&#8217;s a good question, and one of those for which you either have enough cultural context that a one-word answer will suffice or else hundreds of words of explanation wouldn&#8217;t be enough.<\/p>\n<p>The one word is: Emacs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Blogging has been light lately because I&#8217;ve been up to my ears in reposurgeon&#8217;s most serious challenge ever. Read on for a description of the ugliest heap of version-control rubble you are ever likely to encounter, what I&#8217;m doing to fix it, and why you do in fact care &#8211; because I&#8217;m rescuing the history&hellip; <a class=\"more-link\" href=\"http:\/\/esr.ibiblio.org\/?p=5634\">Continue reading <span class=\"screen-reader-text\">Ugliest&#8230;repository&#8230;conversion&#8230;ever<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[33],"class_list":["post-5634","post","type-post","status-publish","format-standard","hentry","category-software","tag-reposurgeon","entry"],"_links":{"self":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/5634","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5634"}],"version-history":[{"count":4,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/5634\/revisions"}],"predecessor-version":[{"id":5639,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/5634\/revisions\/5639"}],"wp:attachment":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5634"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5634"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5634"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}