For those of you who have been following my work on tools to muck out the swamp that is CVS, a new shovel: cvs-fast-export. Note, this is an alpha release for testing purposes; double-check the quality of any conversions you do with it carefully.
Actually, in some ways this is an old and trusty shovel. Keith Packard wrote the analysis code in 2006 to lift the X repos to git. After that the code gathered dust; when I found it it was in a broken state due to changes in the git library API. Some of you may have seen that version as “parsecvs”.
I sawed off the direct interface to git and replaced it with a back end that emits a git fast-import stream. This is much more useful; it means other version-control systems that have importers for this format can use the tool. It also means reposurgeon can use it.
If you’ve been paying attention, you’ll notice that this is the second CVS exporter I’ve shipped recently. And you’re probably wondering why I’m maintaining two of these beasts. The answer is that they both fell into my lap at nearly the same time and I’m not yet sure either dominates the other.
One of the larger items on my near-term to-do list is to write a really good test suite for both of these, then use it to compare them with each other and cvs2git.
Just from reading the code of both I suspect that cvs-fast-export will do a better job than cvsps – Keith’s analyzer looks more powerful, less ad-hoc. But cvsps has one major feature lacking in cvs-fast-export; it can be used to lift repositories to which you have only remote access.
Depending on what the test-suite comparisons tell me, I may have to fix that by transplanting cvsps’s client code into cvs-fast-export (after which I would scrap cvsps). We’ll see. There’s a fair bit of work still to be done before I’ll know enough to make that decision.
In the meantime, those of you interested in this problem can test. And here’s a shiny thing I dangle before you: with the flip of a switch, cvs-fast-export can generate a graphical visualization of the commit DAG.
> Just from reading the code of both I suspect that cvs-fast-export will do a better job than cvsps
Judging from the fact that you’ve already given out the name “csv-fast-export”, you must be pretty confident in that prediction :-)
>Judging from the fact that you’ve already given out the name “csv-fast-export”, you must be pretty confident in that prediction :-)
One reason that was easy to do is that if it’s a mistake it’s easy to reverse. I maintain both codebases, so whatever good bits cvsps has can easy find their way into the other distribution.
I am at one of the few sites in the world using proprietary version control system Surround SCM (which is why the pseudonym). We want to get out of it and move our years-deep repo to git. Surround runs on the roach motel business model, where it will import anything and export nothing. Is there anyone who has got his data out of Surround? Ever? If we can just get it out of Surround to *anything* then we can reposurgeon it to git.
>Surround runs on the roach motel business model, where it will import anything and export nothing.
I don’t know how to rescue your data, but I think your generalization of Ken Thompson’s famous “roach motel” crack needs to go in the Jargon File.
Would you consider making cvs-export-commit more user-friendly, adding a possibility to run it from within CVS-controlled repository (like `git fast-export –all`), or to specify where local repository is located via `-d` option or CVSROOT environment variable?
`find . -name ‘*,v’ -print | cvs-fast-export -k` is not very user-friendly…
>`find . -name ‘*,v’ -print | cvs-fast-export -k` is not very user-friendly…
No, but it does have the good feature that it’s easily applied to an RCS collection as well as a CVS one – no dependence on CVS metadata.
>>`find . -name ‘*,v’ -print | cvs-fast-export -k` is not very user-friendly…
> No, but it does have the good feature that it’s easily applied to an RCS collection as well as a CVS one – no dependence on CVS metadata.
There is minuscule chance that such RCS collection would require reconstructing whole-repository commits from single-file commits (the patchset (‘ps’) part in CVSps); I think that in must such cases you either have single-file history, or single-file changesets.
Anyway if you don’t want to add code related to extracting CVS metadata from ‘CVS’ directory in the case of running cvs-fast-export from CVS-controlled repository, or parsing $SVSROOT/modules and scanning for *,v files in cvs-fast-export (which is not named rcs-fast-export by the way), then perhaps a separate command or commands to do the job? For example:
$ cvs-find-files | cvs-fast-export
or
$ cvs-module-files -d $CVSROOT cvsmodule | cvs-fast-export
>Anyway if you don’t want to add code related to extracting CVS metadata from ‘CVS’ directory in the case of running cvs-fast-export from CVS-controlled repository
That might happen. It’s early days – I haven’t put together all the tests for my performance comparison yet.
> I am at one of the few sites in the world using proprietary version control system Surround SCM (which is why the pseudonym). We want to get out of it and move our years-deep repo to git. Surround runs on the roach motel business model, where it will import anything and export nothing.
A bit off topic: the Veracity (http://veracity-scm.com), a new open-source distributed version control system by SourceGear (incl. Eric Sink) includes now both fast-export and fast-import tools to make it easy to try it out, without risk of trapping (meta)information…
Addition which I think was inspired by yours truly in (if I remember it correctly) my comment in pre-release review of Eric Sink’s “Version Control by Example” (http://www.ericsink.com/vcbe/).
http://arstechnica.com/gadgets/2013/01/17000-linux-powered-rifle-brings-auto-aim-to-the-real-world/
Completely unrelated, but holy shit, it’s your dream come true! I am from Europe* and I believe only Americans don’t realise the benefits of gun control (mainly because they don’t understand how to control the flow of guns to civilians effectively), but still, pretty interesting, if not scary.
* I also grew up in a rural area so don’t give me that ‘hunters’ crap. People that need weapons to do their job will get hold of them and in 99% of cases use them lawfully.
@ AA….hunting. My personal ethic is: outside of food production for ones family, hunting should pit the skill of the hunter vs. the prey. Seems rather unseemly to use an ipad to kill bambi. However, I suppose one could say the same about a rifle or even a bow. A true Luddite, could claim I should face my prey naked with my bare hands. They could have a point.
If anything, hunting has taught me how weak I am…even with technology. Ever faced an angry bull elk with a bow and arrow?…..makes you take stock I assure you.
Something the Luddite is sure to miss
The technology you are showing is best employed against moochers……..not elk.
But when it all fails, its down to the hunting……and the hunted….I know what I am.
Nb. it took me, a non-American, a Google and a Wikipedia for me to get to know that “roach motel” is a type (and a brand) of insect trap:
http://en.wikipedia.org/wiki/Roach_Motel_%28insect_trap%29
Thanks Jakub. As another non-American my first thought was it referred to a less glamorous class of motels…
Ha, thanks! Surround stores all the data in Oracle (yes really — I cannot think of a world in which this is a better idea than flat files — and Surround 2011 performance is enough worse than Surround 2008 for us to finally say “enough to inertia “), but apparently not obfuscated. So I can get it all out, if I understand the conceptual nature of repositories in general deeply enough to translate it all by hand. Which I don’t. Oh, so happy!
@Someone stuck with Surround
If you do dive into that, _Document and Publish Your Findings_. Your work today could save someone tomorrow.
Another variant is “Software California”, by way of analogy to “Hotel California”:
“You can check out any time you like but you(r data) may never leave.”
Hmm… Monotone (http://www.monotone.ca/) is a distributed version control system which stores local revision history in SQLite database. It was considered for version control for Linux kernel after “BitKeeper fiasco”, but it was considered too slow by Linus Torvalds (who performed evaluation of various open source version control systems). Maybe it was Monotone’s emphasis on correctness over optimisation (it was before any optimization work), maybe it was (allegedly) performance bug that hit this specific version… but it might be use of database for storage.
If I remember correctly Veracity (http://veracity-scm.com), also an open-source (Apache license) distributed version control system from SourceGear either can use database as repository storage as an option, or uses database for storage. As a historical note the original development of Subversion used the Berkeley DB (key/value database) for repository storage, but it was replaced (though I think it remains as an option) by FSFS filesystem-based backend, because of problems with Berkeley DB usage.
I think you could have stored the GNU Bazaar branches in a database via creating appropriately crafted repository storage plugin… but what for ;-)
Better to write a tool that translates it to fast-export stream (but putting separate projects in individual repositories).
I don’t know what is the conceptual model of repositories in Surround SCM, but you can find Git conceptual top-level model of repository in “The Git Parable“.
BTW. you can try to ask for help on StackOverflow (there is even [surroundscm] tag…).
@Someone:
“… but apparently not obfuscated. So I can get it all out…”
Shush! You’ll give them ideas.
BTW, ExpressPCB does this same thing. It’s a reasonably nice business model — simple little drawing package, press a couple of buttons, and wham! PCBs show up in the mail a few days later.
It’s a shame they are so paranoid they feel they have to encrypt their files so that you don’t use their software with anybody else’s service. It makes it a PITA to diff your designs.
The tar.gz lacks gram.y and lex.l that prevents this from being built.
BTW. `find . -name ‘*,v’ -print | cvs-fast-export -k` is not safe if filenames contain embedded newlines (and perhaps also leading and/or trailing whitespace), but I guess CVS would fail on such filenames too (CVS/Entries)…
Eric,
I have really admired the massive contributions you have made for the open source movement and am equally impressed with your ideological views – the reason of course being that they parallel mine!
Given that you are in favor of widespread gun ownership I was wondering if you were aware of the following project?
printablegun.com
>Given that you are in favor of widespread gun ownership I was wondering if you were aware of the following project?
I contacted the printablegun folks and was promptly quizzed about licensing. So yes, I’m helping.
>So yes, I’m helping.
Although I do not (yet) own a firearm, Thank You for your work.
— Foo Quuxman
I am still astounded that open source devs have been able to produce so many
powerful tools. Eric, thank you so much for all of your work. Your writing
is a big help for me and many others who have been lucky enough to
connect with it. You have intensity and style that provides an introduction
to a complex field and becomes something akin to an initiation.
When firing up a text editor to modify files such as bashrc, muttrc, and so on
I have been training myself to use version control. I’m grateful for the motivation
to learn useful new skills.
Very glad you can help in any way ESR!
The DefCad IRC is always somewhat active with at least a few users and the wikiweps.org site will start accepting new contributors on Monday.
An entire open source gunsmithing movement is already starting to brew from this, and I can see so many parallels with your advice on open source development and what the DefDist project is starting to become.
@Patrick, and that is why you use geda: http://www.geda-project.org/