Emacs git conversion is done

Finally. After ten months of work, it’s done. Emacs is fully converted to git. You can clone from git://git.sv.gnu.org/emacs.git and if you have commit rights you can push to it and the changes will stick. The bzr repo is still up but only as an archive.

Technically, this was reposurgeon’s finest hour. I’ve never done a conversion this big and messy before, as I noted in Ugliest…repository…conversion…ever. I had to write major new features to handle the job. I guess the most obvious of these is the macro facility.

I don’t expect to have to do one this difficult again. I fervently hope not to have to do one this difficult again.

As I wrote in Dragging Emacs Forward, my hope is this will let some light and fresh air into Emacs development. New talent, new ideas, revitalizing energy.

Happy hacking, everyone!

59 thoughts on “Emacs git conversion is done

  1. Off topic: Microsoft, the sole remaining not very evil major software company (Google and Apple having gone full NSA/Politically correct) has just open sourced its DOTNET stack, allowing C# to run on android, linux, and Windows. So C# is now free as in free speech, and the excellent development environment for C# is now free as in free beer.

    C# uses native UI widgets, which requires some UI customization for each environment.

  2. Awesome work! I’m going to clone the repo to my workstation right now. Who knows, I might even find some time to hack on it!

  3. Continuing on the Churchill riff (and having no other point or relevance, so flush this post if you feel the need – it may be somewhat hyperbolic and is definitely taking the piss):

    “The gratitude of every hacker in our communities, on our Internet, except in the abodes of the guilty, goes out to their intrepid fellows who, undaunted by odds, unwearied in their constant challenge and danger to their sanity, are turning the tide of software by their prowess and by their devotion unstinting. Never before, in the field of version control, was so much owed by so many to so few. “

    • >it may be somewhat hyperbolic and is definitely taking the piss

      I’d laugh, but there was in fact a significant amount of blood, toil, tears, and sweat involved.

  4. Will the mirror at https://github.com/emacs-mirror/emacs be updated as well? Or has this already happened; and how could I tell?

    Looking at https://github.com/emacs-mirror/emacs/commits/master?page=3806 (!) I see that the root commit comment does not have the note suggested in http://www.catb.org/~esr/reposurgeon/reposurgeon.html#style so the Github mirror currently points to the old conversion. Then again, the oldest commit visible from http://git.savannah.gnu.org/cgit/emacs.git/log/?ofs=118080 also does not have this note.

    • >Ummm… blood?

      I guarantee you that if you have to deal with emacs-devel for any length of time, you will feel bloodied. Fortunately, I am unbowed.

  5. Congratulations! BTW, you now have a trilogy of Emacs’-conversion-related entries. :D

    >You can clone from git://git.sv.gnu.org/emacs.git

    Hmm. That’s just one location. Does that mean the trunk/emacs-24 dichotomy is no more?

    Also: did you manage to convince the other devs to replace Texinfo with AsciiDoc? (If not, maybe they’d be more open to the idea of using Org-mode’s own lightweight markup language?)

    • >Does that mean the trunk/emacs-24 dichotomy is no more?

      git isn’t like bzr. When you clone it, you get all branches.

      >Also: did you manage to convince the other devs to replace Texinfo with AsciiDoc?

      RMS has pretty much bought in. To seal the deal I need to write something that can do Texinfo -> asciidoc translation. Doesn’t have to be perfect, but must reduce manual touch-ups required to a manageable amount of work.

  6. Eric, where is the reposturgon script you created for this? That’s what I’m interested in; I’d like to compare it to one I did a while back (way back, at the dawn of time before you added macros).

  7. Pandoc can write AsciiDoc and Texinfo. It does not read Texinfo, though. You could write a Texinfo reader (in Haskell :-) ) or convert it to one of the formats that Pandoc understands.

  8. Well, I always say it’s not a project until you draw blood…though usually I’m referring to busted knuckles and the like.

    RMS bought into replacing Texinfo? Does that mean we can finally drive a garlic-soaked stake through the heart of that crawling horror?

  9. Wow, congratulations :) I’m cloning it now.

    Jay: Maybe it’s just me, but I still see value in the info browsing system, but you only need AsciiDoc to be capable of rendering to that format, I personally welcome the move to AsciiDoc as a source format.

  10. I wonder if there would be one lightweight markup language that would win the mindshare war. AFAIK most popular are various versions of Markdown (with Common Mark as a emerging standarized standard) for web and AsciiDoc for documentation (because of rendering to troff i.e. manpages, and to info, perhaps also because it is extensible without altering standard).

    On the other hand Perl (and CPAN) continues to use its own POD format, so perhaps GNU Emacs and Emacs Lisp libraries could go to using Org-Mode…

  11. @all What would you count as the most unexpected, weirdest thing to use EMACS for? I raise: musical performance through live-coding a modern LISP dialect (Clojure) that generates whole songs. http://meta-ex.com/ Code example (the whole idea of open-sourcing meta-songs through Git just adds to the weirdness, in a good way): https://github.com/meta-ex/ignite/blob/master/src/meta_ex/piano.clj

    (… and I am afraid now I am responsible for destroying hours of productivity for some of you at least. Sorry.)

  12. Amazon had a customer service app that was written in Emacs (wrapping some core bits in C). Eventually it was retired and probably replaced with some enterprisey Java thing, but many of their customer service reps still wondered even years on, “hey, what happened to Emacs?”

    Yegge wrote about it.

  13. > After ten months of work, it’s done.

    >… there was in fact a significant amount of blood, toil, tears, and sweat involved.

    Ah. About a month late then?

  14. RMS bought into replacing Texinfo? Does that mean we can finally drive a garlic-soaked stake through the heart of that crawling horror?

    God I hope so. The fact that Debian frequently has a stub man page for various system tools that says “see the Info page” makes me want to punch babies.

    So, for that matter, does using Info.

  15. God I hope so. The fact that Debian frequently has a stub man page for various system tools that says “see the Info page” makes me want to punch babies.

    I only partially agree. man pages are not good for entire books, which many projects attempt to treat them as (see ffmpeg and mplayer. the man pages are super long and super not-nice-to-read). man pages need to be short (the reposurgeon one is teetering on that edge), but not leaving you scratching your head (like many of the ones generated via help2man, which is where those stub ones come from typically).

    man is good for getting a quick overview of how to invoke and use a program. Info is good for writing and reading entire books. Personally I think that there’s plenty of room for both.

    So, for that matter, does using Info.

    I find that Emacs makes quite an excellent info browser :)

  16. Yeah, Mike, but using EMACS makes me want to punch babies, too, despite Eric’s efforts in helping me make the transition.

  17. > What would you count as the most unexpected, weirdest thing to use EMACS for?

    I second the use of emacs-live and Overtone to live-code music. Although I can’t help but quote Rich Hickey: “Emacs is one of the last things you want to have around when you’re making music. You need Emacs for making music like you need it for having sex. And for the uber-geeks in the audience, just to be specific, that is not at all.

  18. Jay, hopefully the influx of fresh developer blood that the git conversion purports to kickstart will bring Emacs to the level attained by SublimeText in terms of design and modern editing workflow. (Including making cua-mode the default.) Then it will perhaps become more palatable.

  19. Microsoft, the sole remaining not very evil major software company (Google and Apple having gone full NSA/Politically correct)

    Yes of course, that would be the Microsoft that has never collaborated with NSA type organizations in the past, or embedded keys for the CIA into their software. Right?

    Do you even bother to read what you write before clicking ‘post’?

  20. Yeah, Mike, but using EMACS makes me want to punch babies, too, despite Eric’s efforts in helping me make the transition.

    Fair enough, but I think the main point here is that the software used to read info documents sucks, rather than the entire thing sucking. It is possible to get texinfo rendered out to HTML, or even full-fledged printed books… the info command and Emacs are by far not the only ways to read them.

    I think the primary benefit to switching to AsciiDoc is that you needn’t worry about writing directly in texinfo anymore. AsciiDoc can be utilized to write real man pages (rather than help2man generated stuff), and it should be possible to use to write the full-bledge Emacs manual and still convert out to texinfo so existing tools can use it. In addition to a nicer way of getting to HTML and PDF and anything else.

  21. Jeff, my problems with EMACS are all related to the pervasiveness of the default keybindings. Even when I tried switching it to use the WordStar keybindings, that only worked in the editor, nowhere else. All that did was raise the frustration factor.

  22. There is no reason why any program should not respond to the CUA keybindings out of the box. And no reason why they should support X11’s weird middle-click-paste semantics.

    When Wayland drives the last nail into X’s coffin, maybe we won’t have to deal with these UX fails. Meanwhile, despite the best efforts of people like Eric (not to mention RMS), more hackers are *biying* their text editors…

  23. There is no reason why any program should not respond to the CUA keybindings out of the box

    Better tell Apple that; their programs don’t. Just try to use CUA cut, copy, and paste keybindings in, say TextEdit.app.

  24. Better tell Apple that; their programs don’t. Just try to use CUA cut, copy, and paste keybindings in, say TextEdit.app.

    The Mac has its own, consistent set of UI guidelines which imho are superior to the CUA, not the least of the reasons why being the use of the Command key. The fact that virtually all program commands are Command key chords frees up Ctrl for sending actual control codes. ISTR that cua-mode on Emacs for Mac maps the Command key where Ctrl would be normally used, so Command-{X,C,V} etc. all work like they should.

    It will be extremely difficult to get the Command key equivalent (Super/Windows) used for this purpose under Linux, so the CUA and Windows 95 key bindings are the closest thing we have to a universally accepted standard.

  25. Jeff, I’m not talking about the minor Command/Control difference when I say Apple doesn’t support CUA keybindings. Emacs’s so-called “cua-mode” doesn’t support any CUA keybindings, either. The CUA keybindings for cut, copy, and paste (as seen in XEmacs, Windows, Java, CDE, GNOME, KDE, and OS/2) are Shift+Del, Ctrl+Ins, and Shift+Ins.

  26. > And no reason why they should support X11’s weird middle-click-paste semantics.

    I *like* X11’s weird middle-click-paste semantics. Things that you do often (copy and paste) should be easy and fast. Copying text out of one window and pasting it somewhere else under X is fast and efficient. Highlight, middle click, highlight, middle click etc. And synthesizing it by using “both” buttons works just as well. And yes, you can highlight, c-c, back to the mouse, click, c-x, and etc., but it’s a LOT slower.

    iTerm for the Mac gets the next best thing where you can highlight something in the terminal window and it puts it in a temporary cut buffer.

  27. “And yes, you can highlight, c-c, back to the mouse, click, c-x, and etc., but it’s a LOT slower. ”

    The keys in question are on the left hand – most people use the mouse with their right hand. And you lose efficiency in being unable to select text to delete, or to replace with the pasted text, without losing the copied text.

  28. > There is no reason why any program should not respond to the CUA keybindings out of the box.

    Having pre-existing actions defined for those keys from before CUA became relevant to them – especially if you’re talking about ctrl-XCV rather than the older ctrl/shift-insert shift-delete ones – is a reason.

  29. >> “And yes, you can highlight, c-c, back to the mouse, click, c-x, and etc., but it’s a LOT slower. ”

    > The keys in question are on the left hand – most people use the mouse with their right hand. And you lose efficiency in being unable to select text to delete, or to replace with the pasted text, without losing the copied text.

    X11 has two selection buffers… which on one hand side is nice, on the other can be kind of annoying.

  30. > X11 has two selection buffers… which on one hand side is nice, on the other can be kind of annoying.

    I don’t believe the secondary selection is actually part of the standard “select/middle-click-paste” paradigm or used by most programs that implement it. Various behavior has been proposed for it, but I don’t know if any of it can be relied on to actually exist consistently across most programs, if it has even been implemented at all.

  31. I don’t know if it is secondary selection buffer, or clipboard. Usually one is used for selection and middle-click, the other for Ctrl-X / Ctrl-C and Ctrl-V (or “paste” menu in GNU Emacs)

  32. > I don’t know if it is secondary selection buffer, or clipboard.

    Since we were talking, specifically, about William O’Blivion’s desire to _not_ use ctrl-c/ctrl-v or the clipboard, I assumed you meant the secondary selection and some way to do the tasks I mentioned by middle click.

  33. @ ESR: “How reposurgeon wins” needs an update: at the bottom, it describes the Emacs conversion as “in progress”. And while you’re at it, why not add NetBSD to the list? (That one is in progress, right?)
    Oh, and BTW: Reposurgeon’s main page reads “A tool for editing version-control repository history reposurgeon enables…”. There should be a comma between “history” and “reposurgeon”, I believe.

    • >“How reposurgeon wins” needs an update: at the bottom, it describes the Emacs conversion as “in progress”.

      Done yesterday, just not pushed yet.

      >And while you’re at it, why not add NetBSD to the list?

      I won’t do that until I get formal buy-in from the NetBSD maintainers. Right now we’re in an early, exploratory phase.

  34. Why look only to PANDOC or MARKDOWN?

    There is another alternative – reStructuredText. pandoc and markdown invented as is, for RST there is http://docutils.sourceforge.net/docs/dev/rst/alternatives.html

    Despite MarkDown popularity (StackOwerflow + GitHub) it lack fundamental feature – extensibility, even in CommonMak spec. RST has standard syntax for embedding any kind of text data (dot/svg/tex/math/etc).

    RST specs http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html now lighter then CommonMark http://spec.commonmark.org/0.12

    Doday RST have hipster https://readthedocs.org/ and widly used http://sphinx-doc.org/ (which allow embedding offline text search in JS index).

    • >Why look only to PANDOC or MARKDOWN?

      I’m not looking at either. I hate Markdown for being an unstandardized, messy, inferior imitation of what asciidoc gets right.

  35. About reStructuredText – does it support manpage (troff) and info output out-of-the box?

    BTW. AsciiDoc is also extensible… though I don’t think that it supports TOC.

  36. AsciiDoc can generate a table of contents perfectly fine. There is no syntax, it just generates it automatically. What kind of syntax are you looking for, anyway?

    I’ve personally been a fan of AsciiDoc for years — I found a commit where I converted a README file over to it in late 2008, and I had probably been using it for much longer than that. I don’t really recall specifics of RST I don’t care for, although I remember having evaluated it at one point and “messy” is a term I would use for it as well, and re-skimming the specifications I still don’t regret my preference.

  37. Jakub NarebskiAbout reStructuredText – does it support manpage (troff) and info output out-of-the box?

    rst2man(1) comes with official package.

    There are no official converter for texinfo. I think because lack of interest. Most advanced texinfo data may be presented via regular syntax for extensions and tagging in RST. Other formats just provide syntax for sections, bolt, emphasis and links.

    Writing own write is easy, check: https://pypi.python.org/pypi/rst2texinfo

    I didn’t check how pandoc output is good.

Leave a Reply

Your email address will not be published. Required fields are marked *