Looking for C-to-anything transpilers

I’m looking for languages that have three properties:

(1) Must have weak memory safety. The language is permitted to crash on an out -of-bounds array reference or null pointer, but may not corrupt or overwrite memory as a result.

(2) Must have a transpiler from C that produces human-readable, maintainable code that preserves (non-perverse) comments. The transpiler is allowed to not do a 100% job, but it must be the case that (a) the parts it does translate are correct, and (b) the amount of hand-fixup required to get to complete translation is small.

(3) Must not be Go, Rust, Ada, or Nim. I already know about these languages and their transpilers.

67 comments

  1. “Must have a transpiler from C that produces human-readable, maintainable code that preserves (non-perverse) comments”.

    The original C either has that property or does not. If the transpiler could correct that, even a C to C transpiler would be welcome.

    Or to put it more succinctly, Garbage in, Gospel out, the GIGO transpiler!

  2. The weak memory safety is an impedance problem. You are either considering the hardware model, the processor, memory, MMU, etc. or you are considering the higher level.

    The higher the level, the more things are completly abstracted (obfuscated, hidden). Either there are physical ram chips that at worst there will be swapping, or there is some abstract blob of memory with a handle or pointer. You can’t have both, though the latter will always translate to the former.

    C is the highest level you can have and preserve the feel of the underlying hardware. Java is a VM. Go is Java with a C like syntax (as is .Net which allows insane people to safely use sharp, C# objects in their padded cells).

    You either have perfect pitch or need a pitch corrector.

  3. Move Zig, for great justice. At a glance, this looks like one of the top candidates.

    When compiling in its Debug or ReleaseSafe modes, Zig meets the “weak memory safety” criterion and will check all memory accesses, crashing upon failure. It also can translate C to Zig with the ‘zig translate-c’ subcommand. This may be incomplete, but it’s complete enough to self-host!

    1. Who are you and what have you done with Jeff Read? You didn’t come up with some obscure bondage and discipline language…

      Zig looks interesting, to say the least. They say all the right things. I do wonder how it works in the real world.

      1. Ada and Rust are hardly obscure (though they were on Eric’s list of languages he’d already considered and didn’t meet his criteria, hence why I didn’t bring them up).

        And I think decades of buffer overruns, use-after-free errors, and undefined behavior have shown that humans just can’t be trusted with C or C-like semantics. Memory safety is table stakes for a new language (one reason why I don’t think Zig will find much use outside a few interesting niches) — and if you don’t want the overhead of GC, you can’t have memory safety without constraining the way memory (or any resource) is accessed and used. Don’t think of it as bondage and discipline — think of it as liberation. You’re free to solve the problem at hand without having to worry about entire classes of errors because the compiler will catch them for you. You can also think of it as a safety net if you like.

        Honestly, I wish I could mention Gambit Scheme here, but its compilation so far only goes one way — from Scheme to C — and not the reverse. Some of the old Lisp Machines had C compilers which, while they didn’t output Lisp, their output did link seamlessly with Lisp and had memory safety guarantees due to the underlying architecture. C arrays and pointers, for instance, were realized as “fat pointers” that held a reference to an object and an offset or index within that object. Memory accesses outside the bounds of a valid object would result in a hardware trap.

        As cool as that was, it doesn’t generalize to today’s general-purpose CPUs. Maybe I oughta contact Marc Feeley and ask about the possibility of a C-to-Gambit compiler…

        1. “Don’t think of it as bondage and discipline — think of it as liberation.”

          “Slavery is freedom”, comrade?

          “You’re free to solve the problem at hand without having to worry about entire classes of errors because the compiler will catch them for you.”

          What you give up for that is the ability to solve any problem for which the solution does not fit neatly into the compiler writer’s idea of how his language should be used, and for that matter the flexibility to solve any problem in any way that does not fit into that idea.

          No compiler writer knows how to solve my problem better than I do, and anyone who thinks otherwise is full of prunes.

          1. What you give up for that is the ability to solve any problem for which the solution does not fit neatly into the compiler writer’s idea of how his language should be used, and for that matter the flexibility to solve any problem in any way that does not fit into that idea.

            Rust has ‘unsafe’ mode which allows you to use C-like semantics at leisure. Even Ada has escape ropes from its sometimes confining rules. For example, you can have aliased pointers in Ada, but you have to explicitly flag them as such.

            The idea is to not use these tools willy-nilly. Seeing the escape-rope keywords should be a big red here-be-dragons sign to anyone maintaining the code.

            Again, the fact that the existence of a sufficiently smart C coder, who writes in such a way as to guarantee the absence of use-after-frees, buffer overruns, undefined behavior, et-weary-cetera, has yet to be proven suggests a need for such rules and strictures. The closest we’ve come in the C world to a codebase free of such bugs has been where rules you might consider stifling have been introduced as part of an organization’s coding standards and checked with static analysis tools. See the DoD’s standards, etc.

          2. Not to start up a war about type strength in language design on this recommendation seeking post, but I think there is a non-insidious reason to think of a strict typechecker as liberation.

            A type (at least a well-designed one) represents a concept in the problem domain, and a strict compiler tries to make sure the code is interacting with instantiations of that concept correctly. A weak typing discipline assumes the code will do the right thing and so doesn’t bother checking. Because the compiler doesn’t check, the programmer has to enforce correct interactions.

            I started seriously programming in Python, and I thought duck typing was the best thing since sliced bread. I despised learning Java and C++ because it felt unnecessarily restrictive. I could have accomplished the same thing in far fewer lines of code, but instead I have to define all these trivial proxy ideas just to get some code to run. Then I started writing significantly sized programs in Python, more than a few hundred lines, and I ended up making all those seemingly unnecessary restrictions anyways, just to keep my code clear. But in this case, the interpreter wouldn’t help me, because it assumed I was correct.

            I now write C++ for a living, and it’s certainly not perfect. There are plenty of other languages I’d prefer, but it’s workable. And the fact that the compiler will tell me when I’m doing something stupid is liberating. The fact that I need to spell out the concepts I’m trying to get the code to model means the compiler can come back to me and say “Hey, you said foo was supposed to be bar, but you’re using it here like a baz” means I don’t try running code that would crash or have incorrect behavior. I say what I mean when I define something, instead of checking to verify what I don’t mean every time it’s used.

            No type system yet developed can model all of human thought, and it’s questionable whether something like that could even exist. But type systems that encompass a lot of it are very useful, as the compiler then acts as more of a proofreader than blind translator. That’s the liberation I think Jeff is describing, freedom from the minutiae of easily check-able correctness.

          3. >> “Don’t think of it as bondage and discipline — think of it as liberation.”

            > “Slavery is freedom”, comrade?

            Jeff put it badly (his worldview certainly leads to that), but he’s right.

            A language is a tool for solving a problem. A roofer will walk around with a single hammer when he’s re-shingling a roof, but you’ll note that when they’re taking *off* the shingles they’re not using the claw part of the hammer, they’re using a dedicated tool (prybar usually) and a shovel. Three completely separate tools to get a simple job done.

            If you’re trying to solve a problem that uses and disposes of LOTS of memory you may want a language that *doesn’t require you to think about that*. It is *literally* liberating in that it liberates you from having to think about those kinds of things.

            If you need to solve a problem where that safety gets in the way, the *you need a different tool*, one that lets you think about the problems you want to think about, and not the problems that you don’t.

            Consider the “infrastructure” you need to set up in Python3 to execute an system binary and read the results into a variable.
            import subprocess

            proc = subprocess.Popen(‘ping -c 100 google.com’, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
            tmp = proc.stdout.read()

            Contrast that with:

            $output = `ping -c 100 google.com`

            On one hand not a lot different. On the other hand four times the LOC.

            The “liberation” really comes in being able to say “This is a tool for this job, and this is a tool for the other job.”

            All languages are niche languages. Some are niches are just bigger.

            1. Jeff put it badly (his worldview certainly leads to that), but he’s right.

              In the same breath you mock my “worldview” yet acknowledge my correctness. Mightn’t it be the case that my worldview in other domains has a lot to do with a principle that has shown itself in this domain? I leave generalizing it as an exercise.

    2. >At a glance, this looks like one of the top candidates.

      That didn’t last long. c2nim does the smart thing and interprets #include stdio as a directive to map C stdio entry point names to Nim library names. Zig translate-c just barfs on the include.

  4. Depending on your use case, V (github.com/vlang/v) is rather promising. It is basically Go, with a borrow checker, which then transpiles into either Javascript or safe C (similar to rpython -> clang). Unfortunately, the C->V translation isn’t working yet, at the speed they’re going, it’s probably a year or so out from general usability.

  5. Please don’t ban me – check out C++17. Don’t look at it as overdone, instead look at it like TIMTWTDI.

      1. TIMTOWTDI [pronounced Tim Toady]: There Is More Than One Way To Do It. Motto of Perl. (Compare Zen of Python: There should be one — and preferably only one — obvious way to do it.)

    1. >Please don’t ban me – check out C++17. Don’t look at it as overdone, instead look at it like TIMTWTDI.

      I won’t ban you. I’ll just laugh at you, fool, because you completely failed to understand the request.

      1. C++ probably strictly satisfies the criteria in the request. It’s mostly a superset of C (though the more C features you use from post-1989 versions, the more work you have to do to make it C++-compliant).

        We all have our reasons to consider C++ a legacy language in the like of COBOL, and the spirit of your request was definitely not leading toward it :)

        Am I reading things right that Nim seems to be the winner?

        1. There’s a large difference between C++ compliant and *modern* C++ compliant. If you’re using C++20 (I wouldn’t even use 17, as #include is still a mess), your code should have about as much in common with C as the Python or Perl inline C modules do. Your guarantees of safety go out the window the moment you include any non-trivial C.

          1. All right, but one of the criteria is an automated lifter that handles a large part of the job. Does such exist for C++{17,20}? Probably not, would be my guess.

            1. That was my point too. The original claim was that valid C89 is also valid C++, so the identity operator is a conversion from C89 to C++. While that is true, it does not give you C++ you would actually want to use. In fact, if you have all the sane warning flags on, any non-trivial C89 code will spit out a pile of warnings if you try to compile it in C++20 mode.

              1. > The original claim was that valid C89 is also valid C++

                With a few exceptions, e.g. (roughly in decreasing order of how likely actual C89 programs are to use them) you can’t implicitly convert void* to other pointers, call main() from within your program, rely on the type of character constants being int, or start a /*comment*/ immediately after the division operator (though the last one you can’t in C99 and later either).

                1. (Though I guess a tool adding casts to all implicit conversions from void *, spaces between “/” and “/*”, etc. wouldn’t quite satisfy the spirit of ESR’s question…)

                  1. Since the safe subset of modern C++ nearly completely avoids pointers… no just adding reinterpret_cast everywhere would *not* really help…

        2. >C++ probably strictly satisfies the criteria in the request.

          No. It fails weak memory safety.

          >Am I reading things right that Nim seems to be the winner?

          So far, but it’s very early days.

          1. Replace all `char*` with `Ptr` or `Span`, this is some custom types that do always check access, conversion could probably by done by some clang plugin (refactor to common C/C++ code and then to pure C++ code).

            Or hire some student and give him 12h and he will manually do it, same thing like Jason Turner did for doom1:
            https://www.youtube.com/watch?v=Nand3PEV1p4
            Funny thing is that C++ result binary is faster and smaller (<1% but still) than one from original C

        3. It’s interesting that I never heard of Nim until it was mentioned here, even though I am generally interested in the idea of getting the three layers closer together. The three layers are systems programming, which tends to happen in C or C++ and application programming, which these days tend to have something like a LAMP/LAMR backend and a browser-based JavaScript frontend. Not ideal, but that is what it is.

          So this why I got interested when it was mentioned here that Rust has a web backend framework. Bridges the systems and backend world. But still, what about the frontend. The obvious way to get the backend and frontend closer together is to use server-side JavaScript.

          So I look it up now and it says Nim compiles to C, C++ and JavaScript. Neat. Really the whole stack. Except one little problem: this has always been the obvious and easy way to do it. Yet, the langs that won out in the long run all have interpreters, bytecode compilers or native compilers. They do not just generate code. Hence, there must be some caveat there…

    1. >Pascal?

      Please do not suggest languages without giving a link to a specific open-source implementation and a transpiler to it from C.

      Future comments this useless are subject to being deleted as noise.

  6. Your Beast, having completed the GCC git migration, is sitting idle and craves new workloads? Perhaps lifting whole C projects to a higher-level language?

    1. >Your Beast, having completed the GCC git migration, is sitting idle and craves new workloads?

      That was the situation until this morning, when its primary SSD died.

      Now attempting disaster recovery.

      1. That was the situation until this morning, when its primary SSD died.

        Having been kibbitzing with Eric on another channel, I can report that the consensus now is that the power supply of the GBoM has died. A new one is being shipped, and Eric’s local hardware guru will be working with him on Thursday to complete the repairs.

        With any kind of luck (makes warding sigils against Murphy, Finagle, and Eris) he didn’t lose anything on his SSD or motherboard.

        1. >the consensus now is that the power supply of the GBoM has died.

          Yes. The Beast went through the following stages within two hours:

          1. SSD mysteriously write-locked.

          2. Failure to fully reboot.

          3. Failure to get to the BIOS splash screen.

          4. Total brickitude – front-panel light does not light.

          Ladies and nerds, that has all the earmarks of a PSU delivering decreasing undervoltage before shuffling off this mortal coil entirely.

          The SSD may have been designed to protect its data by refusing write requests when the rail voltage falls out of spec – no point in refusing reads, of course, as they cannot leave your file system structures in an inconsistent state wherever they’re interrupted.

          Wish us luck on Thursday.

          1. Nah, the coils are probably fine. PSUs usually shuffle off this mortal capacitor. If it shuffled off this mortal FET you’d probably notice, ‘cos magic smoke smells awful.

            Good luck for Thursday.

          2. Zoinks! Remember to click your heels together three times and spread your ass cheeks when you hit that power button ;)

            Yes, physiologically-speaking that means you will need to hit the power button with your nose….

            Please post a photo

          3. Word of warning… I learned the hard way that the fancy modular PSU cables are NOT standard at the PSU end of things…

          4. > no point in refusing reads, of course, as they cannot leave your file system structures in an inconsistent state wherever they’re interrupted.

            That’s a naive view of SSD firmware. They do internal writes when the host is idle, their PIP features are mostly untested by vendors (anything but “stable full voltage” and “rapidly transition from fully on to zero” is probably untested), they are strongly dependent on internal metadata integrity to make FTL work (so even a single bit loss is very bad), and they have strictly lowest-bidder CPU and memory subsystems (write caches in DRAM with no ECC). It’s something of a technical achievement to stop them trashing lots of data randomly when their power supply browns out; hence, tolerance for power failures is a feature implemented with sensors and firmware, rather than the natural state of every SSD. Contrast with HDD, where the amount of damage bad power can do is limited by how much of the disk surface the mechanical heads can reach without power coming in.

            I’d be worried about the SSD read-only failure apparently happening before the other failures. My guess is that at event #1, the SSD trashed itself, detected it had trashed itself a short time later, and flipped the “permanently read-only for forensic recovery after a catastrophic failure” bit. Usually when there are power issues or other temporary failures, drives drop off the bus completely until power is reset, then they behave normally. They will only enumerate on the bus and present a read-only storage medium after an unrecoverable failure.

            Of course, you have decades of experience living with data storage hardware, therefore you have up-to-date backups at all times, so any single-drive failure is but a minor inconvenience…?

        2. I was a PC repairman in a past life, and for what it’s worth, the drive is probably OK. A blown PSU will not-infrequently take the motherboard with it, but very rarely kills drives. I saw it happen only once, across ~3 years and a few hundred dead PSUs.

          Data loss from the power failure is a little more likely, but not nearly as common as it used to be.

          (that said, you *do* have backups, right?)

  7. “weak memory safety”?
    You seem to mean “not as fast as a language with no memory safety, and not as safe as a language with strong memory safety”… What kind of programming language even are in that niche? How did they end up there?

    “Must not be Go, Rust, Ada, or Nim”?
    I don’t think they even are “weak memory safe” to begin with?

    1. He clarifies what is meant in the next sentence. It’s okay if the out of bounds handler simply calls abort(), but there must *be* an out of bounds handler.

      Rust is strongly memory safe if there are no “unsafe” blocks, and not memory safe otherwise, so I guess that’s sorta weakly memory safe… I don’t know on the other three.

      As for languages in that niche, the subset of C++ which compiles without warnings when all the sane warnings are enabled and the standard is set to C++17 or later, and which does not interoperate with any C libraries, or other libraries using C arrays, qualifies. The issue there is that there is very little you can actually *do* with that subset, as you can’t make any system calls, or at least any interesting ones.

      1. Can C++17 detect or prevent use of an invalidated iterator? That’s pretty much the only issue I run into with C++ code these days.

        Does any C++ standard, present or future, prevent a project from including its own C library?

        1. There’s some work with clang static analysis to detect possibly invalidated pointer use in existing code, there’s also asan to try to detect it at runtime. There’s also the Modern C++ Core Guidelines, which provides a header-only library that can catch at least some bad iterator uses.

          That said, dangling iterators *usually* indicate an error in your data structure, and are quite easily avoided. First choice is to not keep them around long enough for them to get invalidated. This covers most of the cases. Second choice is to store an offset from the start of the container, and use the at() function to bounds check access. This covers essentially every other use, on the odd occasion you have to remember where you stopped mid iteration or similar. In the last case, where you cannot feasibly go from offset to iterator repeatedly, you can store your iterators someplace smart, and make your move assignment and move constructor on the container invalidate them explicitly if they get triggered. I’ve never actually needed to do that, but if you are working with a C library which wants a pointer to a callback, you need to detect moves and update the pointer (epoll, for example).

          The only standard to prevent including a library is a project’s own standard. I do *wish* there were a way to refuse to compile a file with calls by pointer and to refuse to link against a library expecting them, but the most you can get is some warnings for C-isms (there is, for example, a warning for making a pointer from an array). Practically speaking, until someone comes up with an alternative to libc with a modern C++ interface, you *must* link against libc for some basic things (like I/O).

  8. You might want to take a look at eg emscripten. Ik compiles C to WASM, and while I’m not sure it preserves comments, it does seem to meet the other criteria.
    It’s even the case that WASM has a textual format based on s-expressions, so it’s extremely lisp-like and could probably be read by a (somewhat modified) lisp reader.

  9. I think Eiffel meets your criteria.

    Requirement 1: Eiffel’s void-safety property prevents what would in C be called dereferencing a null pointer. Out-of-bounds safety can be accomplished by turning on precondition checking for array operations.

    Requirement 2: C2Eif is a C to Eiffel translator that claims, “The translation is completely automatic and supports the entire C language (ANSI, as well as many GNU C Compiler extensions, through CIL) as used in practice, including its usage of native system libraries and inlined assembly code… The produced Eiffel code is functionally equivalent to the original C code, and takes advantage of some of Eiffel’s object-oriented features to produce readable, maintainable, and easy-to-debug translations.”

    Requirement 3: Eiffel is not closely related to Go, Rust, or Nim. It is influenced by, but distinct from, Ada.

    1. >I think Eiffel meets your criteria.

      I agree, but this project is on hold until I can recover from my hardware disaster.

    2. Unfortunately, there’s a 4th requirement I took so much for granted that I failed to state it.

      Language must have an open-source implementation. If there is such a thing for Eiffel I have been unable to find it.

  10. Johnathan Blow is working on JAI supposed to be a modernized c / c++ just focused on efficient hardware use. If you tweet at him he might add you to the beta. I’m sure he’d appreciate any help.

    1. >Johnathan Blow is working on JAI supposed to be a modernized c / c++ just focused on efficient hardware use. If you tweet at him he might add you to the beta. I’m sure he’d appreciate any help.

      Alas, the purpose behind my question requires a tool that is production-ready rather than a beta.

    1. >https://github.com/mono/CppSharp generates bridges from C (and C++) to C#. It’s not strictly a transpiler since one needs the original libraries too but thus it has the advantage of also working with just binary code.

      This would be interesting if it traveled with a C-to-C++ transpiler. Without it, not.

      1. https://clang.llvm.org/docs/RefactoringEngine.html
        Or
        https://www.kdab.com/clang-tidy-part-1-modernize-source-code-using-c11c14/

        One or another can automatically fix “style errors” in C++, and probably same for C
        This mean you can convert C to C/C++ subset and this can be push to this C#
        Or keep it in C++ but add lot of wrappers (using this tool) that always check for correction.

        As far I know Google use tools like that to refactor they whole code base:
        https://www.youtube.com/watch?v=ZpvvmvITOrk

    1. >What’s the over-arching purpose here?

      Identify a language to which lifting large volumes of C infrastructure code is practical. The objective is to get to the land of memory safety in order to foreclose large classes of security issues.

      -fsanitize-address is a cute stunt, but the overhead is so high that it’s not acceptable for some of the deployments I have in mind. Think for example GPSD targeted to a low-end smartphone.

      1. If you can afford the performance penalty of using asan in release mode, you can afford the performance penalty of writing in Go or some similar language.

        The typical recommendation is to enable asan in your test suite, and get 100% code coverage. It catches probably 90% of memory safety bugs in C programs, but misses any that crop up in the wild but not with synthetic data. There’s also some linting tools that catch a pile of others (grepping for malloc and new should generally not find anything). All in all, it’s much better than 10 or 15 years ago, but still has a ways to go to be generally safe.

      2. I don’t know what the runtime overhead of the sanitizers is, but in combination with fuzzing they might winkle out some unlikely issues.

        If the code is correct, the question is moot…

        Anyway it would be surprising if you found a suitable language and transpiler, because I don’t think any of the likely suspects have taken this approach with effectively infinite resources including not blinking about the creation of a new language from scratch, but surprises are good.

      3. -fsanitize-address is a cute stunt, but the overhead is so high that it’s not acceptable for some of the deployments I have in mind. Think for example GPSD targeted to a low-end smartphone.

        The overhead will be unacceptably high for this use case with any GC language. As things stand, there is exactly one practical memory-safe language that does not transmute battery power into waste heat doing full-heap traversals looking for garbage to collect: Rust.

        If C2Rust is not good enough for you (it looks like it generates unsafe Rust in its current form), there’s an easy way to lift your C code base into Rust: Go to Hackernews and post “Show HN: Rewrite of [my_project] in Rust” with a link to the conversion’s GitHub repo (possibly pre-populated with C2Rust-generated code to start from). You’ll have eager zoomers looking to prove their mettle falling all over themselves to contribute.

      4. -fsanitize-address is a cute stunt, but the overhead is so high that it’s not acceptable for some of the deployments I have in mind. Think for example GPSD targeted to a low-end smartphone.

        From the documentation at https://github.com/google/sanitizers/wiki/AddressSanitizer

        “””
        ulimit -v
        The ulimit -v command makes little sense with ASan-ified binaries because ASan consumes 20 terabytes of virtual memory (plus a bit).
        “””
        20 terabytes….

        20 terabytes??!!!

        FAIL!!!

        1. The Linux kernel uses 16777216 TB of virtual memory, and nobody seems too bothered about that.

          The process memory map under ASAN is, obviously, sparse. Not sparse in a way that ulimit -v understands, but there’s a reason cgroups are recommended instead.

          Sure, O(log(20TB)) is maybe still a large number, especially when multiplied by a thousand processes on a smartphone, but it’s really not the big problem with ASAN. A much more pertinent problem is the “fast” doubling of all the memory access times. It’s only fast when it’s competing with the likes of valgrind.

          This is a tool you only want to use during development and testing, then turn off before shipping, especially on any device that lives on a battery. It’s a brute-force bug-finder, not a run-time memory safety feature. It can only handle stray pointers, UAF bugs, and pointer overruns (of sub-page-sized array objects). It can’t answer “should thread T running function F access pointer X+Y” questions at all.

        2. It seems that one of the ways that ASAN works is to spread the process’s memory objects out sparsely enough in the address space to where a pointer running off the end of one is unlikely to wind up in another, and 20 TiB is enough to do this for all but the most ridiculously huge of working sets. Only the pages that are actually used have physical memory allocated to them.

          Another way to do this would be to make all pointers fat and bounds-checked: express a C pointer as a pair consisting of a pointer to an object and an offset within that object. I realize that this isn’t feasible for all C code, but for user-mode code it is a good approach. It makes me pine for the Burroughs and Lisp Machine architectures though…

  11. I doubt that the transpiler you are looking for is possible as a general-purpose tool—not so much for technical reasons as for social reasons. If the developer is striving to produce a general-purpose tool, it is almost inevitable that he will make choices that put universality (handling all possible C programs) ahead of the maintainability of the output. Over time, the project will asymptotically approach github.com/andybalholm/leaven (my experiment with translating LLVM IR to working but absolutely unreadable [and not memory-safe] Go).

Leave a comment

Your email address will not be published. Required fields are marked *