Open Source — Can It Innovate?

There’s an argument commonly heard these days that open-source software is all very well for infrastructure or commodity software where the requirements are well-established, but that it can’t really innovate. I laugh when I hear this, because I remember when the common wisdom was exactly the opposite — that we hackers were great for exploratory, cutting-edge stuff but couldn’t deliver reliable product.

How quickly people forget. We built the World Wide Web, fer cripessakes! The original browser and the original webservers were built by a hacker at CERN, not in some closed-door corporate shop. Before that, years before we got Linux and our own T-shirts, people who would later identify their own behavior correctly as open-source hacking built the Internet. (I was part of the tail end of that process myself; sometime I’ll blog about how and why the domain-name gold-rush is arguably my fault.)

It seems to me that bringing the Internet and the World Wide Web into being ought to count as enough “innovation” for any one geological era. But it didn’t start or stop there. Nobody even conceived of cross-platform graphics engines before the X window system. The entire group of modern scripting languages descends from open-source Perl, and almost all draw critical strengths and innovative drive from the size and diversity of their open-source communities.

Even in user-interface design, much of the most innovative work going on today is in open source. Consider for example the Facades system. Or just the astonishing, eye-popping visual experimentalism of Compiz/Fusion under Linux.

It’s actually corporations who have trouble innovating. Innovation is too disruptive of established business models and practices; it’s risky, and it involves coping with those annoying prima donnas at the R&D lab. Consequently, even well-intentioned big companies like Xerox that are smart enough to fund real research centers like Xerox PARC often reject the truly groundbreaking ideas from their own researchers. Today you’d be extremely hard pressed to find any of the really cool ideas from Microsoft R&D being deployed in actual Microsoft products.

The process of innovation and deployment in open source is of course not friction-free, but it certainly looks that way when compared to the corporate world. One of my favorite current examples is the way Guido van Rossum and the Python community are gearing up to re-invent their language for its 3.0 release. Their “Python Enhancement Proposal” process for fostering and filtering novel ideas by individual contributors repays careful study; like the Internet RFC process (on which it’s clearly modeled) it produces a combination of innovative pace and successful deployment that even Bell Labs in its heyday could not have dreamed of sustaining.

Yet, somehow we still see earnest screeds like this one by Christophe de Dinechin:

What I’d like to see happen is genuine open-source innovation. But I’m
afraid this cannot happen, because real innovation requires a lot of
money, and corporations remain the best way to fund such innovation,
in general with high hopes to make even more money in return.

The easy, cheap reply would be to write the author off as a blithering idiot who has failed to notice that his entire environment has been drastically reshaped by open-source innovation, and the proof slaps him in the face every time he looks at a browser. But, in fact, I think he (and others like him) are not idiots; they are reasonably bright people making a couple of serious and identifiable errors in their reasoning about open source, closed source, and innovation.

Error the first: ignoring the present value of open-source innovations in the recent past when projecting the difference in expected returns between open-source and closed-source innovation strategies. This is what M’sieu de Dinechin is doing when he fails to notice that Tim Berners-Lee was a hacker operating in open source, and his successors mostly likewise.

Error the second: discounting innovations that are not user-visible and salable by a marketing department. OK, the latest piece of eye candy from Apple is very nice, but if you ask me how it compares to the present value of (say) the open-source BIND daemon, the answer is “no contest”; one just looks pretty, the other is fundamental to the entire frickin’ web-centered economy.

Error the third: ignoring work like Metisse/Facades because it isn’t yet deployed on enough machines to show up on a marketing survey. The problem here is that people like de Dinechin wind up erroneously taking the ability of corporations to sell incremental improvements into an established marketplace as their major proxy for measuring the ability to originate innovations in the first place. This makes their view of what constitutes ‘innovation’ nearsighted even where it’s not altogether blind.

Error the zeroth: confusing two issues, one of which is “which strategy globally maximizes innovation?” and the other one of which is “how do I, the hungry would-be innovator, get paid?” This is the big one; I’m numbering it Error Zero because I think it’s at the bottom of almost all the other systematic mistakes de Dinechin and people like him are prone to, including errors One through Three.

de Dinechin, and people like him, have a simple and linear model of how innovation works. Pay a bright guy like de Dinechin, stand back, and watch the brilliant stuff come out and change the world. In this model, if you don’t pay bright guys like de Dinechin, innovative stuff doesn’t come out because they’re too busy grinding out COBOL or something so they can eat — no world-changes, so sad.

This model is very appealing to people like de Dinechin, who have an understandably strong desire to be paid for being smart and creative. Heck, it appeals to me for exactly the same reason. Unfortunately, and unlike de Dinechin, I know that it is seriously false-to-fact.

I have a very different model of how innovation, at least in software, actually works. One of its premises can be expressed by what I shall now dub the Iron Law of Software R&D: If you are a programmer developing innovative software, the odds that you will be permitted to finish it and it will actually be deplayed are, other things being equal, inversely proportional to the product of your depth of innovation and your job security.

That is, the cushier your corporate sinecure is, the less likely it is that you will make a difference. The more innovative your software is, the less likely it is that you will actually be supported all the way to deployment.

The reason for this is dead simple. Corporations exist to mitigate investment risk. The large and more stable a corporation is, the more resistant it is to disruption in its practices and business model including the unvoidable short-term disruptions from what might be long-term innovative gain. Net-present-value accounting therefore almost always leads to the conclusion that innovation is a mistake.

“But what about Bell Labs?” I hear you sputter. Ah, yes, that archetype of the halcyon days of corporate research. Well, for one, Bell Labs is dead; pressure for short-term returns has made the kind of empire-building it represented effectively impossible today. And even in its heyday, Ken Thompson had to write Unix as an after-hours project on a piece of salvaged junk, then sneak Unix tapes out the back door to deploy it.

The Iron Law explains neatly why most of the research that came out of Xerox PARC was eventually deployed by other corporations, mostly startups with no preexisting business model in jeopardy. And why you get the most actual, deployed innovation in open source — because the people whose revenue streams we’re jeopardizing (if any) aren’t the same people who are funding us (if any).

85 thoughts on “Open Source — Can It Innovate?

  1. So I guess the question, rather than “can Open Source innovate”, is “can Open Source create (via innovation) anything that can’t be stolen and exploited more effectively by corporations?”

    After a few months of hobbyists freely bouncing ideas around, someone comes in, steals the idea, and closes it off.

  2. >can Open Source create (via innovation) anything that can’t be stolen and exploited more effectively by corporations?

    That’s two questions. Unfortunately, we can’t produce anything they can’t steal. Fortunately, their capacity for exploitation is limited by exactly the same kind of institutional problems that generally keep them from using their own R&D effectively.

    >steals the idea, and closes it off.

    Fortunately, the “closing it off” part is almost never possible.

  3. >Fortunately, the “closing it off” part is almost never possible.

    What about MS’s bid to take over HTTP with IE? I know it didn’t work, but it’s at least a workable strategy.

  4. I agree that open source does innovate at the back end very well. (I can count the times a Unix has suddenly crashed on me on one hand.) To a lot of people, especially nontechnical users, most end-user applications appear derivative of proprietary applications. An example would be a certain image-manipulation program that rhymes with “limp”.

    I can point to a few examples, such as Compiz/Fusion and the social-networking explosion (running mostly on scripting languages, open source databases, and Linux) and GNOME as examples of innovative open source end-user applications. But apart from those, it looks like open source developers are copying proprietary programs to a lot of people. And that is a serious problem. It’s why we occasionaly see screeds against “open source socialism”.

  5. > Nobody even conceived of cross-platform graphics engines before the X window system.

    Ah, except AT&T.. “The X Window System” wasn’t licensed under an “open source” license until 1986 when MIT decided to license X10R3 and later versions under what would become the MIT license. The first X Consortium release was in early 1988.

    Stephen Uhler was publishing papers on MGR in 1988. I don’t remember which was ‘first’, but they were at least close in time. (And yes, MGR became ‘open’ later.)

  6. The answer to “how to we prevent corporations from stealing our work” is simple, but would likely provoke a flamewar, especially in this neighborhood….

  7. humm.
    first off, i gotta say, i think you’ve misread what de Dinechin wrote/meant. or rather: have extrapolated way too much about his underlying attitude/beliefs from one point near the end of a wide-ranging post. a post apparently made conversationally rather than declaratively.

    secondly, on the whole, i agree with what you say here about corporations

    thirdly, on the whole, i agree with de Dinechin’s observation: i see far less major innovation in open source than in proprietary software.

     
    i do not see this as contradicting your points re corporations. i see almost no major innovation in corporation-driven software.*
    i do not see this as contradicting your points re open-source. i see much more minor innovation in open-source software.

           * i see almost no major innovation anywhere. major innovation is contrary to human society’s normative norms. women only got the vote a hundred years ago. british doctors still refuse to wash their hands. the french killed god knows how many thousands of their own to remove an aristocracy, then fell over themselves to create a new one 10 years later. humans HATE change.

     
    the key difference is: Quality of contribution. i made the same point some time ago re an early version of your Homesteading the Noosphere paper.

    an analogy might help here.
    i remember nearly 10 years ago visiting an old mate whom i’d worked with for a couple of years 10 years before in a unix-based R&D environment. he excitedly fired up a new linux install, and with a flourish displayed “this ENTIRELY NEW INTERFACE!!!!”
    i stared at yet another sun/win filemanager front end.
    “it’s just another sun/win filemanager-driven interface, but with new graphics/icons and a different place to click to get mac desk accessories”

    he had been really excited, and was then really offended. it wasn’t till a couple of years later and a few more iterations of the same style “innovation” that he understood what i meant.

    it was minor innovation. polishing and reimplementation and serious improvement, but of well-known existing concepts and implementations, with one or two extrapolations of same.

     
    by “major” innovation, i mean things like replacing the filesystem with a tag-based database, with all its consequent subtle yet profound changes in the way you work. like letting you drag&drop the whole OS and do drag&drop kernel hacks/unhacks. like profoundly changing THE WAY you interact with the computer.

    macintosh was a major innovation.
    tivo was a major innovation.
    emacs was a major innovation.
    the mouse was a major innovation.
    the mobile phone was a major innovation.
    the AS/400 was a major innovation.
    the palm pilot was a major innovation.
    the relational database was a major innovation (even if most of the implementations are not relational environments *old fume*).
    the graphics-drawn screen rather than a character-mapped screen was a major innovation.
    hell, curses was a major innovation: 1D-in-2D became 2D-native.

            hell, give me $10-20m and i can bring 3D-native to the desktop at a $20 marginal cost per screen and no goggles

    but ncurses was a minor innovation, no matter the huge social benefits derived from it due to the huge additional technical growth made possible by it.
    linux was a minor innovation, despite becoming subsequently incredibly important as a result of Social momentums (momenta?).
    converting FOSS from nerd-city to major global culture was a major innovation but Social rather than Technical.
    the web browser _could_ be viewed as a major innovation arising from open-source, but you could also see it as just a quickly-hacked-out proof-of-concept using only the read-only side of a much larger ongoing proprietary (if free$) effort to create a wholly network-based/shared-contributions interaction environment. which it was. that’s the reason it was so bizarrely labelled a “browser”.
    beos was a a minor innovation –the mac rewritten from scratch– a rewrite of an existing concept, tho incorporating a key idea from the AS/400 which looked like a major innovation to people with narrower experience. regardless, it was so much better and more useful than MacOS as an OS it was breathtaking.
    hypercard was a minor innovation as conceptually it was”just” an agglomeration of pre-existing concepts. even though it was possibly the single most important event in the Social history of programming (atkinson’s a GOD — i feel guilty just for typing that)
    clipping (regions) was a minor innovation, tho a staggering one with profound implications for every graphics-driven environment (especially FPS games) ever since. (yes, i feel guilty again)
    CDMA was a minor innovation, though another staggeringly brilliant insight and technical design accomplishment.
    endless flavours on filemanager approaches are minor innovations.
    fetchmail was an innovation. and one which became very widely used as significantly better than its predecessor. but the conceptual leap was minor compared to the jump from POP to IMAP.
    web2.0 is a minor innovation, a slight technical rejigging allowing yet another iteration of the ongoing centralised-decentralised-centralised cycle, albeit one that is having a large effect. due to pre-existing Social momentums.

     
    what i’m trying to get at here is the SIZE of the conceptual JUMP between ideas.

    think: emergent properties.

    getting something existing to work better or cleaner or several steps niftier is not a MAJOR innovation. no matter how technically difficult or accomplished it is, no matter that that work brings the idea into a far wider domain than the concept might otherwise have had: the work is minor innovation not major.

    and something i’ve noticed is that open-source lends itself to taking a concept and turning it into something better. cleaner, niftier, whatever.

    but the BIG conceptual JUMPS, have nearly all come from proprietary environments. only very peculiar proprietary backgrounds, true. typically in subenvironments deliberately insulated from the larger corporate context. but nevertheless, and despite me feeling precisely the same way about corporate environments as you do, i struggle to find even as many open-source major innovations as there have been proprietary.

    i would gladly be proven wrong.
    but scratching an itch or demonstrating astounding insight or accomplishment in the sight of your peers, is not the same as changing the way people live.
    and the community tending to reward the former (HtN), seems to me not to have delivered the same level of outputs in terms of the latter as the community which is directly rewarded by the latter (even if its normal internal structures strongly work against it).

    i’m not saying that’s a good thing. i’m just saying: in my experience, it’s a thing.

  8. OK, let’s see a classic example and deduce from that.

    Open Source / Unix comes up with PDF sometime long ago. Then the world moves towards a PC / DOS / Windows direction in what PDF doesn’t exist, and the 99% of users who use PC / DOS / Windows don’t even hear about it. Then a corporation, Adobe brings it back and it quickly becomes a success. It becomes a success because to the uninitiated, it looks noneditable for the recipients. At this point the old Unix guys are probably laughing their assess off, (“Joe! Don’t send that quote to the customer in a Word doc, because then they can edit the price and claim that was our original offer! Send it in PDF, they can’t edit that!” MWAAHAHAHAHAHAHANYIHIHIHI.), but nevertheless, Adobe PDF becomes a big success, despite that every release is slower than the previous one.

    Then Open Source comes up with a faster, better PDF reader and it seems they are just imitating Adobe. MWAHAHAAHAHA.

    But hey. What IS innovation, exactly? How do you measure it? Do you look at the thing itself, creating something, even if few people actually use that? Or do you take a utilitarian approach and say innovation is bringing a lot of benefit to a lot of people? Because if you take the second, it’s Adobe’s innovation.

  9. “The original browser and the original webservers were built by a hacker at CERN, not in some closed-door corporate shop.”

    I think this gives the impression that this was some spare-time or independent project Tim Berners-Lee did, when this wasn’t the case — Berners-Lee proposed the world wide web as a project, and his manager approved him working on the project. Robert Cailliau (also working at CERN), pushed for funding and resources later on, and also (over a year and a half period) persuaded CERN’s management to give the World Wide Web away without royalties.

    “open-source hacking built the Internet”

    Except the foundations of the Internet were started with ARPANET (project funded by DARPA), and TCP/IP came from further DARPA work. Much protocol work was done as research projects at USC.

    None of the above required open-source (for their invention)!

    “the open-source BIND daemon”

    The creation of which was funded by a DARPA grant, and the latest version (the rewrite from scratch) was funded by contracts. Maintenance and development of BIND is done by a non-profit corporation.

    I just don’t see the evidence of open-source innovation here (although open-source has facilitated their success, but that’s rather different).

  10. There are a number of parallels between what I do and the FOSS movement.

    1) The barriers to entry are low. Anyone with a word processor and a spreadsheet can make their own game rules for minis. Making a card game or boardgame is more challenging. This is the Desktop Publishing Crowd.

    2) Most of what gets released by the DTP games crowd is…less than compelling. The hard part isn’t making the game – that’s the fun part, it’s like hacking. The hard part is turning your game into a product that people can play and enjoy, and is the 30% of the work that makes you really regret not being an office drone and getting a regular paycheck.

    3) A lot of the stuff that actually IS innovative gets a limited market share. Production values and user interface matter. So does compelling game play, but production values and UI matter more for market acceptance.

    4) A segment of this industry attempted to form an ‘open source’ product movement with the Open Gaming License. Hasbro hasn’t repudiated the old OGL…but they haven’t released D&D 4th edition under its terms. They have made it a condition that if you’re a third party publisher and want to produce licensed products for 4th edition, that you cease producing new products under the OGL. They have also made the old OGL source materials hard to find. (You now have to use Google, and know what you’re looking for, rather than find it off a link from their home page.

    The net result of the OGL was a 5 year market glut of CRAP, from 2002 to 2005. Where anyone with a copy of a spreadsheet, a word processing program and a credit card to pay the printer’s bill with could try and get their product out….and it fragmented the customer base, resulted in a lot of game stores having clearance bin sales, and, oh, strangling the distribution channels. This ultimately hurt the D&D brand, which is why the focus shift away from OGL.

    There are still companies publishing and printing OGL material. Very few now, and the niche has shaken out. I don’t think it’s a good business model, which is why we’re not doing it. (Well, that and the “Open” in the Open Gaming License is “Open like the jaws of an alligator with a thn mint on its tongue”. Hasbro may not be able to make a legal case for shutting down the OGL, but they have a litigation budget that’s larger than my company’s annual revenue stream. They can afford to let it sit in court until the plaintiff starves.)

    5) There are also attempts to develop FOSS games by Wiki…which turn into lots of Wiki posting, very little player and user testing, and the project collapsing when the person whose ego was driving it gets fed up. Lots of people want to hack game rules; nobody wants to do the actual work needed to make a working product.

    A friend of mine, who teaches at the Naval War College says that the sure fire sign of a new game designer is this statement:

    “Oh, the game is almost done, now I have to write the rules…”

  11. Innovation is in many ways an antithesis of a large corporation. Innovation means a momentary regression at best and a failed concept at worst. When improvements are measured quarter after quarter, nobody wants to appear like they are losing control of the situation.

  12. >I think this gives the impression that this was some spare-time or independent project Tim Berners-Lee did, when this wasn’t the case

    Are you really claiming that the Web and the Internet don’t count as open-source innovation because they were officially funded? Seriously?

    If so, that would be deeply silly. Open source is not about who funds it or doesn’t, it’s about the terms on which the software is released and the development process that arises from those terms. Similarly, the fact that DARPA funded the development of Internet or BIND does not somehow make the lessons of open source inapplicable to non-government-funded projects.

    If you want to make the claim that major open-source innovation can’t happen without a sugar daddy behind it, then whether Berners-Lee was working off-hours or not is perhaps relevant. But that claim is also obviously false. Consider, for example, Perl. Or Emacs. I’d cite more recent examples, but it’s difficult to know whether newer projects constitute major innovations because we haven’t had time to evaluate their impact yet.

    We demonstrably do not live in a universe where major software innovations require major funding. But supposing we did, well-funded projects with open-source terms would still generate more innovation than well-funded projects without them, simply because more human beings would be thinking about their problems.

  13. >The net result of the OGL was a 5 year market glut of CRAP, from 2002 to 2005.

    Right. We have a large amount of crap in the open-source world. But there are a couple of key differences that make it almost a non-problem here.

    1. Most people don’t assemble their own systems. The distribution packagers therefore act as a quality filter, making the crap mostly invisible to end users.

    2. There are reliable proxies for project quality that can be evaluated with low overhead by someone casually browsing project sites. Of these the most important are number of contributors and the amount of time the project has been live since first release.

    3. Crap doesn’t drive out good stuff. In our distribution channels there is no real equivalent of contention for shelf space. This is important because ut means the crap is not toxic to the rest of the ecosystem.

    Still, even given your contention problems, I’d bet that buried somewhere in that crap there were products more original and creative than anything Hasbro managed to ship. Which is not to say they were more polished or better, just more innovative.

  14. >Still, even given your contention problems, I’d bet that buried somewhere in that crap there were products more original and
    >creative than anything Hasbro managed to ship. Which is not to say they were more polished or better, just more innovative.

    Put like that, innovation has very little use.

    When an open source innovation appears, if what it takes to be noticed is a corporation making their own polished version of the general idea – which can be closed-source, depending on the nature of the invitation – then open-source just lost its innovation, and also lost face in the public eye.

    Many such innovations can be taken and closed, because the innovation itself isn’t copyrighted – only the implementation is. And once an innovation is closed, it’s hard to re-open, as evidenced by the current situation.

  15. >When an open source innovation appears, if what it takes to be noticed is a corporation making their own polished version of the general idea

    But this is obviously not the case. Consider Firefox; even the general public noticed that one. Perl, Python, and Emacs didn’t need a sugar daddy either.

    >Many such innovations can be taken and closed, because the innovation itself isn’t copyrighted – only the implementation is. And once an innovation is closed, it’s hard to re-open, as evidenced by the current situation.

    I can’t figure out what you think you’re talking about here. Did somebody patent the browser, or scripting languages, or editors programmable with a Lisp dialect, while I wasn’t looking? If not, in what sense are any of these innovations “closed”?

  16. esr, I’d like to thank you for the blog post itself. I’ve always been speechless when encountering anyone that thinks open source really can’t innovate, it just seems like it should be common sense to think otherwise. Though I’m not the best person for explaining these types of thing well, so the few examples I muster up whenever it comes up never really hits the nail….

    > I can’t figure out what you think you’re talking about here. Did somebody patent the browser, or scripting languages, or editors programmable with a Lisp dialect, while I wasn’t looking? If not, in what sense are any of these innovations “closed”?

    Well, in the web browser market, there’s that mess of vomit of proprietary/non-standard HTML extensions Microsoft and Netscape so happily added in the 1990s. The love for proprietary plugins like Java and Flash is a headache (_why_ anyone thought video in Flash is a good idea is beyond me; the old way of embedding an MPEG worked far better in every regard); though Microsoft’s Silverlight amusingly is a (small) step in the right direction (Microsoft is co-developing with Novell a BSD-licensed plugin for it, and it’ll be the official plugin to use on Linux/BSD/Mac OS X).

  17. >Well, in the web browser market, there’s that mess of vomit of proprietary/non-standard HTML extensions Microsoft and Netscape so happily added in the 1990s.

    And you had to qualify with “in the 1990s” because this strategy is in final collapse now under pressure from open-source competition. Java has gone open source, Flash isn’t yet, but Adobe Flash 10 runs on Linux and when gnash 1.0 ships that lock-in will be gone regardless of anything Adobe does. After that, the count of proprietary bottleneck technologies on the Web will be…zero.

    Sure, you can try this game, but it’s as doomed an effort as DRM has been in the music industry, and for about exactly the same reasons. Remember when there were a dozen competing video and audio codecs jostling for attention on the Web? WMV, RealAudo, Quicktime? Here’s what happened: YouTube achieved critical mass and flattened that space, leaving just one target to be reverse-engineered. It’s accidental that the target happened to be Flash; what’s essential is that at that point the target stopped moving. From that point it was just a matter of time and hacking until the lock-in collapsed.

  18. @esr: “Consider, for example, Perl.”

    Written by Larry Wall while he was an employee or contractor, I forget which, of JPL

    @esr: “Consider Firefox; even the general public noticed that one. Perl, Python, and Emacs didn’t need a sugar daddy either.”

    Firefox, the end-game outcome of Netscape attempting to run off with NCSA’s Mosaic, and then being pounded by Microsoft. Did the “open source” release of Netscape’s source code save it? No.

    And its becoming clear that efforts like Webkit (which is, yes, Open Source, but far more often developed by commercial entities (Google, Apple, and now Microsoft) trump the pure Open Source envrions that have created and maintained Firefox.

    And of Perl, Python and Emacs, only Emacs didn’t have a ‘sugar daddy’. The others did, so you can’t say that they didn’t require one. We simply don’t (and never will) know.

  19. “Are you really claiming that the Web and the Internet don’t count as open-source innovation because they were officially funded? Seriously? ”

    No, not because they were officially funded, but because they didn’t need open-source software for their invention. Probably the World Wide Web wouldn’t have been successful had CERN not been persuaded to make it open, but that doesn’t mean open-source software enabled it to be invented. Similarly for the Internet, DARPA were trying to find (and paying people) to find solutions to problems they had, I don’t see how open-source software was needed for this.

    “If so, that would be deeply silly. Open source is not about who funds it or doesn’t, it’s about the terms on which the software is released and the development process that arises from those terms. Similarly, the fact that DARPA funded the development of Internet or BIND does not somehow make the lessons of open source inapplicable to non-government-funded projects.”

    I never mentioned the lessons of open-source software, I was only concerned with the innovation itself (the subject of your post). The role open-source helped in later software maintenance / development is a separate issue entirely.

    If you want to claim these as examples of how open-source software can innovate, please show the role open-source software played in their invention.

    “If you want to make the claim that major open-source innovation can’t happen without a sugar daddy behind it, then whether Berners-Lee was working off-hours or not is perhaps relevant. But that claim is also obviously false. Consider, for example, Perl. Or Emacs. I’d cite more recent examples, but it’s difficult to know whether newer projects constitute major innovations because we haven’t had time to evaluate their impact yet.”

    I never made, nor intend to make, such a claim. What I do claim is that the examples of yours that I cited do not appear to me to be examples of open-source software innovation. In fact, I consider those to be more akin to a corporate R&D department — people were being paid to propose projects or come up with solutions to problems, some of which were considered by management interesting enough to pursue further; I don’t see the role open-source software played in making that happen.

    “We demonstrably do not live in a universe where major software innovations require major funding. But supposing we did, well-funded projects with open-source terms would still generate more innovation”

    I never made such a claim. However, your claim that innovation happens more with open-source software is arguable — how have you determined the amount of innovation in closed-source software, and that of open-source software (leaving aside the whole issue of what’s actually classed as open-source software)? What metrics are you using?

  20. >No, not because they were officially funded, but because they didn’t need open-source software for their invention.

    If I’m parsing you correctly, you believe that it is a necessary condition for open source to be credited with an innovation that open-source software was used to implement the innovation and no closed-source software would have sufficed to do it.

    I must say I find this completely bizarre. The key moment of innovation for fetchmail, as I described in one of my papers, was when someone in my dev group suggested that one of its delivery modes should be SMTP, and I realized that I should start thinking of it as an adaptor fitting in the space of mail stream protocols rather than a mailbox-filler. On your theory, this doesn’t count because I might have implemented fetchmail with a closed-source compiler! Or, reading you slightly differently, it doesn’t count because I might in some alternate universe have released fetchmail as closed source.

    Sorry, but I think you’re just desperately confused. What mattered in that situation wasn’t the licensing status of my compiler or operating system, it was that I deliberately cultivated a many-eyeballs group of developers and then had the sense to listen when they critiqued my code and design. It’s certainly handy when we have development tools with a pure open-source pedigree — it removes a point of control other people might have over us — but it’s not essential to the method.

    And “might have released it as closed source” reverses on you, actually — there is any amount of innovative closed-source software that could have worked just as well in open source. Are we to not count any of tht as part of the argument for closed-source innovation strategies? By your logic, we’d have to.

    Innovations must count for open source if they were developed in the open-source way — transparency of process, encouragement of review by anyone, and the effective grant of rights to redistribute and modify. They must count for closed source if they were developed in a closed shop and issued under proprietary licensing. Otherwise you’d get really perverse cases like an innovative program developed with GCC under Linux counting as open source despite being issued under a proprietary license!

    So: Perl counts as open-source innovation, despite being written on Solaris with proprietary tools. So do Berners-Lee’s original Web suite written under proprietary NextStep . In both cases, what was critical to their success was the evolution of a larger community of co-developers and stakeholders than a closed-source project can allow without crashing business models based on the secrety and scarcity of the code.

    That is how you know that open source is key to an innovation. It’s about the social machine around the code, not the particular software tools used to build it. It’s about the reciprocity among contributors, agreeing to a practice in which no one has an asymmetrical power privilege over the code.

  21. Eric, in my experience, this is a perspective where you’re deluding yourself.

    It’s about the social machine around the code, not the particular software tools used to build it. It’s about the reciprocity among contributors, agreeing to a practice in which no one has an asymmetrical power privilege over the code.

    I have seen very very few projects go forward without a primary code maintainer, or a primary idea maintainer. That person HAS an asymmetrical power privilege over the code or project in that he can accept or reject contributors.

    Whether or not his project works, or joints the billions of bits of abandoned project work is entirely a function of how good he is at wearing two hats: The “developer/engineer” hat, and the “herder of cats” hat.

    In big coding environments, where I’ve done tech writing in the proprietary world, most well run companies will brainstorm ideas, assign someone they know has some leadership moxie, let him assemble a team of 2-3 developers, and tell him he has to meet a certain deadline. It then goes out for internal testing, and many companies “crop rotate” their engineers from dev to testing and back. Only after it meeds this loop does it go to end user usability testing – and it’s at end user usability testing that things get concrete poured on them, features get frozen, and you get the experience you describe.

    I saw this happening at the place where my father was programming in the early ’80s. I saw it at places where I was tech writing in the ’90s. Per people I know who work in Microsoft, Apple and Google, this is the model they use.

    Aside from giving the end user the source code, this is parallel in many ways to what you describe…and my experience with code monkeys in the corporate world is very different from yours. (I was the tech writer cleaning up the messes in the end…).

  22. “If I’m parsing you correctly, you believe that it is a necessary condition for open source to be credited with an innovation that open-source software was used to implement the innovation and no closed-source software would have sufficed to do it.”

    Not at all.

    However, your claim is that open source can innovate (a claim I don’t disagree with), and you cite the Internet and the Web as examples of this; I’m saying I don’t see the evidence for these two — yes, I think the protocols being made open were important to their success, but that’s nothing to do with the invention itself.

    Initially, the Web was done by a single person, and then grown to a small team, and only after a period of time (after, I would argue, it had actually been invented), was it made open. How is that open source?

    The Internet really grew out of a project specifically requested by DARPA to overcome the problem of having various incompatible networks, and they brought together people to find a solution to the problem. How is that any different to Microsoft forming a group to look at (insert chosen problem here).

    “The key moment of innovation for fetchmail, as I described in one of my papers, was when someone in my dev group suggested that one of its delivery modes should be SMTP, and I realized that I should start thinking of it as an adaptor fitting in the space of mail stream protocols rather than a mailbox-filler.”

    Incidentally, this didn’t require open source — it was the presence of a group working on the project, something which is achievable regardless of whether closed or open (or whatever hybrid you happen to use).

    “Innovations must count for open source if they were developed in the open-source way — transparency of process, encouragement of review by anyone, and the effective grant of rights to redistribute and modify. They must count for closed source if they were developed in a closed shop and issued under proprietary licensing. Otherwise you’d get really perverse cases like an innovative program developed with GCC under Linux counting as open source despite being issued under a proprietary license!”

    Innovation is innovation. A question is whether what comes after is more innovation or evolution. If someone comes up with an innovative idea, knocks out an application, releases it as open source, a community forms and grows the application, is that growth really innovation?

    “So: Perl counts as open-source innovation, despite being written on Solaris with proprietary tools. So do Berners-Lee’s original Web suite written under proprietary NextStep . In both cases, what was critical to their success was the evolution of a larger community of co-developers and stakeholders than a closed-source project can allow without crashing business models based on the secrety and scarcity of the code.”

    Your claim was not about whether projects were successful, but about whether open source could innovate. Did the larger community actually innovate or merely evolve what was there?

    As an aside, you seem to move between using the terms open source software and open source at various points, often with open source used in reference to software, but it isn’t clear to me that you actually mean the two to be the same thing.

    “That is how you know that open source is key to an innovation. It’s about the social machine around the code, not the particular software tools used to build it. It’s about the reciprocity among contributors, agreeing to a practice in which no one has an asymmetrical power privilege over the code.”

    So Guido van Rossum doesn’t have asymmetrical power privilege? Nor Larry Wall? Nor Linus Torvalds?

  23. Open source is disruptive to innovation.

    When you forcibly make an asset free, the little guy gets hurt first and hardest. File sharing of music didn’t hurt Britney Spears; it hurt the small up-and-coming musicians who may not have gotten that record deal because of a fearful record label.

    Open source isn’t going to put a huge dent in Microsoft (Ballmer’s chair-throwing ranting about communism notwithstanding). Open source has almost completely destroyed a cottage industry of individual developers and small companies writing unique applications with the expectation that they would be compensated for their vision, talent, and efforts. This essentially divides the industry into open source (essentially, giving your time and knowledge away) on the one hand and working as a mindless drone in a soulless corporation on the other hand, with the cottage industry of artisan software developers becoming more and more an excluded middle. Having made the acquaintance of a few such talented developers, who became despondent at the prospect of not realizing their dreams, I can only conclude that open source has hurt a vital part of our craft, the wellspring of innovative ideas and the rightful successor to the MIT hacker culture.

  24. I think it bears repeating: The number of smart people in your company (or on your project) is the log of the number of people in your company (or on your project).” (Bill Joy’s other law.)

    Note that the base of the log is not expressed, and that log(1) is always 0, no matter the base. if you’re alone on your project, there are no “smart people” working on your project. :-) (Consider this as a form of support for the “XP” methods no leaving ‘vogue’.)

    The larger effect of “Joy’s other law”, is that on the whole innovation will occur elsewhere. Outside your company, outside the boundaries of your project. Quite simply, there will always be more smart people outside your company than inside it.

    “open source” (as opposed to “free software”, which focuses instead on the user of software) is a model to harness the individual small pockets of innovation that will be found around an idea. It is not the only, or even the most successful model, but it does work.

    Open source licensing is neither necessary or sufficient for innovation to occur. That doesn’t mean its not useful.

    Open Source methods have their roots in the hacker culture of the 60s and 70s. (Some of these ‘hackers’ helped invent the Internet.) It is not true that the inventors and developers of “the Internet” were using open source methods, but rather that “open source” is an evolution (but not the only branch) of their “rough consensus and running code” mindset.

    If you take a close look, of the examples given (CERN’s support of TBL’s ‘web’ efforts, perl, bind, …), NONE of them began as an “open source” project. They all evolved to same. The exception is emacs, and then only if you consider the branch(es) rooted in GNU.

    I also dispute that “open source” is “the rightful successor to the MIT hacker culture”, in part because that culture hasn’t diminished enough to be subservient to the “open source” culture and methods.

  25. I also dispute that “open source” is “the rightful successor to the MIT hacker culture”, in part because that culture hasn’t diminished enough to be subservient to the “open source” culture and methods.

    As do I. I was referring specifically to what Eric calls “proprietary software at the artisan level”. For it is to this segment of the industry that the best and the brightest of the microcomputer world were attracted in the 80s and early 90s. And today they’re getting the shaft, both from above (large well-funded corporations with better advertising budgets) and below (open source).

  26. >I have seen very very few projects go forward without a primary code maintainer, or a primary idea maintainer. That person HAS an asymmetrical power privilege over the code or project in that he can accept or reject contributors.

    Yes. But a project under open-source licensing can always be forked. That’s what makes the lead position a matter of consent — unanimous consent, actually — rather than an actual power privilege. Not just the unanimous consent of the project group, either, but of the entire hacker community. And that applies to Linus Torvalds or Guido van Rossum too, as I can confirm both of them know very well because I’ve talked with them about it.

    In fact, it is a matter of verifiable record that after I explained the open-source social machine in public, both Linus and Guido deliberately tweaked the social machines around them to decrease previous asymmetries in their positions by making forking of their projects easier. They know why they did that; I know why they did that. You’re more than smart enough to figure it out, given the clue.

    >In big coding environments, where I’ve done tech writing in the proprietary world, most well run companies will brainstorm ideas, assign someone they know has some leadership moxie, let him assemble a team of 2-3 developers, and tell him he has to meet a certain deadline. It then goes out for internal testing, and many companies “crop rotate” their engineers from dev to testing and back. Only after it meeds this loop does it go to end user usability testing – and it’s at end user usability testing that things get concrete poured on them, features get frozen, and you get the experience you describe.

    Read Christophe de Dinechin’s posting, on the other hand, for independent confirmation that not only was I describing the general run of corporate development accurately, but that in the twenty years since I was personally trapped in that machinery of suck conditions have actually gotten appreciably worse than I remember. Not that this is news to anyone who has, say, read Ed Yourdon’s “Death March”. Or understood what the agile-programming crowd is reacting against.

  27. >When you forcibly make an asset free, the little guy gets hurt first and hardest.

    I agree, and I’ll play that quote back at you the next time you advocate some collectivist redistribution scheme here. Which I expect to be, oh, about ten minutes after this thread is finished,

    Unlike your preferred class of political solution, open source doesn’t involve forcible anything. There’s no team of Birkenstock-wearing commandos holding .44 Magnums to the heads of garage developers demanding that they relicense under GPL/BSD/MIT.

    As for open source destroying artisan software…these people show up at my talks. And not to throw tomatoes, either, but to thank me for liberating them. Seriously! I remember one bearded gentleman whose single product is a math library for doing applied physics calculations who shook my hand and explained that I’d helped him understand that he is actually in the consulting and training business and both his life and his revenue stream improved as a result.

  28. esr, .44 Magnums are decadent capitalist firearms. Real revolutionaries would only use AK-47s.

  29. Well they’re already wearing Birkenstocks instead of the industry-standard jackboots and balaclavas… :)

  30. I’m a bit familiar with the “cottage industry” when it comes to small-business software, I remember the boom in the eighties when everybody was writing invoicing, payroll, inventory, order processing etc. applications for small businesses in CA-CLIPPER in the eighties and early nineties, and later when it got transformed to FoxPro, and, to lesser extents, Delphi and Visual Basic. This is a boring job, but this what the freelancers can make a living of.

    It did not got hurt by Open Source at all. Most entrepreneurs want a big brand name, Microsoft or SAP, even though SAP B1 has nothing to do with the real SAP, and it really sucks, but they have a blind faith, big brand name = good. Actually, they are right in the sense that the “cottage industry” is awful in support (because these small businesses never pay enough to really finish the software, so it needs a lot of support, but they never pay enough for that too, so they usually get little support) and what they are hoping for isn’t a better product but a better support and they hope they can get that from the big brand-names.

    The other kind of small entrepreneur, those with high time preferences, present-oriented, are still making deals with the “cottage industry” because the time investment in an Open Source product – figure out whether it’s good for you or not, implement it, try to get support etc. – is just too much for them, better have a college kid cruft together something in Delphi + MS SQL in 3 weeks and then at least they have something in 3 weeks and at least they know who to bother if it doesn’t quite work.

    There are really nice Open Source business software out there, I’m kinda impressed with TinyERP, it could have a great future. But the first type of entrepreneurs won’t accept it without a big brand name, and as for the second type, the “cottage industry” likes to write his own software, not to customize an existing one. (And they don’t like Python. For the “cottage industry” Python is some freaky geeky stuff. They grew up in the microsofty culture, they want to use VB.NET, Delphi or FoxPro. And some still remember CA-CLIPPER :-) )

  31. “But a project under open-source licensing can always be forked.” But you’ll probably be passionately flamed by everybody. Case in point: Xemacs.

    Having said that, I think let’s not get too hung up on power relationships. They aren’t very important. I’m sick of the vulgar-Frankfurter approach that “OMG XY has power, oh noes, that’s sooo bad, he is Hilter 2, let’s fight for freedom!”. Power is simply a means to organize activity so that shit gets done at the end. It’s not about egos, and should never be. When shit doesn’t done, power will be challenged anyway.

  32. >Initially, the Web was done by a single person, and then grown to a small team, and only after a period of time (after, I would argue, it had actually been invented), was it made open. How is that open source?

    In quite the usual way, exactly. Projects that start as a wishlist and an empty file are rare. Usually project leads throw together a proof of concept (what I called a “plausible promise” in my papers), throw it open, and rely on attracting collaborators to improve it. Innovation may happen in the prototype, but no project lead in his right mind expects it not to happen afterwards.

    I’ll give you a real-world, current example. I’m working on a prototype called “deheader”, a Python script that computes the subset of #includes that can be removed from a C or C++ file to speed up compilation. In old code this sort of cruft accumulates, and can really slow your build, especially in C++. The innovative bit in it is using repeated automated compiles of slightly modified versions of the file to probe the inclusion list. I don’t think anyone’s ever done this before, at least not where Google can see it.

    In due course, I’ll probably throw this thing on a public repository and announce it. Why? Well, aside from the fact that I’m an open-source guy and I do things that way by habit, there are a couple of harder problems I haven’t solved yet. For some of these — like how to make sure I’m not removing headers that only look disposable because they’re guarded by ifdefs — I can imagine solutions. For others I haven’t got a clue, yet. Like, suppose I try to extend the optimization to .h and .hpp files; if there’s an inclusion duplicated in multiple .h and ,c files that use it, how do I tell where the minimal set of instances should be left? That sounds like a simple question but I have a dark suspicion that the complete answer involves a nontrivial graph-coloring algorithm, or at least something like a topological sort.

    OK. Project this into the future. There’s an open-source deheader project, and I’ve got a bunch of design collaborators helping me solve these harder problems. Is what you’re proposing that my original bright idea doesn’t count as an open-source innovation because I hadn’t thrown the code on a repo yet? Because that seems nutty to me, an elaborate way to miss the point. I’m dead certain it would seem nutty to Linus Torvalds and Larry Wall, too — I know these guys.

    >Your claim was not about whether projects were successful, but about whether open source could innovate. Did the larger community actually innovate or merely evolve what was there?

    Linus thinks they innovated like crazy. So does Larry. So does Guido. So does Tim-Berners Lee, though in his case I’ll admit I haven’t confirmed this face-to face.

    Let’s consider deheader’s hypothetical future. It isn’t at all unlikely that when I publish this thing and the TODO list, some bright spark will land in my mailbox saying “Yup, you need complicated algorithm X for this because anything less will fail in this edge case here, and by the way here’s a patch to implement it and a test load.” Let’s call my benefactor Fred Foonly foe concreteness. The essence of open source is that there are lots of Fred Foonlys out there and they will often have insights that I can’t necessarily duplicate.

    So I ship deheader 2.0. Does the Foonly algorithm not count as an open-source innovation because my prototype took a while to make it to a repo? Is your distinction between innovation and evolution mostly specious? I think I know the answers to these questions; Linus, and Larry, and Guido, and Tim all think they know the answers too, and have expressed that knowledge in their behavior. And even if I’m not actually smarter than you, they almost certainly are.

    >Incidentally, this didn’t require open source — it was the presence of a group working on the project, something which is achievable regardless of whether closed or open (or whatever hybrid you happen to use).

    Boggle…are you just trolling?

    OK, let me lay this out in e-x-t-r-e-m-e-l-y s-i-m-p-l-e terms. If I want “a group working on the project”, I have to supply an incentive
    for people to join it. The closed-source way is to offer people salaries to do it; I can’t do that. Hypothetically, I could make an offer that says: “You contribute to my code; I won’t pay you, then I’ll issue it under proprietary license and capture all the gains myself.” That is…unlikely to work well. The open-source way is to say “If you join my project, I’ll give you the same rights to use my work as you give me to use yours”.

    As a matter of observed fact, the open-source incentive scales better than the closed-source one. The Linux kernel’s contributor list is an order of magnitude larger than Microsoft’s entire programmer headcount — and that’s the extreme case that favors close source, because Microsoft can afford to hire more developers under a single roof than anybody else. Drop down to lower levels of corporate gigantism and you’ll find the open-source advantage in team sizes widens to two, even three orders of magnitude. With a corresponding increase in frequency of innovation, even if it rises sublinearly by team size (n log n would be my guess for reasons I can explain in detail).

    So, yes, open source does make a difference. A rather large one. I invite you to run your theory past Tim Berners-Lee or Fred Baker sometime; I hope I can be there to watch them laugh.

  33. Back to my earlier comment – how do you define innovation? ? What about the big picture – is replacing one small part of a huge system innovation? If yes, is every bugfix an innovation? If no, then only revolutions count as innovation?

    Or take a utilitarian approach, innovation = delivering a lot of benefit to a lot of people IF they perceive it not only as something more, but something new? In this case PDF is Adobe’s innovation, because 99.9% of the current users don’t know it existed long ago on Unix.

  34. >It is not true that the inventors and developers of “the Internet” were using open source methods, but rather that “open source” is an evolution (but not the only branch) of their “rough consensus and running code” mindset.

    As usual, your ignorance is exceeded only by your tendentiousness. I’ve had this conversation with Fred Baker; he attended the 1998 launch conference of the open source movement while he was chair of IETF, and he told me that I had captured the essence of their process on both current and historical levels. The chair of IETF then became one of the two dozen tribal chieftains who voted on the “open source” rebranding. And voted for it.

    >I also dispute that “open source” is “the rightful successor to the MIT hacker culture”, in part because that culture hasn’t diminished enough to be subservient to the “open source” culture and methods.

    I’ve been watching the MIT hacker culture closely since before the Jupiter cancellation; I was personally shown around 545 Tech Square by RMS before either of us was famous. I know what that culture was like then, and I know what it’s like now. And…well, just keep talking. You get funnier every time you open your mouth.

  35. >Back to my earlier comment – how do you define innovation? ? What about the big picture – is replacing one small part of a huge system innovation? If yes, is every bugfix an innovation? If no, then only revolutions count as innovation?

    I think scale is not as important to the definition as originality. The technique I invented for deheader is a tiny innovation — I’ve made larger ones before, and probably will again — but genuine enough. Bugfixes aren’t innovations because they repair a thought that already existed rather than originating a new one. Feature additions can be innovations, but usually aren’t.

    I think the idea that innovation and collaborative evolutionary development are mutually exclusive is partly a holdover from romantic myths about lone-genius inventors and visionary artists. Speaking as somebody who’s been called a visionary and a genius fairly often, I think these myths are really unhelpful. Yes, there have been a couple of times in my life when I had the classic lightning-strike moment of loner genius, and at least one of these moments turned out to be historically important. But that moment came from twenty years of preparation, and that preparation involved working intensely with other people in what we now call an open-source style and an evolutionary, gradual way. In at least my experience those two things aren’t separable.

    I’m sure that Linus and Larry and Guido and Fred Baker would agree with me about this, and pretty confident about Tim Berners-Lee.

    (Hm. Thomas Kuhn’s utterly wrongheaded but immensely popular take on the structure of scientific revolutions has probably helped confuse people about this; see Brother, Can You Paradigm? for dissection.

  36. Shenpen, how was PDF around before Adobe? From what I have read, Adobe created PDF.

    ESR, money sometimes works better than open source. Case in point.

  37. Of course money sometimes works better than open source. Sometimes socialism works better than capitalism. And sometimes pigs *do* fly, if you throw enough of them hard enough.

  38. As usual, your ignorance is exceeded only by your tendentiousness. I’ve had this conversation with Fred Baker; he attended the 1998 launch conference of the open source movement while he was chair of IETF, and he told me that I had captured the essence of their process on both current and historical levels.

    “he told me that I had captured the essence of their process” but you left unstated that their process existed prior to your providing a name (and perhaps a few enhancements) to the process already employed by the IETF.

    Can’t you see it, Eric? *You* *captured* what *they* had *already been doing* for years. Over a decade. You weren’t the first with the “development mode”, though you did give it its most popular name. And now we’re right back at the place where you and I do agree. “Open Source” is a marketing term.

    What I said:

    It is not true that the inventors and developers of “the Internet” were using open source methods, but rather that “open source” is an evolution (but not the only branch) of their “rough consensus and running code” mindset.

    They were running under the rubric of “rough consensus and running code” long before “open source” became a descriptive term for a software development model. The phrase was coined by David Clark in. 1992. To quote the OSI:

    The ‘open source’ label was invented at a strategy session held on February 3rd, 1998 in Palo Alto, California.

    The ‘rough concensus…” phrase existed 6.5 years before the term “open source” was applied to software.

    Jebus, man. I wasn’t taking anything from you, I was just straightening out the time line.

  39. I’ve been watching the MIT hacker culture closely since before the Jupiter cancellation; I was personally shown around 545 Tech Square by RMS before either of us was famous. I know what that culture was like then, and I know what it’s like now.

    OK, so you’re OLD. :-) Happy 52nd birthday in advance.

    You were given the tour by RMS, and that qualifies you to speak about the culture “why?”

    I don’t think I got to 545 Tech Square until 1983. My interested deepened upon reading the Aug 1980 “The Hacker Papers” in Psychology Today, though I’d been subscribing to TAP for a couple years by then.

    Woooooo, you sure pulled out the can of whoop ass on me with that one.

    But wait, you didn’t trip across Unix until 1983, and I’d been running it on a pair of Vax 11/750s since very late in 1980.

    While you were taking the nickle tour and dicking around with your IBM PC writing a serial communications program, I was taking a deep dive into Unix device drivers and early-stage TCP/IP networking.

    And…well, just keep talking. You get funnier every time you open your mouth.

    What is it with you and Russ Nelson? You both get combative rather than support your position. If I do humor you, then I’m glad you’re enjoying this. If you’re being sarcastic, well… thats your problem, not mine.

  40. >Jebus, man. I wasn’t taking anything from you, I was just straightening out the time line.

    I didn’t think for a moment that you were taking anything from me. You don’t have the capacity to do that.

    Instead, you were backing an argument that the Internet doesn’t count as open-source innovation. Now you’ve changed the subject to what my role in elucidating the process was, which is a distraction. The point here is that that the IETF did prior to the invention of the term “open source” counts as open source innovation because the chair of the IETF recognized it as such, having principled and correct reasons for doing so.

    And that, in turn, illustrates why “open source” is more than a marketing term. It’s not just a description of the sizzle, it’s a description of the steak. Recognizing that what you do is open source has generative consequences; in particular, it supplies a theory of how to improve your process that you (probably) didn’t have before. I should blog about this.

  41. > Instead, you were backing an argument that the Internet doesn’t count as open-source innovation.

    no, I was advancing an argument that your claim of perl, netscape and the work of TBL being “open source innovations” rings false. Yes, they all catalyzed open source innovation, but none were invented as open source innovations. That came later.

    I did allow that emacs was founded in Free Software, and is thus open source.

    Or, to put it more succinctly, someone recently said, “… it takes sound design of the overall system at so many higher levels that open source is really only a minor part of the toolkit.”

    Amen, brother.

    > Now you’ve changed the subject to what my role in elucidating the process was, which is a distraction.

    Quoting:

    I’d say I had to get one of these because it’s got some of my software in it, but as a one-time maintainer of GIFLIB (not to mention named contributor to libpng) just about every cellphone has some of my software in it. (And every browser. And the Microsoft X-box. I am become Shiva, destroyer of worlds ubiquitous and omnipresent.)

    Look,

    Is open source necessary for innovation? No.

    Is open source sufficient for innovation? No.

    Can open source catalyze innovation, or otherwise act as an oxidant to accelerate the market burn-down of a given software solution (product) space with a direct consequence that the market space for the application is dominated by only open source solutions?

    Yes. Witness Firefox and linux.

    I think we agree on the big picture. My issue is that your examples are often weak or broken.

  42. > And sometimes pigs *do* fly, if you throw enough of them hard enough.

    Normally “flight” involves something more than ballistics.

  43. This is a good post.

    But about Firefox: it wasn’t totally developed on the open source model, was it? I mean didn’t IBM or somebody put some corporate muscle behind it?

  44. A few comments up Eric mentioned his position that open source innovation occurs irrespective of corporate backing. Best not to retread terribly much; he’s hopping mad now :)

    But you raise an interesting point: a lot of open source products started off as closed source; oftentimes this happens when an innovative product fails to compete in the marketplace and its original developers want to see it live on. Kind of hammers home my point, though, about artisan developers unable to make a buck.

  45. >But about Firefox: it wasn’t totally developed on the open source model, was it? I mean didn’t IBM or somebody put some corporate muscle behind it?

    It’s complicated. Here’s a brief and possibly oversimplified summary:

    0. First, there was Tim Berners-Lee’s original browser. Open source by intention, but nobody understood licensing then. (It was 1991 and even the ink on GPL 2.0 was barely dry.)

    1. First, there was Mosaic. It was developed under nearly open-source terms, but with some restrictions on commercial reuse of the code. (Nobody had figured out that these sorts of restrictions damaged the method yet, because it was 1992-1993 and nobody understood the method — they just did it.)

    2. Then there was Netscape, which was basically Mosaic with tailfins and chrome and a proprietary license.

    3. Then there was open-source Netscape, which was kind of a mess because the code had bloated badly.

    4. Firefox is what happened when some smart people stripped the rendering engine out of Netscape and threw the bloat away, then re-skinned what was left.

    So, basically, the period during which there was lots of corporate money behind the development was the period of bloat and things going architecturally wrong.

    What you’re probably thinking of is actually IBM’s funding of the Apache Foundation.

  46. @JimThompson: what’s the launch speed of a pig? Maybe one of them *can* fly, and it’s just a question of getting enough of them up to speed to launch the one that can fly? Like a scramjet?

  47. What is it with you and Russ Nelson? You both get combative rather than support your position.
    I don’t get combative, I just get quickly frustrated with you. You are obviously intelligent, but you lack wisdom. It’s like you have this blindness in your life where there are certain things that you simply do not perceive. Have you ever had a stroke? Or head injury? Or other kind of brain damage? My father-in-law had a stroke and now has “left neglect”, where pretty much everything works, except that nothing to the left of him exists. Might you suffer from economics neglect or something similar?

  48. JimThompson: what’s the launch speed of a pig? Maybe one of them *can* fly, and it’s just a question of getting enough of them up to speed to launch the one that can fly? Like a scramjet?

    I suppose if one were to accelerate the pig to “escape velocity”, (11.2 km/s) then you could get the pig into low-earth orbit. (I hope you like the smell of burning bacon.)

    Are you aware that no flown scramjet has ever been successfully designed to survive a flight test?

  49. Have you ever had a stroke? Or head injury? Or other kind of brain damage?

    Is this a coy way of asking if I was dropped on my head as a child?

    Might you suffer from economics neglect or something similar?

    We have different ideas, Russ. I’m more in line with E.F. Schumacher’s economics. Having admitted that, perhaps you’ll understand the origin of “smallworks”.

  50. “[A modern economist] is used to measuring the ‘standard of living’ by the amount of annual consumption, assuming all the time that a man who consumes more is ‘better off’ than a man who consumes less. A Buddhist economist would consider this approach excessively irrational: since consumption is merely a means to human well-being, the aim should be to obtain the maximum of well-being with the minimum of consumption. . . . The less toil there is, the more time and strength is left for artistic creativity. Modern economics, on the other hand, considers consumption to be the sole end and purpose of all economic activity.”

    “It is clear, therefore, that Buddhist economics must be very different from the economics of modern materialism, since the Buddhist sees the essence of civilisation not in a multiplication of wants but in the purification of human character. Character, at the same time, is formed primarily by a man’s work. And work, properly conducted in conditions of human dignity and freedom, blesses those who do it and equally their products.”

    “The most striking about modern industry is that it requires so much and accomplishes so little. Modern industry seems to be inefficient to a degree that surpasses one’s ordinary powers of imagination. Its inefficiency therefore remains unnoticed.”

    “Ever bigger machines, entailing ever bigger concentrations of economic power and exerting ever greater violence against the environment, do not represent progress: they are a denial of wisdom. Wisdom demands a new orientation of science and technology towards the organic, the gentle, the non-violent, the elegant and beautiful.”

    “No system or machinery or economic doctrine or theory stands on its own feet: it is invariably built on a metaphysical foundation, that is to say, upon man’s basic outlook on life, its meaning and its purpose. I have talked about the religion of economics, the idol worship of material possessions, of consumption and the so-called standard of living, and the fateful propensity that rejoices in the fact that ‘what were luxuries to our fathers have become necessities for us.’

    “Systems are never more nor less than incarnations of man’s most basic attitudes. . . . General evidence of material progress would suggest that the modern private enterprise system is–or has been–the most perfect instrument for the pursuit of personal enrichment. The modern private enterprise system ingeniously employs the human urges of greed and envy as its motive power, but manages to overcome the most blatant deficiencies of laissez-faire by means of Keynesian economic management, a bit of redistributive taxation, and the ‘countervailing power’ of the trade unions.

    “Can such a system conceivably deal with the problems we are now having to face? The answer is self-evident: greed and envy demand continuous and limitless economic growth of a material kind, without proper regard for conservation, and this type of growth cannot possibly fit into a finite environment. We must therefore study the essential nature of the private enterprise system and the possibilities of evolving an alternative system which might fit the new situation.”

    “The way in which we experience and interpret the world obviously depends very much indeed on the kind of ideas that fill our minds. If they are mainly small, weak, superficial, and incoherent, life will appear insipid, uninteresting, petty, and chaotic. It is difficult to bear the resultant feeling of emptiness, and the vacuum of our minds may only too easily be filled by some big, fantastic notion – political or otherwise – which suddenly seems to illumine everything and to give meaning and purpose to our existence. It needs no emphasis that herein lies one of the great dangers of our time.”

  51. “I think scale is not as important to the definition as originality.”

    Hm. The idea behind the hypertext browser was to make the reading of scientific publications easier: citations and references link to the original source and you can jump there. This is exactly how researchers and students used libraries for centuries: get a book, take a note of the more important citations and references, get those books. Thus Iit’s only the automatization of a well-known and quite mechanical human activity that was begging for automatization, exactly because it’s quite mechanical. Many, if not most innovations follow this pattern. Thus the ideas are hardly original – their technical implementation might be.

  52. >Thus Iit’s only the automatization of a well-known and quite mechanical human activity that was begging for automatization, exactly because it’s quite mechanical.

    I disagree with your implied theory. Sometimes reducing the cost of an activity makes qualitative changes in the surrounding system; there’s a good rule of thumb that this is likely to happen when the cost reduction is more than an order of magnitude.

    But even if I accepted your theory, you’ve missed a tremendously important thing about hyperlinks — that they can be passed around. They don’t just reduce the overhead of research, they reduce the overhead pf presenting your evidence to peers. That matters a *lot*.

  53. Facades for Innovation? Really? How’s Facades significantly different from NextSTeps user interface builder?

    I do believe though that OSS can innovate. I just don’t believe it can apply enough polish to make those innovations palatable to the end-user. (At least not without corporate sponsorship. Let’s face it, a lot of UI work is rather uninteresting – it’s not an itch too many people want to scratch)

  54. >How’s Facades significantly different from NextSTeps user interface builder?

    Well, for one thing, Facades isn’t a UI builder. It’s more like a UI architecture that makes the arrangement of a UI customizable on the fly. A better analogy to what you’re thinking of would be GNOME Glade.

  55. IB was truly an innovation. Facades is giving the user a knob they really shouldn’t have. There’s a reason why Apple disabled UI themeability when Mac OS X dropped, kids.

  56. ESR,

    a related topic: do you want to blog about what you think about the ideas of Richard Florida, that economy prosperity comes from creativity and creativity comes from tolerance? There is a big debate about it at the Hungarian Conservtive/Libertarian blog I’m contributing to and some people agree, while I and a few others think that f.e. when creative Dutchmen (Dijkstra, Van Rossum) move to the US, despite that it’s a less tolerant culture, is because of more capital. Creativity needs capital, and therefore capital accumulation is still more important for economic prosperity than creativity or innovation. For example, there are lots of Afrian students in British university and they can’t do much at home, because what good does it do if you become a really great web developer or aircraft engineer if there is no capital for a good Internet infrastructure or for aircraft factories? That’s what I think and I would be interested in your ideas, it would put this innovation topic into a bigger context.

  57. Eric,

    I just noticed this post, I’ve been busy.

    I completely agree with Saltation, you have missed my point by quoting two short sentences at random near the end of my post. My key point was that in all cases, developers need money to fund any innovation, open-source or otherwise. Tim Berners-Lee is a prime example, although in his case, it was government money and not corporate money, but that doesn’t make much of a difference. In the end, he could spend time on his invention only because he had a stable revenue. That only reinforces the point I made, namely:

    In the end, we all need to eat, we all need someone to pay us. It’s not that different in the open-source world, except maybe for a few lucky stars that are about as representative of the open-source community as Bill Gates is representative of the corporate programmer.

    I don’t see that your reply addresses this specific point, only unfortunate name-calling and ridicule (“blithering idiot”, “M’sieu de Dinechin”… talk about “screed”…) Well, help me find a generous open-source-friendly sponsor to fund XL and provide a free competitor to Simonyi’s Intentional Programming, and then maybe you’ll convince me. Meanwhile, I took the second best option, I got a job that indirectly funds it, if slowly…

    Your error number zero is to confuse ideas with innovation. Innovation is ideas turned into reality with hard work. Anybody can have an idea. Turning ideas into something that actually works is where the difficulty is, and where the funds are needed. When I look at my browser, I don’t see open-source innovation. I see an original idea by TBL, followed by years of painstaking corporate innovation, including fiber optics, routers, cheap Ethernet chips, fast microprocessors, high-resolution displays, I see Netscape, Microsoft, Cisco, Google, and who knows else… Implying that hackers built the World Wide Web is like implying that I built today’s 3D game market, just because I happen to have written the first game ever with immersive 3D on a micro-computer…

  58. >unfortunate name-calling and ridicule (”blithering idiot”, “M’sieu de Dinechin”… talk about “screed”…

    I quote from my original: “But, in fact, I think he (and others like him) are not idiots; they are reasonably bright people making a couple of serious and identifiable errors in their reasoning”. If you interpret this as ridicule, you have a problem with your reading comprehension of English.

    >My key point was that in all cases, developers need money to fund any innovation, open-source or otherwise.

    An assertion not actually supported by evidence, and falsified by numerous cases in which innovation in whatever way you define it has happened without funding. Talent does what it can; genius does what it must. You are articulating the assumptions of someone who is merely talented. I, on the other hand, have known geniuses; in the relevant does-what-it-must sense (which is that you can’t actually stop geniuses from innovating, no matter how hard you try), I may actually be one myself.

    >Implying that hackers built the World Wide Web is like implying that I built today’s 3D game market, just because I happen to have written the first game ever with immersive 3D on a micro-computer…

    The cases aren’t parallel, because the philosophical commitments made by those original hackers are still extremely potent in shaping the market around the Web. I speak from direct knowledge, because I was there and at least one decision I personally drove still shapes the Web today.

  59. Eric,

    My comprehension of “M’sieu de Dinechin” as an attempt to ridicule me is based on being a native French speaker. If this was not your intent, you should not have used pseudo-French. And what about the argument that I am unable to correctly compare BIND and Aqua? What with “hungry would-be innovators”? What with “nearsighted when not altogether blind”, with “simple and linear model of how innovation works”, and so on? And now you play stupid and blame my poor English reading comprehension? Who are you kidding?

    Regarding “does what it must”, you have a point if, like me, you have once found yourself in a foreign country, 9000km from home, without a job, on a restrictive L1 visa, with a family of 5 to feed, having lost all social protection back home, and if, with 3 days to go before being sent back home practically bankrupt, you were still pulling all-nighters to fight for your project and save it. Or if, like me, you have spent 15 years implementing a programming language that nobody cares about, just because you really really want a language where one can define pointers from scratch, or write a “maximum” function that doesn’t plain suck. Been there, done that, your turn…

    A word of caution, if I may: I would leave it to posterity to decide who is a genius. My own standards are pretty high. I worked with Frederick Raynal and negotiated with Bruno Bonnell in my late teens and early 20s. I met Dave Packard and Bill Hewlett before I was 30. I have personally worked, in a couple of cases for years, with folks who brought up practically every single OS there is on Itanium (HP-UX, Windows, Linux, OpenVMS, FreeBSD…) My friends include the primary maintainer of GCC and a co-founder of VMware. I’ve discussed technology with Avie Tevanian, Bertrand Serlet, Mendel Rosenblum, Phil McKinney, Susie Wee, and each of them is really pretty smart. I even had a memorable e-mail dispute with Alan Kay about the limits of SmallTalk, so you would not be the first self-proclaimed genius I happen to disagree with.

    Now, if you still want to keep lecturing me about who’s a hacker and who is not, here is a tidbit you should keep in mind. To the best of my knowledge, there are less than 30 people on this planet who can claim having designed a successful operating system entirely from scratch, having written its first line of code, or having watched its very first burps. I’m one of them, and the “children of my brain”, to use your expression, run many of the largest systems of the largest computer company in the world.

  60. “To the best of my knowledge, there are less than 30 people on this planet who can claim having designed a successful operating system entirely from scratch…”

    This is not true, although I suppose that it’s quite possible that there are *fewer* than 30 who can claim that particular distinction.

  61. “# Russell Nelson Says:
    November 9th, 2008 at 12:41 am

    @JimThompson: what’s the launch speed of a pig? Maybe one of them *can* fly, and it’s just a question of getting enough of them up to speed to launch the one that can fly? Like a scramjet?”

    Shouldn’t that be “Like a *ham*jet?” :-)

  62. What I see is a lot of people I respect talking past each other.

    Not surprising, really. And after things cool down again, this particular small, but “open” pot of idea stew will have produced its share of innovative product in the minds of both participants and spectators. Some of that innovative product (new ways of thinking, okay?) will be good and some will be not-so-good. Life as usual. Flame wars seem to be useful for heating the pots, although the temperature does get a bit high at times.

    This so-called conflict between the “open” and “closed” ideologies has been continuing since, well, since Adam and Eve and Cain and Able. But it’s just another spurious paradox.

    Opening up to new ideas is important. Otherwise, none of us would ever sleep, and those who did would never wake up.

    Shutting out the distractions is important to cleaning up the new ideas and making them useable for others (and even for oneself, ten years down the road).

    Historically, over the past century, we have used contracts, patents, and copyrights to shut out the distractions. The present trend called “open source” points towards a better way (or points back to the way that patents and copyright in US law were originally intended, possibly).

    Once the invention is cleaned up and made generally useful in some sense, opening it up again allows it to remain valuable and viable, and that is part of what the “open source” processes point to.

    But it’s a cycle, like many other cycles that operate in our world.

  63. Generic,

    ESR
    1. I speak from direct knowledge, because I was there and at least one decision I personally drove still shapes the Web today.
    a. And what would that be?

    Even Linus T. can’t say he wrote Linux. Multitudes have invested their knowledge into that software. Nobody is solely responsible for any software this day and time. If you compiled it then you should thank the thousands that contributed to gcc, yacc, vim..kate, bash, intel, S3, …

    The only innovators in this world are the ancient men who picked up the stone struck the first blow to oblivion. ugg ugg

  64. >a. And what would that be?

    I was the guy who argued successfully, in 1983, that the “classic” .com/.org/.net/.gov domains shouldn’t be killed off in favor of the rigid geographically-oriented system the IETF had been planning. I happened to be at the meeting where the change was to be finalized, and I asked “What happens when people *move*?” One other person, who I think may have been Henry Spencer (I’ve asked; he doesn’t remember) took my side. We talked the room around. Only took about 90 minutes; those IETF guys were way too smart not to notice when they’d been whacked with a clue-bat.

    Thus, I’m arguably responsible for the whole domain-name gold rush, and cybersquatting, and all of that. :-)

  65. all of this name dropping and I see two people that both need to work for a living. Neither of you have written anything worth paying for. I’m not going to say whose a genius and whose not, but I’d bet the guy that wrote Snood is probably smarter than both of you.

  66. >Thus, I’m arguably responsible for the whole domain-name gold rush, and cybersquatting, and all of that. :-)

    Sir for this mess you are willing to take full credit ?

  67. >?I was the guy who argued successfully, in 1983,

    So at an earlier age you didn’t want to work with others ?

    I am not a educated person such as yourself. I am American born and served.
    I think you are a smart person. I also think you should also use that brain to
    do better not just make yourself look or appear better.

    Yes my IQ is above 100
    which means nothing.

  68. >So at an earlier age you didn’t want to work with others ?

    Au contraire. In 1983 I wanted to work with others, and did so effectively enough that I was able to turn the IETF’s design plan for DNS around. Most of the credit goes to the IETF guys of that era for being open-minded enough to realize that sense was coming out of the mouth of a junior-grade geek nobody had ever heard of before, but part of it stays with me for pointing out their huge and basic error in a diplomatic way.

    >Sir for this mess you are willing to take full credit?

    The messy aspects are side effects of the decision to keep an important part of the domain namespace autonomous from physical geography. That was a good thing; the domain-name gold rush was an unavoidable side effect. I’m pretty sure I did the right thing, though I’ve occasionally doubted it.

  69. Can I just say that I myself played no part in any of these momentous events in history? Though somehow that doesn’t make me feel so very inferior to those who did, when I read what they are writing in the comments above.

    But anyone who doubts the importance of ESR’s work needs to go and read his writing again. Sure, it’s oversold. Sure, he’s a little vague on the down-sides, and plays down that transformative change tends to be good for society but bad for the investors naively making it (consider how many of the early telegraph and railway companies went bankrupt) – a tricky one to square if you are a libertarian who read Heinlein’s “Farmer in the Sky” as a kid. But all the same ESR would have made a major contribution to the development of computing just by publishing his essays, even if he’d never written a program in his life.

    If it comes to historical authenticity, Fred Brooks told me last year (oh, I get around) that “The Cathedral and the Bazaar” was one of the most important books of our time. I’m nobody much, but I agree. (And if everyone who had read “Code Complete” had also been made to read “The Art of Unix Programming” as a counterpoint, we’d have a lot more versatility in the coding world than we have now.) So can we stop telling ESR that he’s a n00b now?

  70. >If it comes to historical authenticity, Fred Brooks told me last year (oh, I get around) that “The Cathedral and the Bazaar” was one of the most important books of our time.

    Really?

    Wow. He’s never gotten around to telling me that. Cool!

    This is especially meaningful because, in a lot of ways, I wrote CATB as a direct reply to The Mythical Man Month. And it was intended as a respectful one. Um, under what circumstances did you have this conversation?

  71. While I agree with esr that much more innovation could theoretically come from open source, I don’t think it does in practice and I agree with Saltation that much more of today’s innovation comes from proprietary software. The problem is the supposed Error the zeroth, most people are not going to bother to innovate if they’re not getting paid for it (esr doesn’t really justify his 0th error in the original post, he simply points out that corporations are almost as stultifying). Let’s leave aside the argument over esr’s examples of open source innovation and the few geniuses who HAVE to innovate: the record is clear over the last 10 years that more innovation has come from proprietary software, despite the fact that corporations are often highly risk-averse. The problem is that most open source software has no business model, it has to rely on the few who profit from open source software- Red Hat, IBM, Google- funding further development.

    I think the ultimate solution for software development will prove to be a blend of the two models, open and proprietary. The MPL family of licenses allow a mix of open- and closed-source code; I think this will prove to be the winning solution, allowing the openness to innovation and external input of open source and the property rights and money of closed-source. For example, one can write closed-source hardware drivers for the Solaris or FreeBSD operating systems and provide the binary drivers alongside the open-source OS. I don’t know why nobody does this today, it seems so obvious. This is why I have been saying for some time that GPL code will all die out within a decade, when the transition to mixed-license code happens, as the GPL doesn’t allow such mixing (I’ve already stopped using GPL code as much as possible as I don’t want to use tools that will soon be obsolete).

  72. >the record is clear over the last 10 years that more innovation has come from proprietary software, despite the fact that corporations are often highly risk-averse

    This isn’t at all clear. What do you think you’re measuring, and how?

  73. >A word of caution, if I may: I would leave it to posterity to decide who is a genius.

    Quite. That’s why I said “may be one myself”. There are complicated definitional questions…

    I am not really able to extract anything else other than resentment from your comment. I think you took my essay far too personally. I didn’t write it to attack you, though I admit to using you as a foil as I pounded on several serious mistakes that could be taken to be implied by what you said. Nor am I going to play who’s-dick-is-bigger with you; that would just make both of us look petty.

    I’m sorry you’ve had bad experiences trying to do innovation on a shoestring, but since I’ve actually done that fairly routinely I’m not really very interested in hearing anyone else complain about how tough it is. Job security is always available to anyone as bright as you or me if we’re willing to do some boring, artistically non-fulfilling shit to get it; therefore, neither of us has any business acting martyred or put upon if we choose to follow our bliss and wind up far from home, jobless, and bankrupt. Make your choice, live with it, and don’t whine — whining just diminishes you.

  74. Clearly with innovation it is difficult to measure what’s going on as it’s somewhat subjective what innovation even is. Still, almost nobody would argue that more innovation has come from open source software over the last 10 years, because the record of proprietary software being much more innovative is clear to see and it is easy to see why this would be so. While open source doesn’t necessarily suffer from the politics and bureaucracy of a business organization, the money to be made from closing up your code far outweighs an open but cash-poor environment in encouraging innovation. However, my problem is that there’s not much innovation going on overall, when one combines all the proprietary and open source innovation together. I think a mix of the two approaches, along the lines I suggested, will lead to much more innovation overall and will prove to be a business model that both extremes of completely open or completely closed software cannot compete against. I would like to see much more money from sustainable business models plowed into more open forms of development, because I think this will lead to the most innovation. I think the mixed-source approach is the best solution to that.

  75. >Sure, he’s a little vague on the down-sides, and plays down that transformative change tends to be good for society but bad for the investors naively making it

    I had to think about this for a while, and go back over my old papers to check. It turns out that I issued some clear warnings to early investors:

    (1) I observed that none of the nine business models I described had much of a track record.

    (2) I warned that I expected the investor multiples of open-source software comnpanies to drop from levels typical of manufacturing firms to levels typical of service firms.

    (3) I predicted that more and more software providers would be structured like legal and medical (skilled-service) practices, which tend to do well for partners but generate no investor returns at all.

    Back in the boom days, my audiences seemed curiously unable to absorb these caveats….

  76. >record of proprietary software being much more innovative is clear to see and it is easy to see why this would be so.

    Could we have some actual demonstration, rather than bland assertions? Tell me what you’re counting as innovation and why you are counting it.

  77. Nope, I’m uninterested in getting into a pissing match about something subjective like this, particularly when almost anybody who knows the subject well would agree with me. Many people may use the internet for such subjective flame-wars but I’m uninterested in this particular one. ;) If you want to criticize what I’ve said, please do so about my mixed-source proposal, for which I’d be glad to have some feedback.

  78. Apologies to all who rightfully wrote “shut up, both of you!”. You are right, guys. I’m not particularly proud of what I wrote above. But then, there’s no undo command for the Internet :-( I was angry mostly because I found ESR’s ad-hominem argument, namely that my comments were worthless because he’s famous and smart, to fall way short of my expectations… After all, he is famous and smart. By the way, not to contradict Todd or anything, but evidently a lot of people have paid for ESR books. Believe it or not, some even paid for my stuff.

    Anyway, I was not attempting to compare the worth of open-source and corporate innovation, and even less of open-source and corporate developers. My point was not that there is no innovation, but that there are too many ulterior motives behind open-source innovation, so that the desire to share rarely appears to be the primary motive. In my personal case, what ESR interpreted incorrectly as “pay me and I will innovate and share” really is “I want to innovate and share, but I’m bound by financial constraints”. Some people may have freed themselves of such financial constraints. I’m glad ESR found an arrangement with his wife that puts him in this category. But I reckon that these lucky bastards are a tiny minority.

    – Contributors on their free time have to depend on some other source of income. This limits how much time they can dedicate to open-source. So they innovate, but at snail pace compared to what they would be capable of. That’s the category I’m in for most of my open-source work.

    – Corporate contributors work on the corporation’s agenda. Innovation in that space may end up being open-source, its motivation is still to benefit the corporation paying for it. A small Linux kernel patch I submitted recently falls in that category, and so does more significant work such as btrfs, Linux kernel scalability, porting Linux to Itanium, … Linux, Python and Perl are so successful largely because corporations make money with them, so they pay back in their own interest.

    – Universities often release their work as open-source, but again open-source is not the primary driver. If the inventors can make it big by going corporate, they often do, to wit VMware or Sun.

  79. Talent does what it can; genius does what it must.

    And Aperture Science does what it must because it can.

  80. Could we have some actual demonstration, rather than bland assertions?

    Did Linux change the world?

    The Mac did. Everything we know about UI design in the modern era we owe to work done at Apple. Not Xerox, Apple. They had been working on the problem since before encountering the Xerox team, licensed the interesting bits from Xerox legitimately, and improved on them substantially.

    Multimedia computing exists almost solely due to the emergence of the Amiga in 1985. It was the first affordable home PC to sport hardware graphics acceleration, multichannel sound, and capability to interface with video hardware.

    Hell, I think Windows 95 changed the world more than Linux; after its release the bulk of commodity desktop users could get Mac-like ease of use and easy internet access. Most of the technologies that enabled today’s “Web applications” to work appeared in Internet Explorer several years before the competition.

    Linux enabled programmers to question long-held assumptions about the software development process, but it did nothing to challenge the way we use computers at the end user level, where the rubber meets the road. By the mid-nineties most of the Unix variants were almost completely fungible with respect to one another[0], their prime variation being in which make of expensive proprietary architecture they ran on. Linux has lain waste to this landscape due to its low price, but on the desktop it can’t even get a user base above statistical noise, let alone drive Microsoft’s share below 90%. (It’s lower than it used to be, but the difference is almost entirely made up of people who switched to Mac.)

    Proprietary software companies have achieved several paradigm shifts within our lifetimes. Open source has barely managed one (arguably, the transition from standalone desktops to a networked world, but the real enabling technologies at the end user level were proprietary, and many still are). That’s what Christophe means when he says more innovation occurs at the proprietary level.

    [0]IRIX was a possible exception due to its OS-level support for cool graphics. But I have it on good authority that a next-gen Amiga would have absolutely blown away anything SGI was doing had it not been for gross mismanagement and corruption in Commodore’s upper echelons.

  81. Proprietary software companies

    And hardware companies with an interest in selling software that worked closely with their hardware.

  82. actually, i’d take your observation back half a decade but on similar basis.

    MS Office 1.0 changed the world more than Win 95 did. it created for 95+% of the world’s computer users (later, and as a consequence, and for over a decade, 98+%) the bulk of the most obvious parts of the mac interface in everything they NEEDED to do.

    the upside opportunity of the fuller interface they never saw, so never missed. the same goes for at least 90% of today’s mac os x users.

    and so here we are.

    all using the same 1990-vintage interface.

    woo.

  83. Apple’s about to revolutionize the industry for the nth[0] time in its 35-year lifespan with its tablet device. Despite several abortive attempts by Microsoft and others (and at least two attempts with the collaboration of the open source community: the PepperPad and the CrunchPad), the concept never really took off. Now it will.

    The rest of us just play the game; Apple changes the rules and the field. That’s innovation.

    [0]n is at least 4, but depending on what you’re willing to count could be 5 or 6.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>