Back in 2012, Poul-Henning-Kamp wrote a disgruntled article in ACM Queue, A Generation Lost in the Bazaar.
javporn
It did not occur to me to respond in public at the time, but someone else’s comment on a G+ thread about the article revived the thread. Rereading my reaction, I think it is still worth sharing for the fundamental point about scaling and chaos.
There are quite a lot of defects in the argument of this piece. One is that Kemp (rightly) complains about autoconf, but then leaps from that to a condemnation of the bazaar model without establishing that one implies the other.
I think, also, that when Kamp elevates control by a single person as a necessary way to get quality he is fooling himself about what is even possible at the scale of operating systems like today’s *BSD or Linux, which are far larger than the successful cathedrals of programming legend.
No single person can be responsible at today’s scale; the planning problem is too hard. It isn’t even really possible to “create architecture” because the attempt would exceed human cognitive capacity; the best we can do is make sure that the components of plannable size are clean, hope we get good emergent behavior from the whole system, and try to nudge it towards good outcomes as it evolves.
What this piece speaks of to me is a kind of nostalgia, and a hankering for the control (or just the illusion of control) that we had when our software systems were orders of magnitude smaller. We don’t have the choice that Kamp wants to take anymore, and it may be we only fooled ourselves into thinking we ever had it
Our choices are all chaos – either chaos harnessed by a transparent, self-correcting social process, or chaos hidden and denied and eating at the roots of our software.
Poul-Henning Kamp, not Kemp; though I fear there’s other comments in the queue about this.
I haven’t read the piece in question, but I think the primary role of a project “dictator” is to impose an aesthetic vision wherever possible. Think Torvalds and Git.
Apropos the things you never responded to: have you seen (and if so, do you plan to respond) to this blog post from Jeff Atwood? http://blog.codinghorror.com/given-enough-money-all-bugs-are-shallow/
Control is an illusion. The faster one comes to understand that in life, the better. Look at anyone’s life. Who you married, what your kids are like, the city you live in, who you work with, your job itself. Honestly looking back, some of the most critical influencers for major life direction are at the core, random. Why would large software be any different?
Nit: “Back kin 2012” should be “Back in 2012”; and there should probably be some punctuation and a space after ‘ACM Queue’.
I don’t see how autoconfiscation can possibly the fault of the bazaar; isn’t it an example of GNU software handed down on stone tablets by some guy with a long beard?
It’s possible Kamp is being confused by the undoubted success of BDFLs as bazaar leaders (Linus and GvR come to mind) and thinks that their kind of control is just “what a cathedral would do, only less so”.
But then I haven’t read the article, maybe I’m spouting nonsense.
Also, I can imagine Eric writing a deeply analogous criticism of an Ostalgie-piece from an East German hankering after the illusion of control in economic systems. Except, there’d probably a bit more vitriol, since programming cathedrals don’t tend to send people to the Lubyanka.
The single benevolent dictator pulls order out of chaos when it works right. Um… The dictator doesn’t control, he chooses?
Our choices are all chaos – either chaos harnessed by a transparent, self-correcting social process, or chaos hidden and denied and eating at the roots of our software.?
A better description of markets vs. “planned” economies could hardly exist.
I loved that ACM Queue article, but then, I liked what Poul Henning-Kamp (Kemp?) wrote about HTTP/2 woes on the IETF list serve, in February 2015 or so, too. Yes, I realize that ESR is the founder of The Cathedral metaphor (I think?), but it has always bothered me, to the extent that it crosses over with nouveau communist “free! free!”
Here’s an example of why I am unconvinced about the virtues of GNU and FOSS and open source for everything. I love SAS, which is licensed software. There is great documentation and it works, really well! Yet whenever I praise SAS, people who make millions of dollars a year doing high frequency trading, or working at hedge funds, say, “Oh, SAS is too expensive. Use R.” SAS is NOT that expensive! IBM mainframes are NOT that expensive. SAS is licensed by the number of CPUs and IBM mainframes cost less than $75,000 now (although maintaining them is another matter). I am not here to shill for SAS and IBM though. I feel guilty even bringing them up, but ESR is tolerant…? My point is that it seems like developers and computer scientists live in one world, and everyone else who uses computers and software lives in another, with the latter being Microsoft and closed source.
I think that you just got lucky with Free BSD and Mozilla Firefox and Red Hat as examples of successful open source, collaborative projects that produced wonderful things. For a short while, there was a critical mass of highly motivated, highly skilled people who were willing to give of their time and brilliance without being paid directly for it. The same can be said for Wikipedia. There do exist people who like to code and do math, physics or other applied sciences for fun in their free time! I think that you are coasting now, on what was achieved before. Mozilla is a case in point. After their former CEO was witch-hunted out of his job, I haven’t seen promising signs for the future of Firefox. Google Chrome browser is just eating up everything. Google is closing down Google Code for hosting open source projects. Here is a post by a nice man about open source not being profitable:
http://kellblog.com/2014/11/18/it-aint-easy-making-money-in-open-source-thoughts-on-the-hortonworks-s-1/
Please, do NOT brigand his blog with hostile comments! I always feel horribly guilty when I cause that to happen with Bitcoin posts on reddit.
Anyway, when I think of the metaphor of The Cathedral and The Bazaar, I think of Carl Jung saying that the modern fell off the roof of the cathedral, into the abyss of the self. I also think of skilled tradesmen, most of whom still follow the path of guilds, from helper to apprentice to journeyman to master, be they carpenters, mechanics, plumbers or electricians. There’s a lot to be said for that, and it is definitely Cathedral-style. It makes me want to tell all the programmers and computer scientists and applications developers (that’s me, for statistical and data analysis), “Get back in that Cathedral and do things right! No crufty code or cargo cult programming. Make sure everyone gets paid market wages for their work! This isn’t the USSR.”
I’m back.
@DVK I just read Coding Horror’s post. It isn’t bad, although it is too long and a little wishy-washy. I would be curious what ESR would have to say about it, although that would require a lot of writing to respond point by point (maybe… unless he can formulate some grand general case response ;) After being annoyed by autocratic moderators on Stack Overflow too many times, I am compelled to mention that Coding Horror’s own Discourse is kind of awful, at least the way it is implemented on his blog comments. There are LOTS of good comments though. Just be prepared to pull out your hair as you try to make any sense of the thread order while scrolling wildly. NB Discourse is open source.
@Edward Cree Squeeeeeeeeee! You look so cute!!! Please write more fun comments.
@Ellie Kesselman
“I think that you just got lucky with Free BSD and Mozilla Firefox and Red Hat as examples of successful open source, collaborative projects that produced wonderful things.”
And with LaTeX, and R (it is the current standard), and Apache, and Python, and Perl, ….
@Ellie Kesselman
“For a short while, there was a critical mass of highly motivated, highly skilled people who were willing to give of their time and brilliance without being paid directly for it.”
Sorry, but there are still a lot of brilliant people who are willing to give their time and brilliance to make the stuff they find important. The point is that at a certain level of “brilliance” (whatever you mean by that), money is a bad motivator and hierarchical control is a deterrent.
But I think our host has written more eloquently about this.
So basically, he doesn’t like autoconf and wants the dot-com kids off his lawn
Besides what Winter says, open source isn’t even dependent on folks being willing to give their time and effort for free. Many open source projects have been successfully crowdfunded on sites like Indiegogo. Also, it’s well known that most contributions to the Linux kernel (and possibly other core infrastructure) are paid professionals, not hobbyits.
Also, R may be crappy (certainly so from a language-design POV), but I can’t believe that people are actually defending freakin’ SAS. That’s just noise.
I’ll defend SAS, from a different perspective: It’s the mainframer’s Swiss-army chainsaw, and was well before Perl was a gleam in Larry Wall’s eye. You can do just about anything to anything on MVS-style operating systems with SAS. In particular, it’s *the* language of choice for analyzing MVS system performance data, something its designers had no inkling it would become when it was first built.
@ESR
Since I’ve read Kamp’s article sometime after discovering your blog and else, I’ve been wondering if you had read that article and what your answer would be (perhaps I was the one mentioning it on G+?). I confess I expected something a bit more elaborated, but at least now I don’t need to wonder.
On the article itself, one of the major defects I found is that he takes a single distribution (FreeBSD in this case) and extrapolates its problems to the whole Bazaar concept. While he rightly complains about the dependency hell, he doesn’t acknowledge how other distributions handle it wonderfully (for most cases at least).
Also, I think he misses the other very important aspect of the Bazaar model: the freedom of choice. Most of the problems he points out (libtool being the main example) are, in my opinion, result of the freedom of choice people have when using Unixes in general and OSS in particular.
guest: “open source isn’t even dependent on folks being willing to give their time and effort for free. Many open source projects have been successfully crowdfunded”
Not to mention Bruce Perens’ point that for the things you are doing that are not your unique selling point, it makes economic sense to collaborate with others rather than roll your own.
@Winter
>The point is that at a certain level of “brilliance” (whatever you mean by that), money is a bad motivator and hierarchical control is a deterrent.
I think this is a fairly eloquent way of putting it, actually. I think you’ve put it correctly that it is not really a general feature of human behavior but a feature of the very small percentage of people who can code well. I think you’ve put it correctly that they dislike corporate jobs.
But you aren’t 100% right with money. It is not that money motivates them, it is more like they still want to live a comfortable living. There are quite a lot FLOSS projects where there are no hardware sales or consulting options, and basically zero money made, and I think the authors have another job somewhere to pay bills. These ones would benefit from a corporate sponsor.
I have always admired ESR’s rather carefree attitude to getting a regular paycheck (if that is the good way of putting it) but I know I would not have this courage. I know I’d want to sign contracts that promise a livable salary and not just hope things will work out financially in the longer run. This is not really as much as money being a motivator but more like being worried about unsure finances being a stressor and stress is a demotivator.
Another vote for SAS – it’s one of the few companies that seems to understand that R&D + support are what counts. They plow something like 30-40% of profits back into software R&D, and they’ve finally figured out the pricing model for higher ed (where I work). Support is unlimited for an annual maintenance fee – first year fee is 2-3x the following years because they figure most of the support will be the first year.
The CEO Jim Goodnight is also a programmer. Don’t know if I would call him a hacker, but last I heard he still codes the stat PROC GLM.
THIS. Open source philosophy appeals to college students and those who can afford to live like college students. When you are programming to put food on the table, your attitude tends to shift abruptly, and suddenly those Microsoft development tools and operating systems don’t seem so bad. (Especially when the increased productivity Visual Studio gives you means you can impress the boss much more easily than you could have fucking around with Makefiles and m4 scripts.)
It’s kinda like with music. Amateur musicians sitting on huge trust funds, or those who have already enjoyed enormous success, may favor Napster and other disruptive distribution models but when you hope to feed your family with your music I guaran-fucking-TEE you will come down on the side of universal unobtrusive DRM for digital distribution. Taylor Swift’s decision to close her catalog to freewheeling digital outlets like Spotify is not the mark of a bitchy diva, it’s hard-nosed business sense.
“When you are programming to put food on the table, your attitude tends to shift abruptly, and suddenly those Microsoft development tools and operating systems don’t seem so bad.”
No, they still suck…it’s just that starving sucks worse.
universal unobtrusive DRM
Pull the other one, it creates a selfless benevolent god-bureaucrat.
Do you have kids, Foo? Do they play commercial computer games? And if so, do they still buy them packaged or download them through Steam?
I bought Civ 4 on Steam. Two computers ago, it stopped working under Wine (the Steam client itself was fine, but there was some problem launching Civ 4 itself). A cracked download later, I have a copy that runs just fine under Wine, no Steam required.
Not to mention moving goalposts. The entire point of DRM is to be obtrusive: I want to be able to play my music on the platform of my choice, not select from one or a handful of officially blessed platforms. Games are inherently significantly less portable, though as my experience shows, the DRM or something indistinguishable from it can bork even stuff that would otherwise work cross-platform.
You’re both right and both wrong.
@Jeff Read
“When you are programming to put food on the table, your attitude tends to shift abruptly, and suddenly those Microsoft development tools and operating systems don’t seem so bad.”
I do not know the situation is in the States, but over here there is a whole universe between your kids starving and flying your own private jet plane and entering yacht races.
For instance, I think Andrew Tridgell is doing fine working on Samba.
Wanting a lot of money is more of a status thing for people who cannot show their worth in other ways. People like Tridge do not need a car to show off.
Even in the States there are a whole lot more people like myself and Shenpen, who really are programming to put food on the table, than there are like Bill Gates and Larry Ellison — neither of whom made their fortunes through coding. (When Bill Gates wrote his last line of code, Microsoft was still a smallish company that catered to a niche of microcomputer hobbyists — the real computer industry was centered around the mainframe and mini markets.)
@Jeff and Jay: ““When you are programming to put food on the table, your attitude tends to shift abruptly, and suddenly those Microsoft development tools and operating systems don’t seem so bad.”
No, they still suck…it’s just that starving sucks worse.”
So true. We will do things for our family that run counter to our ideal conditions….
This Libertarian took a job with the (shudder) government to keep his little girls from having to move around the country on the contracting merry-go-round. My friends still mock me relentlessly over the irony.
<i.No single person can be responsible at today’s scale; the planning problem is too hard. It isn’t even really possible to “create architecture” because the attempt would exceed human cognitive capacity; the best we can do is make sure that the components of plannable size are clean, hope we get good emergent behavior from the whole system, and try to nudge it towards good outcomes as it evolves.
What does this imply for the development of artificial intelligence?
I’d like to toss my spare change into the mix. Over the past few years I’ve become very disillusioned with the “quality” claim of Open Source software.
I work for a company in the enterprise data storage industry. Standard disclaimer: these are my comments, not my employer’s, not that it matters much. You know it’s in the enterprise space because we charge 4x as much. I do most of my coding in-kernel working without protected memory in multi-threaded, multi-processor, multi-system coordinated code.
What’s interesting is watching our internal process and comparing that to the experience that I’ve had with my Ubuntu Linux desktops at home and at work.
1) My home desktop has a RAID-1 configuration using the Linux RAID tools. Upon doing a minor kernel version update I stopped being able to mount the root filesystem. Just a kernel panic and message about no rootfs. Boot the previous kernel version and everything worked fine. Booting into recovery mode allowed me to mount the rootfs with no special options “mount /dev/md0 /mountpoint” worked fine. It took me about a year to finally get some resolution on the issue. It turns out that many major releases ago I’d tried using LLVM and I didn’t get rid of that correctly. The md code was silently ignoring the strangeness and mounting until it decided to silently not mount any more. This is a *horrible* supportability issue.
Where I work we have a lot of different mechanisms to allow for debugging of issues like this. Every feature implemented is required to prepare a plan for how it will be supported, and how we will get out of error conditions.
2) Configuration management. Dear God, this is *horrible*. Eric, if there is one thing Linux/OSS needs, it’s a good tool to manage configurations across software updates.
As a part of using my machine(s) for some years, I occasionally make changes to the system configuration files. Then when I install a new version of the software (or, worse, update the whole OS) the solution is to either discard my whole existing configuration, keep my existing configuration, or throw a 3-way merge at me. This wouldn’t be bad if the only changes I needed to contend with were my 2 lines of change and the version number listed in the default config file. Unfortunately, in the intervening time the default config file was re-indented, had the lines re-sorted, comments updated, so a 3-way merge is basically useless at this point. Here’s an idea: Something already has to parse the existing version. Have that parse the existing version and have it emit the code in the new “correct” format. Or something. If I’m doing a whole OS upgrade, I don’t have time to re-learn the Apache or sendmail configuration format so that I can continue with the upgrade. Do the Right Thing, dammit.
At my job we plan to handle things like unplanned software upgrades in the middle of operation. Imagine your database software crashing, and restarting with a new version, which now has to do transparent recovery and start serving data in a timely manner. We actively test this. We change complete backend configuration architectures and transparently handle issues. I’d point out we also test downgrading, too (non-transparent operation). Type a few commands, get back the set of steps you need to perform, and once you’re told things are good you can safely revert to the next-oldest version of the product.
3) Limited testing environments. My work desktop is attached to a network and mounts my home directory via NFS. No local disk is available to be written by user applications (my config choice). It just so happens that the network switch port my computer is connected to is a little … off. When the switch reboots, it randomly decides whether to be nice or not. When “not”, it drops or corrupts a very small number of the packets going over the wire. No biggie – TCP and NFS are both resilient. Which is true. I’ve never experienced a data loss.
However, my desktop is KDE. KDE likes to parse config and status files all the time. A momentary hickup means that my text editor stops being responsive. Typed text in my terminal application doesn’t display. At least for several seconds. This breaks workflow and is incredibly annoying for an interactive user. Somewhere between the kernel NFS implementation and KDE (and, perhaps NIS), I’m slowly being driven mad.
What’s the actual cause of UI latency? I don’t know. I don’t have any good stats or counters. I can look at packet traces, but they only tell me how long an operation took, not whether they were blocking the View of the application.
I think any discussion of the future of the proprietary/open/free software debate needs to take a hard look at what, for lack of a better term, I’ll call “mod culture”. The kids these days (I’m speaking specifically now of current young teens and of course overgeneralizing) think closed software is acceptable but mostly uninteresting, or reserved for minor tasks. *Interesting* software should be ‘modable’; they don’t much care if it’s proprietary, but they’d like a large and active use community making significant modifications to functionality (“mods”).
Minecraft is the key case here (vanilla Minecraft is a tiny fraction of the user base AFAIKT); Kerbal Space Program is another interesting one. KSP, while proprietary, was heavily modded while still in alpha, and the developers have embraced this. At this point they’re really using the modders to explore design space, then cherry-picking the most broadly applicable and successful (even, in some cases, hiring mod developers as permanent or contract devs to integrate their work with the main game).
These are mostly games, of course, but it’ll be interesting to see the attitudes of the group growing up with this cultural backdrop.
>These are mostly games, of course, but it’ll be interesting to see the attitudes of the group growing up with this cultural backdrop.
Not hard to predict. They’ll see the sense in the open-source prescription when they find the limmiits of moddability.
Anyone claiming that central planning is inherently superior has to answer two questions to convince me: What influences do you believe caused the Soviet Union to collapse? and Why were the Soviet governments unable to counteract this?
(The first one really is a null question: it’s the second — how did central planning fail? — that maters.)
> when you hope to feed your family with your music I guaran-fucking-TEE you will come down on the side of universal unobtrusive DRM for digital distribution.
Uh, no. Even Apple’s iTunes Music Store is DRM-free these days. The thing is, for non-superstars (and Taylor Swift arguably qualifies as a ‘superstar’ in this context), music making should be seen as a semi-professional endeavor at best. And once you accept that, crowdfunding models like Kickstarter/IndieGogo and Patreon suddenly look a _lot_ more promising than just throwing your music on a DRM’d distribution platform and hoping for good-enough sales volumes.
Jeff Read schrieb:
It is to laugh. I have a nine-to-five (well, ok, eleven-to-seven) programming job to put food on the table. Know what I work on? A Linux kernel driver and its automated tests. Turns out, the improved productivity Python and scriptable Unixy tools give me are quite sufficient to impress the boss. (And we don’t touch fucking m4. It’s not 1994 anymore Jeff.) There’s plenty of money in writing open-source software, which is not surprising since — as it’s the more efficient mode of production — the market naturally favours it.
As for music, if you want to make money from it, find a patron and write what he likes (it worked for Mozart…). If you want to be unfettered artistically, get a job and make music in your spare time. Don’t expect the world to subsidise your desire to be an artiste; if they don’t want to support your business model, that’s your problem. (This is not me philistinically sneering at music; I’m an amateur musician and composer myself. But I don’t expect to make money that way, nor would I even if I were good at it.)
Eric, I know this is completely off-topic for this thread, but:
You may recall that I mentioned on G+ a while back that UKIP seemed to be showing an interest in bringing back RKBA in the UK (they’re the nearest thing we have to an electorally credible libertarian party). Well, I was at a public UKIP meeting this evening (in Cambridge) and I asked the candidate (Patrick O’Flynn) about UKIP’s policy on firearms; the answer was more-or-less “not interested”. I also felt like the rest of the room were all looking disapprovingly at me for raising the topic. On the other hand, a few people did speak to me afterwards in rather more positive terms, so it seems there may be some support for RKBA, just not much.
Just thought you might like to know the state of gun politics in (one city of) the UK.
Do you have kids, Foo? Do they play commercial computer games? And if so, do they still buy them packaged or download them through Steam?
No, I do not have kids. But I have played a few games. One notable case was an RPG that never ran properly, till I tried the GoG version which doesn’t have DRM. Strangely enough it worked immediately.
Also you are nuts if you think that typical musicians get their money from album / song sales. They get it from selling shirts, buttons, stickers, etc. at concerts. Bonus: any albums the band sells had to be purchased from the record company. The internet caused destruction of the record companies can’t come soon enough.
How do software engineers/programmers get paid working with Open Source software? (that is, outside the Software as a Service model: I understand how that one works, I just loathe it as a user.)
So basically, he doesn’t like autoconf and wants the dot-com kids off his lawn
I’m beginning to hate autoconf too. Is it bad that whenever autoconf/configure barfs on me, I go find one of the example makefiles, get the gist of how things are supposed to compile, then compile and link things myself using a python script to call compilation steps? (Which I wrote after losing a day trying to get make to call the compiler in a way that I was explicitly telling it to, and having it mysteriously ignore me and do something else?)
SAAS isn’t really all that open, or it wouldn’t be salable either. SAAS programs are usually hiding on someone’s server, and are only open to the developers.
Why do people hand your organization money if your organization’s business model involves developing software they are giving away?
I’m genuinely curious. I hear a lot of anecdotes from people who have apparently made a lot of money at this, but it isn’t happening by any mechanism I really understand.
ams asks:
Answer the first: read Eric’s paper “The Magic Cauldron”.
Answer the second: in my case, it’s the “Widget Frosting” model from that paper. The company I work for (Solarflare Communications) designs (somewhat specialist) Ethernet cards. (Both the silicon and the boards are designed in-house but manufactured elsewhere.) Obviously any value created by writing software to exploit capabilities of the hardware increases the value of that hardware, and thus the company can capture the returns without charging one thin cent for the software. This is true of both our kernel driver (in-tree, though admittedly we’ve not been very good at submitting patches to davem recently) and our surrounding ecosystem of value-add software like our kernel-bypass network stack OpenOnload (which also has an Enterprise version distinguished by a paid-support offering, another classic open-source funding pattern).
There’s also the ‘software-for-use-value’ side of things; for instance, my first task when I joined the company was to write scriptable remote control software for a thermal testing chamber we’d just bought. When it was done, no-one had any problem with my slapping a 3-clause BSD license on it and uploading it to https://github.com/ecree-solarflare/ovenctl — which may not have produced any obvious gains to us (since it hasn’t received a single pull request in its two years online), but does have the signalling value of saying “We’re good open-source citizens, who share because we have no reason not to”, and perhaps helps convince other idealistic developers to come and work for us.
> Why do people hand your organization money if your organization’s business model involves developing software they are giving away?
Yes, I’m curious too. Please don’t tell me Indiegogo, as someone else mentioned earlier, though I don’t recall the exact context.
“I think, also, that when Kamp elevates control by a single person as a necessary way to get quality he is fooling himself about what is even possible at the scale of operating systems like today’s *BSD or Linux, which are far larger than the successful cathedrals of programming legend.”
Google and Apple would disagree. All of the major operating systems used by consumers (Android, iOS, Windows) are built using cathedral development practices. And these operating systems are more complex than GNU/Linux with far richer user experience.
While not even Jony Ives has total control over iOS and OSX he’s the equivalent of the Pope of Apple design. Apple is the example that focused vision can reduce chaos.
Even for open source bazaar is not the clear winner given that Android is clearly done in a cathedral style where all the major development and decision making is done within the Google inner sanctum and source code is only released when Google is ready to release. For example 5.1 source code got pushed but only after the official launch of 5.1. Android M source only exists inside Google.
There is far less chaos inside the cathedral and that model has been FAR more successful at producing operating systems that normal people can use on a daily basis over the GNU/Linux or BSD bazaar model which has been absolute failures at achieving any sort of consumer market share in comparison to the Android or OSX/iOS cathedral derivatives.
Successful cathedral development isn’t legend. It’s on your phone and it dominates the computing landscape.
I also find it humorous to declare the victory of Bazaar over Cathedral on a Google+ thread.
This is the subject of Eric’s book called “The Magic Cauldron”. Basically he believes that 5%(but could be as high as 15-20% without affecting the result) of software is actually developed for sale, the rest is all interface shims, “Maintenance” software and specialized business process automation stuff that has no market value outside of the company it was created in.
The book is available at this location.
> Google and Apple would disagree. All of the major operating systems used by consumers (Android, iOS, Windows) are built using cathedral development practices.
Funny that you mention Android and iOS. Both of those have bazaar-developed OSS kernel at its core.
> So basically, he doesn’t like autoconf and wants the dot-com kids off his lawn.
BTW. I have found very nice introduction to autoconf and automake in a very good book “21st Century C, 2nd Edition” by Ben Klemens, published by O’Reilly Media (and available from O’Reilly in DRM-free formats).
> SAAS isn’t really all that open, or it wouldn’t be salable either. SAAS programs are usually hiding on someone’s server, and are only open to the developers.
>
> Why do people hand your organization money if your organization’s business model involves developing software they are giving away?
There are many examples of programs / solutions where you have open-source source software and you have hosted SaaS solution (perhaps with additional bells’n’whistles that are missing in Community Edition, and are not costly to develop). You pay for not having to set up, support and maintain server…
Not really. The xnu kernel is really only maintained by Apple; they don’t accept patches from all comers as is the case in the Linux development process.
Android’s modifications to the Linux kernel are also internally developed and not available to the public until an official release is done.
> Android’s modifications to the Linux kernel are also internally developed and not available to the public until an official release is done.
But getting those modifications in-tree is done bazaar style.
Note that Agile can be thought of as bringing team-sized bazaar development [also] to close-source software (proprietary and in-house).
It’s something that our host talked about, but I cannot find the blog post in question (tagging here is a bit sparse and sometimes missing).
@JEff Read
“Android’s modifications to the Linux kernel are also internally developed and not available to the public until an official release is done.”
But the modifications are not the kernel. Individual patches are almost always developed “in private”. But the kernel is not. The reason Open Source works is that it much more efficient to develop the whole project in the open for many reasons eloquently described in the books of our host. And more efficient projects tend to win out in the end.
And Android is not even a good example as Google did not made that many changes in the kernel code:
http://www.forbes.com/sites/quora/2013/05/13/what-are-the-major-changes-that-android-made-to-the-linux-kernel/
Two particular cases that I use: CloudBees and Blitline. Both of these are cloud-hosted versions of open-source software (Jenkins and ImageMagick), where I don’t want to have to deal with infrastructure issues myself. Instead, I pay someone a service fee to run OSS software on some EC2 instances for me.
Funny that you mention Android and iOS. Both of those have bazaar-developed OSS kernel at its core
a) they don’t. Apple’s kernel is not.
b) who cares? Linux was used because it was free and they could garner some free community good will for using it over a closed kernel. They don’t need a large percentage of the kernel functionality and they could have purchased or written one if desired. The kernel is a small commodity component like drivers or file system.
The key for success for any consumer product has been user experience and Google, Apple and Microsoft has invested tremendous effort on making overall ecosystem and UX right as opposed to a huge number of half assed bazzar implementations that all suck.
App Store, cohesive API, stablity, HIG, UX support, development tools, and functional business model are far more complex and important than the fucking kernel.
The only ones that don’t horribly suck are the ones the community hates like Ubuntu because they want to drive it in a more cathedral style of design and decision making.
,Note that Agile can be thought of as bringing team-sized bazaar development [also] to close-source software (proprietary and in-house).
No. Agile teams are not coordinated in a bazaar style. They are coordinated in a more traditional hierarchical style since all the teams are building toward a common product and common design.
In fact there isn’t even a requirement for teams to expose code as some teams are in different companies and source may or may not be commonly accessible for IP reasons.
>> Note that Agile can be thought of as bringing team-sized bazaar development [also] to close-source software (proprietary and in-house).
> No. Agile teams are not coordinated in a bazaar style. They are coordinated in a more traditional hierarchical style since all the teams are building toward a common product and common design.
Yes, proper Agile with self-organizing team is bazaar-like; this was at least partially motivated by OSS success.
@nht
“b) who cares? Linux was used because it was free and they could garner some free community good will for using it over a closed kernel. They don’t need a large percentage of the kernel functionality and they could have purchased or written one if desired. The kernel is a small commodity component like drivers or file system. ”
I think you just showed you did not understand anything about the reasons the developers of Android chose the Linux kernel.
One simple bullet point would be the availability of drivers for almost any hardware, another file systems, yet another a network stack.
Yes, proper Agile with self-organizing team is bazaar-like; this was at least partially motivated by OSS success.
“Proper” Agile makes no such requirement. “Self-organizing teams” essentially doesn’t exist in large scale commercial development.
>> Yes, proper Agile with self-organizing team is bazaar-like; this was at least partially motivated by OSS success.
> “Proper” Agile makes no such requirement. “Self-organizing teams” essentially doesn’t exist in large scale commercial development.
Development with no self-organizing teams is not Agile, as defined by Agile Manifesto (preference for Individuals and Interactions) and Agile principles (11th principle).
I think you just showed you did not understand anything about the reasons the developers of Android chose the Linux kernel.
One simple bullet point would be the availability of drivers for almost any hardware, another file systems, yet another a network stack.
Little of which they needed at all since they needed only to support a small set of hardware on ARM.
They needed only two file systems and Yaffs was not part of the mainline kernel anyway. I can’t remember when or if it made it in since it seemed like they’ve been working on getting into mainline since forever (or 2009ish anyway).
Who doesn’t understand what again?
vfat they got for free sure but that’s MS format not a native Linux one.
They only need one network stack that works to include in the API every app developer would use.
They used Linux because unlike Apple, Microsoft, Blackberry, Nokia they didn’t have their own customized mobile OS. Unlike vxworks or qnx it was free and they could easily control the direction of development without buying the companies. And they could do so without giving anyone but hardware partners the source until phones started shipping.
It certainly wasn’t a bad choice but it’s a very minor part of Android’s success.
The keys to Android’s success was Java, their APIs, the App Store and a good enough UI to evolve to something almost as good as iOS (just like Windows) or in the case of Samsung able to make look like iOS as much as possible.
Development with no self-organizing teams is not Agile, as defined by Agile Manifesto (preference for Individuals and Interactions) and Agile principles (11th principle).
Oh yes, you are correct I forgot about number 11 because its so poorly worded so I remember it as “the best stuff comes from teams that don’t suck”. If you read what the “founders” write about their principles this is pretty much what it devolves down to.
For example:
“And voila! There are the people. I was sure they’d turn up. The people need to be self-organizing. And they need to have the ability, not just the right, to improve the design. They improve the design over time. It changes. It emerges.”
The right to change is the “shall” part of the requirement for Agile. The “should” part is the ability to change which I contend that if you have that ANY methodology works. I’m not an agile fan. I’m staunchly in the James Bach/Tom Demarco Good Enough/Heroic Team/PeopleWare camp.
If you have great devs using waterfall you will outperform so-so devs using Agile.
So my bad. In my defense “self-organizing” means something different to me since this was a short fad in the 90s to attempt to do real self-forming teams in the corporate environment. Wow was that a debacle.
@Jeff you may have read too much into my “carefree” comment. My point is not that regular paychecks in the FLOSS word don’t exist, but merely that the ratio of that happening is quite different between those two fields, and the scene could use far more corporate investment, would benefit from it, and it would spur it forward quite a lot, like, figuring out new ways how to use it to sell hardware.
Interestingly, hardware sales was one of the earliest predictions in the scene i.e. that that would be one way to milk the open source cow, and at some level I suspect this is why we see these gadgets like Kindle or Nook.
Perhaps a new prediction can be risked: as corporations always like some kinds of locked-down monopoly, and GPL doesn’t allow them to do it with software if they want to reuse a FLOSS basis, they do it with locking down the hardware hardware. I predict many Kindle-like devices, unique UI but Android/Linux under that all the way down, the source dutyfully published but pretty much useless unless you want to copy the hardware, which is patented, gotcha. Clever.
> I don’t see how autoconfiscation can possibly the fault of the bazaar; isn’t it an example of GNU software handed down on stone tablets by some guy with a long beard?
The autoconf software itself may be. However, in today’s bazaar ecosystem, everyone “knows” that it’s the correct thing to do when a project gets to be a certain size, and there’s no guy with a long beard to propose a better way.
@ESR
What do you mean by: “It did not occur to me to respond in public at the time…”? Is that not you responding in the comments section of that same article: “Eric S. Raymond | Mon, 20 Aug 2012 17:52:05 UTC “?
>What do you mean by: “It did not occur to me to respond in public at the time…”?
I didn’t blog about it. And IIRC my comment was some time after the article was originally published.
@ESR
I did not know you read articles from the ACM Queue. So, shameless plug time. Have you read mine? http://queue.acm.org/detail.cfm?id=2742696
I would welcome your opinion on it.
>I would welcome your opinion on it.
Ahh, you’ve independently reinvented Mark Miller’s “promise” abstraction from his E language. Yes, it works.
From Henning-Kamp’s article:
Autoconf and libtool are hairballs, and pains in the ass of any self-respecting software developer. But they kinda sorta do their jobs well: making GNU software work not in a bazaar environment, but in an environment of multiple cathedrals competing with one another. As for “standardizing Unix”, has he ever served on a standards committee? It can take years to get the details right, and that’s when all the players are friendly and have closely aligned goals. We’ve seen the gnarly hell of OSF and The Open Group, and we’d never have gotten as far as we have if we had stuck with that model. Maybe what Henning-Kamp is suggesting is that we all standardize on a single-vendor solution, like Microsoft Windows? Because that’s the only way effective standards seem to become established with any swiftness.
You can kind of see this in microcosm in the OpenGL standardization process. The OpenGL API is codified by a consortium of vendors with an interest in the project, all of whom are pursuing different interests. What tends to happen is that the old guard of legacy CAD software vendors dominates the discussion and the API is held back by legacy cruft and a lag time of months to years to hammer out the details, meaning that it is well behind the cutting edge of 3D accelerated hardware. And no, extensions are not a solution to the problem; they pollute application code bases with additional code paths to test and handle the presence of sometimes one, sometimes multiple extensions to do the same thing. Meanwhile the single-vendor solution — Direct3D — Just Works with the latest hardware, presents an easier to use API, and works more closely to how the actual hardware works.
Maybe things will change with Vulkan. My hopes aren’t terribly high, as long as something like the ARB (whether they call themselves Khronos or something else) exists.
I think he just meant “blog about it”. Expressing his thoughts on the matter here rather than in a comment on the original article would have given them a major signal boost.
Oh, and the 86open project should stand as a hilarious counterpoint to Henning-Kamp’s piece: in the late nineties, the remaining x86 Unix vendors decided to get together and agree upon a standard binary format for x86 processors running Unix. What they found was that between SCO and Sun’s use of ‘lxopen’, the kernel personalities provided by the BSDs, and of course Linux itself, most every x86 Unix already supported running Linux binaries so they made Linux-ELF the standard and disbanded.
Consensus and working code, bitches.
You’ve conflated an example with an argument, & to be frank it is a very strong example indeed. Yet surely the bazaar doesn’t imply the hell that PHK describes any more than monarchy implies war, conscription, & graft. It just seems to work out that way more often than not.
Any idiot, including the greatest architect who ever lived, could built a stall made of sticks to hawk their tat in the bazaar, but a billion idiots with shitty stick assembling skills won’t add up to a single cathederal no matter how hard you squeeze your eyes shut & pray.
> One is that [Kamp] (rightly) complains about autoconf, but then leaps from that to a condemnation of the bazaar model without establishing that one implies the other.
This reminds me of the leap made by Charles Darwin of natural selection to suppose something called “evolution”. Here is the totally unrelated “Kemp” error, by the way.
I think this is an example of tragedy of the commons. Unix and Desktop Linux essentially belong to nobody.
At least with Windows, the OS belongs to somebody. What compatibility (APIs) the next Windows is going to have is decided by that somebody and that’s it, take it or GTFO. In the Linux (and Unix) world, you have all those “distros” selectively embracing things (for example systemd and wayland in linux, pf in Unix) at different times, and selectively depreciating things.
There is a reason Firefox runs better on Windows (for example it had WebGL accel first), and you can whine about Mozilla not having it’s priorities right, but if you can’t get mozilla to get their priorities right (aka supporting the myriad Linuxes and Unixes out there as well as Windows) good luck getting anyone.
And that’s why everyone hates Canonical for not playing well with the community, even though no self-respecting beard would use Ubuntu unless forced. It rose out of the pile and become the default distro to recommend to average users for a while, thus bringing some order to the chaos. Now that Canonical became uncool, we have everything from Mints to Mageias to Debians to elementarys being recommended, and good luck finding software dear average user. Let’s hope you only need OSS and you don’t want to have the latest version, because it may not be in the repos, and if it’s not in the repos… (we all know the text that follows)
“What this piece speaks of to me is a kind of nostalgia, and a hankering for the control (or just the illusion of control) that we had when our software systems were orders of magnitude smaller.”
OSes are getting too big for their own good. Stuff that should have been released as libraries and statically linked to apps is provided as APIs. This is because nobody wanted to give out their treasured code to be statically linked to third-party apps. so they just welded it to the OS.
This was a dumb business decision that doomed computer science: OSes became too big and carry all kinds of junk for back-compat purposes. Stuff that should have been statically linked is now part of what defines the OS. Apps became less portable because they depended on code that was in the OS too much. If Dennis and Ken followed the same decision when designing Unix, modern Unixes would still have to replicate bugs of the original stdio.h
PS: Of course, with Windows, at least you have one mess instead of dozens
Fortunately, the trend is reversing, as both iOS and Android follow the principle of a small core OS, and lots of libraries provided by the SDK for static linking. I like that.
But those are evil, evil cathedral OSes. The bazzar of Desktop Linux keeps going down the rabbit hole of moar APIs and moar API calls.
@ kurkosdr
Re: static linking
What’s your opinion about the sta.li project?
(Anyone else’s opinion is welcome as well.)
> even though no self-respecting beard would use Ubuntu unless forced.
If by forced you mean because nothing else is usable; otherwise I am not sure what you mean. Ubuntu has a great stride for LTS releases, not too long (Debian), not too short. If you want quicker releases between LTS releases, there are reasonably spaced releases available. Providing third party software to people and asking them to use a distro that will be abandoned within 6 months is an insult to said customer. Trying to keep compatibility with RHEL5 while using C++11 or even C++14, on the other hand, is a nightmare. Which segues to:
> What’s your opinion about the sta.li project?
Well, not sure about sta.li, but musl, the statically linked C library, looks really interesting. Trying to support scores of distros with different libc versions is difficult. If the app statically links whatever it can and uses RPATH for the rest, life is eased a bit. The weak link is libc, which is hard to statically link (no one seems to offer advice on this). It looks like musl makes this easy, so thanks for the tip. The LSB is great, but hard to support when really ancient releases of GLIBC are part of the standard.
sta.li is dead. As in I don’t think anyone is actively working on it.
Dynamic linking has fucktons of advantages: there is one, and only one, authoritative copy of a library on disk and in memory. This makes patching security vulnerabilities (like motherfucking HEARTBLEED) a relative doddle compared to the alternatives; and it also means that there aren’t multiple copies of the same code floating around in RAM. To get that with static libs requires transparent deduplication in the filesystem, which is available in btrfs but not guaranteed. The people who say that dynamic libs offer no advantages are all Plan9 beardos who don’t know much about how shit gets done in Linux.
I find it hilarious that Linux beards used to laugh at Windows users for suffering “DLL hell” — which is now a 100% solved problem on Windows thanks to SxS deployments and will probably dog Linux from now until the time_ts wrap (number of bits left unspecified).
Musl is more interesting, but niche. I used it to compile software for Terminal IDE on Android, to see if I could build more common Linux tools than were supplied in the Terminal IDE toolset. I may yet reapproach the problem of using it to build a chroot-less Android-compatible “distro”. It would be neat to have a real Linux programming environment ready to go on any Android device without resorting to rooting , chroots, etc. I like the idea but the mainstream is glibc, glib or Qt, Wayland, systemd. Iron rule of software: use the tools that everyone else uses if you want to avoid pain.
>It would be neat to have a real Linux programming environment ready to go on any Android device without resorting to rooting
This would probably be a solved problem already, if not for the fact that most people who would bother setting up a real programming environment on Android are going to root their devices as soon as they buy them on general principle.
hard to predict. They’ll see the sense in the open-source prescription when they find the limmiits of moddability.
Lol…even gaming in Linux is dominated by closed source steam games.
And the limits of moddabity are not very confining when you can change nearly every aspect of the AI and graphical skins and some games are built with world building as part of the game editor.
The fact is many modders are not hard core coders anyway and depend on the game devs providing the editing tools for scenario development and equipment mods. The mod support tools allows regular gamer the ability to create new maps, scenarios, campaigns, equipment and skins.
Or people will just reverse engineer it and mod anyway, like Minecraft.
I’m coming late to this party, but:
“Apropos the things you never responded to: have you seen (and if so, do you plan to respond) to this blog post from Jeff Atwood? http://blog.codinghorror.com/given-enough-money-all-bugs-are-shallow/”
Based on my personal experience I think Jeff Atwood is correct.
I have worked on bugs in several pieces of open source code that have been there for quite a while, at least one I inserted myself.
The best example I can think of is a problem in winbindd that had been known about for 6 years before I found and provided a fix that Jeremy Allison rejected but between us we fixed it correctly. The reason that the bug had been around for 6 years was that no one could figure out how to provoke it, and it was only because I had a customer with the problem where it reliably occurred every 10 minutes or so.
Complex systems are really hard.