One of my commenters pointed me at an article by John Sonmez over at ElegantCode, Why Software Development Will Never be Engineering. The article makes one very shrewd, well-argued point, but then disappointingly fails to build on it. Read it and see if you spot the problem before I analyze.
I want to start by praising Mr. Sonmez for very logically and thoroughly arguing against a common kind of excuse-making. Often, when software engineers are asked why software engineering doesn’t have the reliability rates of, for example, bridge-building, the answer is: well, the discipline hasn’t had time to mature yet.
Sonmez observes that we humans have built a lot more software in the last 40 years than bridges in all of recorded history. His underlying point – which he makes very well by looking at how online play has terrifically accelerated the evolution of poker – is that in thinking about time to maturity in a discipline, cumulative hours of practice by humans matters more than the number of calendar years the techniques have been practiced. In terms of human practice time (Sonmez argues) software engineering has actually had at least as much time to mature as bridge-building.
I could quibble with this argument if I wanted to. The degree to which more people in a discipline can speed up its evolution is limited by the speed with which they can learn from each other in each round. There’s a diminishing-returns effect from adding more practitioners in parallel; it’s a lot like (and may actually be formally identical to) the Amdahl’s Law limit on speedup from processor parallelism.
But I think that really would be just a quibble, and that Sonmez’s underlying point about comparative evolutionary times is basically sound. And I wish he had stopped there, because it is just after this point that his reasoning goes all to hell and handwaves.
Before I get to dissecting that part, I’m going to note one startling detail that is orthogonal to Sonmez’s main arguments. At one point, as an example of change in the methods of software engineering, Sonmez says “Ten years ago waterfall was all the rage.” an assertion to which I can best sum up my reaction by quoting P.J. O’Rourke’s immortal words from Parliament of Whores, to wit: “What the fuck? I mean…what the fucking fuck?”
Here in the land of open source, waterfall development has been considered a form of doomed, brain-damaged idiocy as long as I can remember, let alone “as recently as ten years ago”. I suppose I could take Sonmez’s aside as a pleasing confirmation that my peers and I were ahead of the curve, but in fact my reaction is of gobsmacked shock at the thought that his peers were that far behind it. It gives me a suspicion that most of the software out there is actually far worse than I realized, a sort of nauseous sinking feeling.
But that’s not what I’m here to write about this fine evening. It’s probably not mostly Sonmez’s fault that he lives in a cultural surround where “ten years ago waterfall was all the rage”. It is his fault that, having constructed a lovely argument against one of the standard excuses for shoddy software engineering, he immediately begins failing even basic logic immediately thereafter.
Sonmez wants to argue that, since both disciplines have had comparable times to mature, the fact that they don’t have comparable error rates means software is fundamentally unlike bridge-building. But he has neglected to exclude the simplest possibility: that they are fundamentally alike, but the differing error rates are a consequence of differing complexity scales.
OK, so what do I mean when I say “they are fundamentally alike”? They are both disciplines in which the local behavior of components is usually simple and easily mentally modeled by a designer. The complexity, in both cases, generally comes from long-range couplings among parts that may interact with each other in unexpected ways. This is a hard distinction from, for example, biology – in which, basically, there are no simple components at any level.
I didn’t choose biology as a contrast at random. Sonmez appears to think that software engineers are more like surgeons. But that assertion just sort of hangs in the air; not only doesn’t Sonmez actually argue for it, he never even tells us what he thinks it means, beyond some poetic handwaving about software systems being “unruly beasts” and “keeping it alive”. Really, I could say “Software engineers are like blinching flugletharps” and convey exactly the same amount of information, which is to say none at all. What up, does Sonmez want us to wear scrubs and facemasks when we code? What’s the actual point of the metaphor, here?
The sad part is I don’t think Sonmez really has one. Yes, software has to be maintained. So do bridges, and taking a busily-traveled thoroughfare out of service for repairs has opportunity costs so high that it’s not done a lot more often than taking a large software system out of service for repair.
Now, it could be that the “fundamentally alike” case is wrong and that software is more fruitfully viewed as being like biology. The trouble is that Sonmez never gives us any reason to believe that. Everything after his “Software development is different” heading is pure handwaving. And, after the buildup, very disappointing.
I don’t think I know for sure whether or not software engineering is fundamentally different from bridge-building. But it’s a thesis I’m suspicious of precisely because it would in many ways be very comforting, almost an implicit excuse for our dreadful error rates. Sonmez isn’t doing us any favors by making a bad argument for the proposition, and he might not be helping matters even if it were a good one.
UPDATE: My wife the attorney acidly points out yet another way in which Sonmez’s argument is flawed and unhelpful. “If surgeons had the failure rate of software engineers,” she observes, “they’d all be in jail.”
Anytime someone talks about how great bridge-building is, I think of the Tacoma Narrows Bridge.
It’s a cautionary tale of how the real world messes up our pretty theories.
>Anytime someone talks about how great bridge-building is, I think of the Tacoma Narrows Bridge
I don’t think that’s fair. The Tacoma collapse is well-remembered precisely because such failures are very rare. Would that the same could be said of software engineering.
I was thinking about how the “cumulative hours of practice by humans” was strengthened, and broadened to more humans, by open source when I first read the article.
Nice article. I find your arguments correct, but too polite :-) I think surgery and software development are miles apart. Surgery is about fixing a complex system which already exists. It is not iterative as such and it is done in one go with –as you point out– very little risk of failure. In contrast, software is about creating things that never existed. It is iterative, is never done in one go and has a high risk of failure.
I agree with your wife a lot that surgeons do not fail because their motivation to not-fail is very high. Programmers are sloppy because they can get away with it via patches. Hardware designers put less bugs because they know that they will get fired if they do. In fact, programmers on the Wall street do not put bugs either because the steaks are too high. At the end, its all about money:-)
On a completely different note: I disagree that it is Amdahl’s law. Its the law of diminishing returns from the field of economics. Here, the argument is not centered around parallelism but about the number of people contributing to it, parallel or serial.
I think one major relevant distinction between bridge failure rates and software failure rates is that bridges usually are traveled over and blown about by the wind. They aren’t used as small-arms targets, they aren’t used as stages for heavy-metal concerts, and they aren’t even used to carry loads above a specified capacity. Software is frequently repurposed for puzzling, inefficient, and even laughable uses; see the abuse of the spreadsheet as a database for one of the most common.
I don’t mean in the slightest to detract from the absolutely correct point that most software is poorly written, but alongside the observation that there’s just so much more happening in software than in bridges, it seems to me that software is also abused in significantly more creative ways.
I was unable to find serious problems in the article (well, except for the vagueness of the final part of the article), instead I’ve found problems with your analysis.
1. @esr: Here in the land of open source, waterfall development has been considered a form of doomed, brain-damaged idiocy as long as I can remember, let alone “as recently as ten years ago”.
This is certainly true. This perception is still there and people from “the land of the open source” like to pretend they don’t use waterfall and that waterfall can never work. But let’s look on the reality, not the perception, shell we? Think the most popular distribution: Ubuntu. It’s developed on tight timeframe (release twice per year), rarely slips (the last slip was five years ago when Dapper Drake become Ubuntu 6.06, not Ubuntu 6.04) and at each stage of the development follows pre-determined plan with few variations. If this is not waterfall, then what is? Since this “doomed, brain-damaged idiocy” made Ubuntu the most popular distro it was adopted by others: Fedora, for example. Things like this are classic waterfall development. But let’s forget distributions for a time. What happens on the other side of the fence: in the proprietary land. Is waterfall still used? Oh yeah. Game development, for example: most games are classic waterfall developments. They are conceived in the heads of the designers, then they follow pre-determined steps, then they are released and supported by a few patches which don’t change the core but only change some minor bugs. Classic waterfall as it’s described in your link – and yet this is area where proprietary software utterly dominates. “Your peers and you were ahead of the curve” yet interest you managed to catch is minuscule at beast.
2. @esr: The complexity, in both cases, generally comes from long-range couplings among parts that may interact with each other in unexpected ways.
Actually there are huge difference. Let’s take a look on a single component of our usual computer systems: kernel. Linux kernel will be good example because all the information about this beast is public. There are about 10 million lines of code in kernel – and about 2-3 million are actually compiled in (depends on the set of drivers. Number of different components in modern aircraft or bridge is measured in thousands. Usually less 10’000, sometimes closer to 100’000, but never beyond that. Of course each components is more complex (compare simple screw with “*p = 0” line), so complexity should be similar? Nope. Broken (or incorrectly chosen) screw can only affect bridge or airplane locally. Error in assignment similar to what I showed above can affect state of the program globally: it may destroy some totally unrelated memory location if the uninitialized variable is used. And since noone guarantees uninitialized p even contains “sane” pointer connection between components will depend on timing, on the hardware, etc. Usually quite hard to reproduce and fix.
What are the proposed solutions? Well, offered solution 10-20 years ago (when people tried to make software more like bridge building) and today (when all pretenses were thrown away) are wildly different. 10-20 years ago “the path to the future” was obvious: compartmentalization. This is where the infamous “Microkernels have won” comes from. This where Java comes from. This where JavaScript comes from. If we can split software in small pieces and each one will be easy to understand then we can use them like screws and beams in bridge construction. Yet today we have monolithic kernel, JVM which is over 10 million lines of trusted code and similar (but so far smaller) JIT-compiler in browsers. Compartmentalization helped a bit, but not all that much. Our hope is shifted to tools like coverity, Valgrind and Tsan today. Instead of trying to make code more predictable (like other engineers are doing with their creations) we are studying the behavior of our code and try to fix it by carving pieces of it and observing the results. If this is not a work of a surgeon, then what is?
You may say that unlike surgeon (who have no “right for failure”) we can always study software in isolation and conduct the experiments “in vitro”. Unittests, emulated systems, etc. Well, that was true 10 years ago. Today… it’s not true anymore. You can not really debug things like μTP or Facebook/Google/Twitter server software “in vitro”. Some things are only ever observable “in vivo” (I know people behind TSAN: they are trying to invent something which will reduce slow-down from 20x-30x to 2x-3x to make it possible to observe “patients” in vivo – under real load with real users).
3. @esr: This is a hard distinction from, for example, biology – in which, basically, there are no simple components at any level.
This is not true: very low-level things (like DNA works or even mitochondrion operation) are pretty-well known. At least “in vitro”. It’s only when you pile layers upon layers and/or add iteraction between organons and organisms you eventually reach the stage where you can not predict anything with 100% probability. Well, the software is moving in this direction – and it’s pretty far on this road. I’m not sure I can agree with Kurzweil (who argues that in 30-40 years software will overcome and subsume biology) but the fact they it moves in this direction is clear.
>If this is not waterfall, then what is?
It’s not even close. Not remotely. To point out just one disqualifier, for Ubuntu development to be waterfall all of the features for the next release would have to be specified at the start of the cycle before any integration at all of new code was done. Having been a developer on several games, and knowing war stories from many more, you are just as systematically wrong about that. It is extremely common to add features that were not initially planned as the narrative logic of the game suggests it.
>Error in assignment similar to what I showed above can affect state of the program globally: it may destroy some totally unrelated memory location if the uninitialized variable is used.
Yes, exactly. That’s a long-range coupling. You’re reiterating my analysis while under the illusion that you’re refuting it.
>This is not true: very low-level things (like DNA works or even mitochondrion operation) are pretty-well known.
Yes, I knew somebody would bring that up. When you’ve reached the level of simple parts in biology, it’s called “chemistry”. This distinction is not accidental, it relates directly to the complexity of information flowing in and out of what you are studying.
@esr: My wife the attorney acidly points out yet another way in which Sonmez’s argument is flawed and unhelpful. “If surgeons had the failure rate of software engineers,” she observes, “they’d all be in jail.”
Are you sure? For example mortality risk of a heart surgery is 3 percent and in complex cases much more. I think you are comparing apples and oranges: failure rate of some singular change with failure rare of the whole surgery process. I’d say that software engineers are doing much better: redesigns (like KDE4 or GNOME3) may lead to temporary problems, but rarely they lead to dead or unusable codebase.
Yes, transformed software can fail in marketplace, but this is separate issue.
In the software there are separate process which is not yet attainable in biology: creation of the totally new “organisms” (aka programs). Here the failure rare is high, but since in biology we can not do that at all (yet?) it’s hard to compare these activities.
@Aater Suleman: I agree with your wife a lot that surgeons do not fail because their motivation to not-fail is very high.
Not really. They rarely fail because there are the whole industry which supports the patient after surgery. Nurses are supposed to patch the patient till he’s ready to function autonomously (which usually takes days, sometimes months). Without them failure rate of surgery will be close to 100%.
@Aater Suleman: Programmers are sloppy because they can get away with it via patches.
Well, yeah, but usually software after surgery is much more operable then patient after surgery. Your still need patches to stop unforeseen complications (aka bugs) after surgery (aka redesign), but their number and severity are much smaller in the software world compared to health care world… so far.
Complexity of the software grows so it’ll not clear it’ll be true forever.
The best programmers I’ve met were all artists, or more correctly artisans, not engineers.
However, I’d say that the last 20 years have seen advances to towards a more engineering like approach through the much better use of OO programming languages and the much better availability of high quality libraries for doing just about everything..
Maybe some day.
>The best programmers I’ve met were all artists, or more correctly artisans, not engineers.
Ah, but the best bridge-builders are artists too. That’s not exclusive of them being engineers, nor is it in the software case.
What you’re noticing is that as a practitioner become more skilled in the mechanics, his attention increasingly shifts to higher-level considerations that present to him as esthetic gradients in design space. This is a characteristic of mastery in all forms of engineering, hardware and software, no exceptions.
>However, I’d say that the last 20 years have seen advances to towards a more engineering like
I wouldn’t say “more engineering like”, because I never though engineering and art were opposed in the way you assume. More rigorous, yes, though I don’t think increasing use of OO languages has more than a superficial connection to that.
My candidate for the two things http://www.csun.edu/~dgw61315/thetwothings.html about programming (originally inspired by a ycombinator comment via apenwarr):
1. Like every other significant field of human endeavor, programming is fundamentally unlike anything that came before it.
2. It became the bottleneck in human progress much faster than any of the others.
Nice break from all the smartphone posts.
Always thought provoking.
@khim
“This is not true: very low-level things (like DNA works or even mitochondrion operation) are pretty-well known. At least “in vitro”. ”
DNA simple? Ever looked at DNA folding? The way histones work? The way how promoters are able to steer the de-coiling of DNA? In short, the interaction between the nucleotide sequence and the secondary and tertiary folding of DNA coils. That way, all that “junk-DNA” has been found to have vital roles in gene expression. On step up, the transcription machinery interacts with histones, promoters, and the gyrase proteins. That is still a target of massive research efforts. And the whole question of DNA repair is behind basic Cancer research. No one has claimed to have solved that part.
A mitochondriom is more like a free living bacteria (from which it originated) than a bag of chemical. No one will claim to know how they work beyond the very basics. Even the ribosomes, which are much smaller, are still understood only on a very cursory scale.
And this is just the in-vitro part. In vivo, things are much worse. Any idea how T-RNA’s can find their anti-codon so fast in-vivo? What are the thermodynamics behind this feat? What are the trade-offs?
@khim
“They rarely fail because there are the whole industry which supports the patient after surgery. Nurses are supposed to patch the patient till he’s ready to function autonomously (which usually takes days, sometimes months). Without them failure rate of surgery will be close to 100%.”
I have heard surgeons describe themselves as “just plumbers” in earnest. Their low failure rate is not based on fear of jail, but of emphasizing on best practices. I hope I will never ever have to be under the knife of a surgeon that only tries to keep out of jail.
Surgery is trying to patch a running system with limited means (to invert the metaphor). That has to be done with the utmost care. Like patching a running kernel in RAM.
The surrounding care is a BS argument. Every aspect of care is important, and everybody knows that. That is not different from developers coordinating with compiler builders and hardware designers.
khim:
> In the software there are separate process which is not yet attainable in biology: creation of the totally new “organisms” (aka programs). Here the failure rare is high, but since in biology we can not do that at all (yet?) it’s hard to compare these activities
The molecular biologists are necessarily very far off. Some time ago a lot of noise was made about the first ‘synthetic’ bacterium. Of course, it was more accurately the synthesis of the natural genomic sequence of Mycoplasma mycoides from scratch in a test tube. Even so, the minimal set of genes required for a viable bacterium is known to a reasonable estimation, it can be tinkered with at will and it has been shown that the synthetic DNA can be used to ‘bootstrap’ a blank cell.
@Mikko
“ven so, the minimal set of genes required for a viable bacterium is known to a reasonable estimation, it can be tinkered with at will and it has been shown that the synthetic DNA can be used to ‘bootstrap’ a blank cell.”
This was an important milestone from Craigh Venter. He was indeed able to copy and paste a minimal set of genes (based on a framework of the smallest bacterial set known) and synthesize the genome from only a nucleotide sequence. The “blank cell” was an existing cell minus genome.
This is indeed like replacing the OS on a computer. It does not tell you how to build a computer. And if you cobbled the OS from existing modules, it does not tell you how to write and build the modules. But it is a necessary step in the direction of understanding how a cell works.
A next step would be to try to write a replacement gene from scratch.
I think I have can propose an answer to the problem via a specific example.
Everybody knows, ERP software sucks. Quite often it cannot even properly calculate your COGS and it means it cannot even tell you how much money you made then what it is good for? The point is, developing a good COGS / FIFO algorithm / data structure / business process / GUI / recommended Chart of Accounts / etc. is hard. Not hard in the sense of higher math, but hard in the sense of a million exceptions, special cases, and “Why nobody told me THAT?!” – kind of smashing-your-brow-on-the-keyboard moments.
It can take _decades_ to get it right – for f.e. Navision it certainly did take around 20 years, and the currently popular open source ERP software like Compiere or OpenERP don’t even _try_ to get it right. SAP is said to have some… issues in this field. I don’t think there are many larger corporations who have 100% correct published earnings…
If software was like civic engineering, the answer to that would be
1) academic cooperation, like, you know real RESEARCH which requires a culture of sharing and not secrecy i.e. an open-source way of thinking
2) degrees / certifications.
So basically academic CS folks should publish COGS / FIFO algorithms and data structures in journals. From them books should be written. Such books should be taught in business IT courses and without a good knowledge of the subject no one should be able to get a business IT degree. Then the ERP software development companies should hire no one to develop a COGS / FIFO feature without that person having such a degree. In the first iterations, the whole thing will be too academic and impractical. However, after 20 years of developing business software, the certified business IT engineer ought to go back to academia, write new journal articles and books which also incorporate the practical field experience gathered. With a few such iterations, say 50 years we can have published pseudocode and data structures and everything in journals and books, which are taught at courses, and if you make sure you hire someone with the right degree then it is almost certain that he will write a good software for the purpose. This is largely how civil engineering works – people learn in academia, take it out into the field, then bring the experience back to academia – and share it all the time.
None of it is currently true. Businesses hire all sort of good coders to develop such software, and not necessarily ones that have a degree that reflects knowledge of such books that give domain recipes. Most ERP programmers don’t have a specific business IT degree. I do have, but such things were not taught to me. I don’t even think if there are such books published. Nobody expects me to read such books because software – at least this subset of it – is very short-term-oriented, the point is just coding up something fast and then go selling it. Given the sales-oriented attitude, sexy charts and suchlike are what considered important and deep underlying structural problems are swept under rugs. We don’t really have the professional pride of civil engineers… we are more of a wild-west, take-the-money-and-run culture… We are paid a lot and often don’t have the right degrees, therefore people like myself don’t bring their experiences back to the badly paid academia. Most important of all, a culture of closed source, and secrecy prevents the kind of cooperation that is a precondition for academic research and its code. Navision got the data structure and algorithm after 20 years roughly right but they will never share it. Their hard-earned experience will not be ever taught in the universities. Thus everbody is doomed to repeat the same mistakes.
Thus you are right in pushing open source, this is how the positive feedback loop can get started. But don’t expect quick results – OpenERP will take 10 years to get it right, then maybe in 20 years it will be taught in the universities…
Maybe a third aspect is the following. The construction industry is perfectly capable of doing a shitty job too, everybody has seen houses where the paint was coming off after 3-4 years and suchlike. What is indeed rare is only the kind of catastrophic errors like bridges collapsing. It is similarly rare in software too. If you would consider every imperfectly vertical wall, every roof leaking water, every wall not meeting in a right angle, every squeaking door, every window you must rip out to open because some dumbass has painted it too thick – is it sure the construction industry has so much lower failure rates? Of course these are not engineering mistakes – but in IT is the engineers themselves who are laying the bricks so their mistakes are inseparable.
@shenpen
“What is indeed rare is only the kind of catastrophic errors like bridges collapsing. It is similarly rare in software too.”
A current hot news item in my country is that the ICT system of our national police has been judged a complete failure. Entering a bicycle theft can take up to three hours (no joke). An hour for a simple crime is the average. This was a 10 year software project involving many people. And they all failed on every level. This would be a poster child project fail. But they had a hardware project, all emergency services on a special phone service, that failed just as hard (firefighters being unable to use the short range radio’s). So I think it is not the software design that is to blame, but the systems design stage.
I think that one reason software projects fail catastrophically so often is the design by committee (and I consider a camel to be a better horse, especially in the desert). The members of the committees are basically ignorant, because the design principles have to be reinvented every time. No one knows beforehand what a piece of software should do, how it should be done, and what context it will be used in. Because it is different every time. Except with OTS software. That is just bad quality. But OTS software is only a minority of projects.
Best practices for software development projects
http://www.ibm.com/developerworks/websphere/library/techarticles/0306_perks/perks2.html
Link to Dutch newspaper article (sorry, no English)
http://fd.nl/Print/Bijlage/FD_Extra_ICT___Overheid/2011/04/11/politie-ict-faalt-hopeloos_bron_fd_selections
Well, the obvious distinction between software and bridge-building is that I would imagine that there’s a lot more variability in software. Bridges all basically do the same thing, though their methods may vary tremendously; software, apart from “do stuff with data” which is uselessly generic, doesn’t have any such common purpose.
Just a fleeting thought … I wonder how many patents are are on bridges.
What would a bridge look like if it had to be designed from scratch such that it held no superficial similarity to any other bridge in the world.
“ICT system of our national police has been judged a complete failure”
And in mine the Soviet-style blockhouses (“panel house”), which house at least 5% of the population, have turned out to have very serious issues to them, as in, wall panels simply falling out on the streets. They will have to be torn down within a decade or two, and replacing them will put a huge dent on the budget.
Design by committee: well, or very often no design at all. I could show you stuff that began as “oh its just a simple Excel macro” and ended up with a horribly large and complex kludge of Excel tables, Access and SQL databases… IMHO the point is really short-termism. People just want a report right now. Programmers deliver it the simplest way possible. Then people want another and another. At no point is there time or money available for a redesign – the higher-ups simply want stuff get done fast.
I think a good explanation is design cost / total cost ratio. A large office building has such a high non-design cost (labor, materials etc.) that doubling design cost is not an issue. However if you extend your house with a patio you will probably not hire an engineer, just make a quick sketch, plan for few hours, then get busy. Software is IMHO very often developed the same way as such similar DIY home improvements are made – make it up as you go. Given that in IT it is the computers who “lay the bricks” and every line of code is by definition design, doubling design costs means doubling total costs.
Sorry,
Flipped the links
@Michael Hipp
“What would a bridge look like if it had to be designed from scratch such that it held no superficial similarity to any other bridge in the world.”
Like the US aviation industry pre-WWI.
http://en.wikipedia.org/wiki/The_Wright_brothers_patent_war
Like the US movie industry before they emigrated from NY to California.
http://www.cobbles.com/simpp_archive/edison_trust.htm
Like the British industry during the lifetime of Watt’s patent.
http://lpuk.org/2011/04/patent-pending/
In short, few bridges would be build, and all at great cost.
Bridgebuilding is a discipline that must deal with natural laws that cannot be broken – strength of materials, physics, etc. There are no natural laws that software must follow. It is an expression of thought. That is why the most that can be done is to put some little procedural discipline into the activity. That is why the field of Software Engineering is mostly process and the more critical the software (aircraft flight control for instance) the more critical the adherence to process.
Given that the title of Sonmez’s post is Why Software Development Will Never be Engineering, I’d say hit post fails at a more fundamental level. He never discusses the fundamental component of engineering: problem solving. Sure there’s some science heavily mixed in there, but ultimately an engineer is trying to use science (both scientific knowledge and the discovery process) to solve a problem. As software developers are constantly trying to solve problems, be it an application to make email management easier or a library to support that application, I think it’s a doomed effort to make the claim.
As to his bridge building example, I’d say the main difference is maturity. Most aspects of bridge building have been quantified: materials, spans, brdige type, etc. More recently, the “art” of bridge building has added some complexity to the designing of bridges because aesthetics have become more important. So the problem nowadays isn’t just putting something there that allows people to go across some span. Even so, at the end of the day, bridge building is still a small, well-defined problem space: how can we get to the other side?
Software development, OTOH, is constantly evolving because the problem space is constantly evolving. Even with 50 years worth of iterating through software engineering, the fact that the silicon, tools and targets are constantly changing (software engineers just got smartphone to play with) makes it difficult for the field in general to “master” any one of those. If I ask what’s the best software to write an email app with, it’s a guarantee there won’t be a consensus on language, let alone approach.
So points about error rates and the like are superfluous to Sonmez’s stated thesis, (software development != engineering). He needs to demonstrate something more fundamental than discussions of error rates or the fact that software development isn’t cookbook-like enough. He plainly fails to do so.
I think the major reason why we see a much higher rate of failure in software versus construction is the tradeoff between costs and quality. With software, a failure (in most cases) has much less critical consequences and, in addition, people have come to expect the occasional crash or malfunction. As a consequence they are not willing to invest that much in terms of money, effort and time to get to the level of reliability of, say, bridge building.
For an example of software written to comparable standards of reliability take a look at the stories about the team writing code for the space shuttle. Here’s one link: http://www.fastcompany.com/node/28121/print
Turns out that when there’s human lives and millions of dollars at stake, software writing can be done to the same high standards of engineering. It’s just that in the vast majority of cases, people don’t care enough to go through with the effort.
“Software development, OTOH, is constantly evolving because the problem space is constantly evolving.”
Reminds me of Bjarne Strostroup saying plain simply users want their software 1) quickly 2) cheaply 3) don’t mind bugs, as shown by their behaviour on the markets i.e. people buy the first and / or cheapest piece of buggy crap that promises to solve a given problem. So that’s what they get. I think there is a lot of truth in that that the _possibilities_ are expanding so fast that simply the industry has little other choice but to keep on adding features instead of spending time on getting the older ones right, because people simply want everything that is possible to do ASAP.
The difference between software and bridges? Software can be fixed easily and more or less on the fly. Bridges? Not so much.
Furthermore, developing software is an iterative process — you write a some code, test it, debug it, write some more code, test it, debug it, lather, rinse and repeat.
Bridges necessarily have to be all designed upfront. Traditionally, you can’t iteratively build the bridge the way you do software, because only a completed bridge will actually function (though these days we have simulation). That’s why the SDLC waterfall method was so popular in corporate coding culture 10 years ago: the idea was to do the design (gather requirements, write a spec., etc.) upfront, just as bridges are built. The problem with that is that reality doesn’t work that way with software. Often you can’t determine what your final design will look like until you’ve attempted to write it a couple of times (the “be prepared to throw one away” part of CatB and Brooks’ Mythical Man Month).
@Shenpen-
> I think there is a lot of truth in that that the _possibilities_ are expanding so fast that simply the industry
> has little other choice but to keep on adding features instead of spending time on getting the older ones
> right, because people simply want everything that is possible to do ASAP.
I think the possibilities are expanding so fast (for now anyway) that it’s impossible to come up with a “right” way. 20 years ago, a smartphone couldn’t be made because the hardware wasn’t small enough or fast enough to handle the task. For that matter, just think about the fact that even the lowliest smartphone has more computing power than a PC from 1990. How can a discipline possibly hope to “master” something so that a solution can be copy-pasted together when faced with that rate of change?
I actually thing that his core argument about amount of effort is bogus. He tries to argue that 12 million programmers have made 3 programs each, which is (a) wrong and (b) meaningless. I’ve probably written dozens of programs, but most of those were Perl scripts clocking it an less than 100 lines. This is a “program” in the same way that a plank over a creek is a “bridge”. If we restrict the count to large, complex pieces of software made by large teams, which is the only thing that’s remotely analogous to a bridge, then I highly doubt that we’ve built more software programs than bridges.
Taking the long human history of bridge-building into account, I think that it’s still true that we’ve spent an order of magnitude more man-hours building and designing bridges than we have building and designing software. (However, aside from this point, I agree that “immaturity” is not a core reason for the fact that software engineering sucks.)
I think that Shenpen, while clearly a victim of credentialitis, nevertheless has an important point to make.
The problem is that software changes too quickly for the knowledge-loop process he’s talking about. If state-of-the-art metallurgy changed as radically and as quickly as state-of-the-art memory management does, then engineers who build bridges would have the same knowledge weaknesses that engineers who build software have.
This, I think, is a big part of why technology credentials are losing their value in the job market. It’s not that hard to teach a business-domain expert to implement his knowledge in this year’s fad programming language — certainly nothing like as hard as it is to teach someone who’s spent his life head-down banging out code about how your business actually works.
Civil engineering certainly has had its share of fiascoes, too – see, e.g., “The Big Dig” here in Boston for a 10X overrun in time and cost on a few miles of highway.
But I do think the (at least, a) key issue with the software market (defined very broadly) is that it takes as much effort to thoroughly evaluate a piece of code as to write a minimal version of it in the first place. This leads to a huge information problem in the market, which is much more pronounced for low-volume products.
In the comments, Sonmez distills his argument to two main points (link: http://elegantcode.com/2011/06/22/why-software-development-will-never-be-engineering/#comment-233535080)
> 1. You can’t apply statistical analysis to software development like you can other engineering
> disciplines. (In that regard software development is not engineering.)
>
> 2. Software development has already advanced further than most people think in terms of
> evolution. If we are waiting for software development to “mature” and look more like other
> “mature” disciplines, it is not going to happen.
As for 1, OK- he’s largely correct. But could someone point me to where engineering is defined by “the ability to apply statistical analysis.” It’s merely a tool to be deployed against a problem.
And while I’ll agree that software development has come a long way from its infancy and is not mature (perhaps like a teenager?) as a discipline, I don’t really see why that disqualifies it from the domain of engineering.
Thinking further, I wonder what “mature software engineering” would be? Everyone agreeing on how to deal with email or desktop UI’s for example? Or maybe everyone writes the software with the same techniques and organization? Perhaps a large percentage of current known software tasks are accomplish bug free? Everyone writing code for ARM processors? Perl becomes the defacto language of choice for all development enforced by GNUru’s in hooded robes?
Excellent critique of my post. I don’t 100% agree with all your points, but you have caught was I was trying to throw. Many readers didn’t quite understand the comparison I was originally making. I have to admit that I never really intended to compare software developers to surgeons. I was just wrapping up my post and trying to disassociate them from engineers. A surgeon worked nicely for that purpose at that time.
Although, as I thought about it more and more, it became obvious that someone could draw a conclusion about the similarities of software development to surgery. I am not sure how much value would be there.
Good point about the complexity scales. I didn’t want to go there, because I think that has been said more often than the point I was trying to make, but I agree with your statement. I have often told my wife, there is no human endeavor more challenging and more taxing on your brain then software development.
Two words – cyclomatic complexity. It doesn’t matter how large or small a bridge is, it’s cyclomatic complexity is always restricted by the properties of physics. On the other hand, software gets more complex the larger the application. Therefore, the use of bridge building as a comparison against software development is invalid.
Matt – seriously?:
It’s not that hard to teach a business-domain expert to implement his knowledge in this year’s fad programming language — certainly nothing like as hard as it is to teach someone who’s spent his life head-down banging out code about how your business actually works.
Yes, and I assure you, there’s a 80% change that code will be a non-scalable, unstable maintenance nightmare once it expands past demonstration size.
Now, it’s also true that trying to explain what you want to a programmer is hard at best, and a lot of “professional programmers” are just as hopeless as those domain experts at coding. But at least your software architects do need to know what they’re doing or you’re likely to end up in deep trouble…
@patrioticduo
“Therefore, the use of bridge building as a comparison against software development is invalid.”
Personally, I would prefer to compare software engineering with terraformation and biosphere design and management.
That is a comparison where software development would come out as a glorious positive example.
The explanation seems simple to me. Goes back to a catchphrase I learned Years ago: “You can’t manage what you can’t measure.”
The physical properties of the components and the overall system in a bridge are well known. There are relatively few things to measure and relatively few measurements to make. It is simplicity on a large scale.
I’ve still not seen anyone come up with metrics for software that are of much use, much less as simple and reliable as say, the compression strength of a batch of concrete. Software is complexity with exponentiation.
Anyone want to try to measure anything of import on, say, Microsoft Office?
>(though these days we have simulation)
Simulation doesn’t always help much; But It Worked in the Computer Simulation!.
@William B Swift:
I actually almost wrote (and don’t get me started about how well simulation works) but opted against it because I didn’t want to inadvertently start a debate on the merits computer simulation! :)
The principle degree which I have is in Software Engineering, which is a recognized engineering degree in Ontario, Canada. Had I stuck around for a while and fulfilled the apprenticeship requirements, I would have fulfilled the requirements for P.Eng after my name. There are a number of reasons that this change was made, including the Therac-25 radiation machine problems which killed several people due to software errors.
All engineering involves trade-off selection between costs, time-to-delivery, robustness, etc. In addition, it is safety and failure conscious as it typically assumes that the end users of the product will not readily be able to change the software after delivery as well as that it may have life-critical applications. Avionics software isn’t something which is reasonable for the pilots to debug and repair mid-flight if the software decides that your plane really should be flying backwards because of some integer overflow. You cannot reasonably replace the software in a pacemaker on an ongoing basis, or at least between the time it malfunctions and the time you die due to that malfunction. Software used by other engineers for their analysis (like some CAD software) falls into this category; even though readily replaceable, any decisions made based on the output of the software may quite literally be set in stone at that point.
This is quite different from most other industries. Even something as paranoid and heavily regulated as the banking industry can afford a few minor software failures from time to time, depending upon the likelihood of hitting a bug and the results. Something which might cleanly crash when doing interest calculations at the end of the day once every 5 years or so and which automatically recovers, simply adding a minor amount of down-time probably isn’t worth an extra million dollars in development and testing time in order to be able to solve right now. This makes failure analysis, failure mitigation and failure recovery essential in the context of the product being developed.
The problem is that there are a huge number of people in the world who are writing software who aren’t actually engaged in software engineering. This isn’t a problem unto itself. However, it does confuse people about the difference between the two and leads people to assume that software engineering doesn’t exist. It does, it’s just that most of the software people interact with doesn’t meet that standard, and most of that software gets direct interaction with only a small part of the population. It is like assuming that since most bridges by number built are made from erector sets by pre-teen kids and usually fail if you run a full-sized car over them that clearly bridge-building must not be a mature or reliable engineering discipline. Of course we don’t look at it that way – it is an absurd thought.
*shudder*
That’s true, but it assumes that *nothing* can be measured in software engineering. This is very obviously not true. For example, LOCs are generally a good measure of code complexity. OTOH, LOCs per day are often a poor measure of programmer productivity. On the gripping hand, Checked-in LOCs per programmer are generally a good way to measure who has contributed the most code and effort to a particular project. Bugs per KLOC seems to be a good objective measure of software quality, but features per KLOC will differ widely between projects.
It’ll sound brash, but I’ll posit that a huge part of the problem is actually just that OSS hasn’t won yet—which leaves software as a discipline that still basically shuns the very idea of peer review. And if engineering is applied scientific knowledge, and peer review is a requirement for whatever science underlays engineering, then not-having peer review means that you can’t even try to develop an engineering discipline. You can’t base engineering on junk science.
How far do you suppose bridge-building would have developed if neither physics nor chemistry ever thoroughly embraced peer review?
I forced myself to RTFA even though I felt like it flew off the rails only a few sentences in. One of the things that really annoys me about software developers analyzing their field is when they try to make analogies to fields with which they have absolutely no experience. Usually all this does is illustrate their own ignorance and annoy practitioners of the field in question. It seems obvious to me that Mr. Somnez hasn’t ever really talked with any civil engineers who work on bridge construction and hence has no idea what they do but still feels it appropriate to make a comparison anyway.
JS Bangs is correct in stating that his comparison of bridges built to programs written is completely asinine. Bridges are generally large multi-year construction project involving multiple firms and millions upon millions of dollars. The large bulk of software written does not compare to this. Even at my day job I work on a team no larger than 40 people (fairly large in terms of software) and a really large project would probably cost 5-7 million or 1/5th of what one new pedestrian bridge in my hometown will cost.
He next tries to bolster his point with a poker metaphor which again makes no sense when comparing software development with bridge building. While bridges may present their own unique construction challenges they also rely on general construction principles that are universal to all building projects. So instead of just looking at how much bridge building has been done to how much software we have written, it would be instead be more appropriate to compare the entire history of construction with that of software. Material science, basic static and structural analysis, aerodynamics and vibration analysis are all applicable to bridge building but not restricted to it. Once that comparison is made it becomes clear that we still have much more software to build before we can have a body of knowledge to build on similar to that which civil engineers can depend on.
Finally I would state that I disagree with the basic thesis of both the article and of esr’s comment on it. I fundamentally feel that software engineering is an immature discipline, and that our failures are a function of the small amount of experience we have with building programs. My view is that at the base of the pyramid is the turing machine (or lambda calculus if you prefer), an abstract mathematical model. And upon that we keep building more and more layers, but fundamentally what we are grappling with is a way of expressing mathematical relationships and constraints which will be executed by our concrete implementation of that abstract mathematical concept. I mean “mathematical” in a very general, non-numerical sense. See Alan Kay’s “The Computer Revolution Hasn’t Happened Yet” speech for more detail on this, but in general I feel that we are still working on new and better ways of expressing mathematical and logical concepts, for creating abstractions and expressing the constraints they expect, and the notation we do so in. Its an ongoing quest and it seems to me that we still have plenty of different spaces to explore before we can say that we have approached nearly the same level of maturity and development as civil engineering.
There’s something that many commenters have alluded to, but didn’t come out directly and say:
An engineer will specialize and become expert in some field, like bridge-building. The days when an engineer like Brunel could design bridges and ships and… are long gone.
Software developers know how to develop software, but they come to a big project with no knowlege of the problem domain. They have to learn that on-the-fly, gaining experience by making mistakes. Still worse, the people who commissioned the project in the first place don’t know what they really want, and find problems communicating with the software people. This is much harder than telling an engineering firm that you need a bridge from here to there, to carry so much traffic, and they provide a solution.
OTOH, the above is not all bad. Software’s inherent flexibility allows the business to explore solutions that they might otherwise not be aware of (if they have the nerve to keep funding the project). Trying out a suspension bridge, then a cantilever, then a simple truss design by building them all wouldn’t fly in the engineering world; the engineer would be expected to do that on paper and propose the best one.
@esr: Apropos your wife’s comment, this from 25 years ago:
“If builders built buildings the way programmers write programs, the first woodpecker would
destroy civilization.”
Sorting algorithms and data structures are bridges and roads. Software systems are cities. You might build a bridge that will carry all the traffic you expect, but then have no traffic show up. We’ve probably all seen 4 lane roads to nowhere that average maybe 3 cars per hour. You may build a road that will carry all the traffic you expect, but then something you don’t expect happens and you get 100 times the expected traffic — the two lane state highway that has a constant stream of traffic.
Software developers are city planners, not bridge builders.
> So do bridges, and taking a busily-traveled thoroughfare out of service for repairs has opportunity costs so high that it’s not done a lot more often than taking a large software system out of service for repair.
The opportunity cost of bugs in software is very low. Imagine trying to do bridge-building according to the build-test-fix cycle. The fact that testing software is effectively free means we can develop it in a fundamentally different way.
The opportunity cost of taking software systems out of service is lower than for bridges because you can hot swap them. I suspect Google web search has received more tinkering in the last six months than most major bridges have in its lifetime.
Actually, Cathy’s argument at the end is quite a bit deeper than it appears on the surface. (I don’t know if she meant it that way, or if it was just an idle quip.) The point is that a bridge failing has hugely negative consequences, software failing often does not (outside of a special class of software that is usually dealt with entirely separately.)
The reason software error rates are so high is that we, the users, consider that acceptable when offset of against the benefits. I’d offer as an example Windows. It is much less reliable than a solid Unix system but the benefits of ubiquity, and availability of various software packages etc. offset the negatives for many classes of user.
What is ironic is that building quality software is actually cheaper than building low quality software (up to a certain point anyway) because the cost of late remediation is so very high. However, from a corporate software development point of view, the basic problem is encapsulated in the statement “The first 90% of software takes the first 90% of the time and budget, the last 10% takes the other 90% of time and budget.” Which is to say, software development managers are very bad at incorporating all the costs of developing software, the majority of which does not involve typing lines of code. This is an unhappy coincidence of the narcissistic attitude of many programmers that the world revolves around their code, and the willing acceptance of naive mangers that their unrealistic timelines can be padded with more features for free.
When it comes time for release we apply what I shall christen Cathy Redmond’s rule of software releases: “If doctors had the same failure rates as software guys, they’d be in jail”, which is to say the release date is more important than the growing bug list — because the evidence is that this is the choice the users want (holistically speaking.)
I think the essence of Eric’s argument is that software has a higher bug rate than bridges because software is vastly more complicated than bridges. I agree. Factor in the aforementioned rule, and the fact that software, unlike bridges, is under constant pressure to work at the absolute bleeding edge of technology, and the current situation is not at all surprising.
You know as a follow up to what Peter Davis says, I think there is a pretty interesting point there. As I mentioned the cost of developing quality software is actually lower than the cost of developing low quality software (up to a point) because the cost of late remediation is so high.
Based on the point that Peter makes, it is obvious that this is taken up several orders of magnitude in bridges, late remediation is not only expensive, but ruinously expensive. Which feeds back into the rule that building a quality bridge is much cheaper than building a low quality bridge because the cost of late remediation is ruinously high.
Sometimes reality is big enough to overwhelm our own vanities.
Perhaps the problem is just that there’s usually just less incentive to make absolutely sure programs work. If an engineer builds a bridge and it fails catastrophically, people get killed. Generally, this isn’t the case for software; the worst-case scenario is usually a bit closer to ‘severe annoyance’, and it’s easier to release a software update to fix discovered problems than a bridge update.
I think esr once remarked that GPSD was an example of a program where a bug might actually get someone killed, and that he was exceptionally rigorous in his testing as a result…
Aha, I think Jessica Boxer beat me in making my point.
Morgan Greywolf Says:
> Bugs per KLOC seems to be a good objective measure of software quality
Agree with everything you say Morgan except for this one sentence.
Every time I see some book/paper/blog talk about a certain project having a certain “faults per KLOC” I want to scream “How do you know how many faults you have when you haven’t yet found them all yet!!!?”
I wouldn’t trust any developer who said he knows how many faults are in his software.
I agree with many of the points made by the commenters above above the relative costs of software failure vs failure in construction but I have to take issue with Jessica Boxer’s point that software is vastly more complicated than bridge building. If you actually have experience in building bridges then I retract my objection, but my guess is that you don’t, and that comments like this reflect the peculiar arrogance of software developers who somehow believe that no one has ever had to build complicated artifacts before computers were invented.
I took Software Engineering in university which meant that I was able to hang out with people in other different engineering disciplines and I keep in touch with many of them today. One of them even has a job working for a bridge engineering firm and let me say that, based on my conversations with her regarding the process behind constructing a modern bridge, I absolutely cannot agree that software is some sort of magical field unmatched in its complexity. Again, I posit that what helps her accomplish her job is the thousands of years people have been building physical structures and the accumulated wisdom and knowledge base that provides.
I find that building software is very much the same as building a bike shed.
You might try to to a clean elegant job, you might want to get something up quick and cheap — right now, before it starts raining. You might need to shelter twenty fancy bikes for years, with room for working on them, or you might just throw a tarp over the bike for tonight.
Good software engineering gives good reliable results — it has been some years since Linus allowed a kernel release that brought systems to a smoking halt, and as someone pointed out, wall street is in the same position as the bridge engeneers, it must not fall down, it is very very bad if it does.
And, if you look around, there are an awful lot of quite marginal bridges out there — not on major highways or freeways, but there are a lot of little bridges with few users. And a significant number of them are a bit shakey — or worse.
Granted, there is an awful lot of awfully awful software out there, but the same thing applies to engineering — look on the cheezburger.kludge list for some spectacularly ingenious methods of doing something that works, sort of, maybe.
The comparison of bridge engineering with software engineering is made looking at the freeway and railroad bridges and comparing them to the lesser end of software development.
In my job in the manufacturing plant, I don’t fear the spectacular blowups. They are embarrasing, and can cost money — it’s the silent failures that I fear — the software gives bad data that looks plausible without throwing an error.
Jim Hurlburt
Yakima, WA
> I have to take issue with Jessica Boxer’s point that software is vastly more complicated than bridge building.
Consider that software development is entirely about producing blueprints, it’s all design information. Then consider that an estimate of the cost of producing the Linux kernel by proprietary means estimated a cost $1.2 billion dollars[1]. Since this is a based on lines of code, I think it is a reasonable proxy for complexity. Because I’m feeling kind (and despite the fact that is almost certainly an underestimate due to re-factoring) I’m going to divide that by 1000. Who spends $1 million designing a bridge?
[1] http://ldn.linuxfoundation.org/blog-entry/more-milestones-linux-kernel-development-history
Its all about throughput.
When people are building a bridge it’s implicitly understood that it takes a long time an a lot of effort. But, as has been stated here, its actually not procedurally that terribly complex, but it is very very detailed.
IF we build software the way they build bridges, we would plan everything out, we would diagram the systems, we would model the systems, we would prepare the environment, and we would begin to build. Sounds very waterfall, eh? That’s why waterfall has so much traction in the business world, because it makes logical sense to the non programmers.
Software development allows you to model, in the actual (or at least a near duplicate of the actual) environment. It lets you try things on the fly, it lets you change stuff on the fly and retry. You can’t do that with the physical bridge. BUT, building software like this lets you go FAST, much much faster than if you had to be truly careful as if lives are at stake.
The fundamental difference isn’t whether software can be built like a bridge, its that people would never accept it being built that way. You can write a perfect program, you just have to move very very slow (comparatively) to regular software development to do so. What people don’t realize is that if bridge building were like software, then everyone would expect bridges to be built in a couple of weeks instead of bridge projects taking years from initial design to completion. We’re hardly ever allowed that much time to develop software. The aforementioned Space Shuttle programmers were given that amount of time and were expected to deliver perfect code, and that development effort took a looooong time.
I think the answer to the overall question is no. Engineering and software are such different disciplines that to ask whether they will ever be the same is like asking will apples ever be oranges. They have some similarities, but they are fundamentally different. As a mechanical engineer and probably the least software savvy of the bunch here I might offer a couple of my opinions from what may be a different point of view:
1. The reliability difference is just a question of economics. I don’t think it is a worthwhile comparison. I would consider it a similarity rather than a difference. A bridge has a higher economic value than a dollar store widget. Likewise the software running an Airbus has a higher economic value than that running on my computer at home. If the market says it can afford to pay for reliability, it does. If it doesn’t need that much reliability it can get the end product for less cost. Hurray for free markets.
2. The real difference in my mind is that engineering is applied physics where programming is applied logic. Engineering is a thing. Software is an idea. Why does this matter?
a. It gives a different cost progression which leads to different paths of optimal efficiency. By the time a thing is starting to be built, the engineering is largely over. Economics dictate this. Engineering is often said to be 10% of a project’s cost. The bulk of the cost is material and labour. Software is a different animal. Without 90% of the project cost tied up in relatively immutable material (which fixes the idea in time and space) the design can be a lot more dynamic. Changing your mind is different than changing your oil.
b. Materials and ideas behave differently. I can make a bridge stronger by adding more of the same material. You can’t add function to software by adding more of the same idea, you need new ideas. A thicker piece of steel is stronger than a thinner piece. A piece of logic is either true or false and you can’t make it truer or falser. Now there is a similarity in that elegant engineering and elegant software both have nothing more than is needed, but I think that is more of an aesthetic preference than direct area for comparison.
Software engineering is not like surgery because in surgery you’re basically giving an assist to a self-repairing system.
The surgery is only the first step. The real work happens during the recovery time, when the self-repair mechanisms kick in.
Consider putting a pin in a broken leg: much less simply setting the bone & waiting.
Andrew Bennett raises a few important points, the one which is that he believes that Programming is Applied Logic, whereas Engineering is Applied Physics. However, this is a pointless and misleading generalization.
But before going there, I would first like to point out that dismissing Engineering as Applied Physics leads to the wrong impression regarding Engineering. By phrasing it this way, it sounds like Engineering is just an after-thought, that the engineers just picked up an idea from some physics paper and applied it. Engineering is *not* like that, far from it. Engineering is about creating something that solves a practical problem. The tools that is required to create that thing include Mathematics, Physics, System Design, Chemistry, Biology, Electric Engineering, Programming and what not, depending on the problem at hand.
When we say Software Engineering, we mean the specialization of Engineering that focuses on Programming. Some Problems have solutions that requires it to run on a computer, and this is what this is all about. All Programming is not Engineering, because it might not solve a particular problem. All Engineering is not Programming either, for obvious reasons.
Within Programming, there may exist sub-problems that do fall into the category of Engineering. These things are special because the Thing is not physical – and they are indeed more Ideas than a Thing. However, they still have so many resemblances with physical engineering that we still call it engineering. The crux of the matter is that the Idea has to be manifested into something which is real – bits in a computer are indeed real. Heat and execution speed are real things and constraints that have to be considered. The amount of memory is limited. There *are* ways of measuring if an algorithm is faster than another on a particular machine.
Engineering is a wide field. To think that there is not a lot of real engineering going on in programming is just foolishness.
Andrew Bennett beat me to a large portion of what I was going to post.
As an electrical and instrumentation engineer involved with industrial clients, my job consists of developing drawings and contract specifications for electrical contractors to implement. The actual fabrication/construction is done by the contractors to my specifications. These specifications focus more on materials with some focus on methods — “Contractor shall use rigid galvanized conduit. Contractor shall support conduit in accordance with the NEC. Contractor shall perform 5-point loop checks for all instruments so indicated in the instrument index, and shall provide written documentation of the results of these loop checks. Contractor shall perform work indicated on the contract drawings.” Often the contractor is given liberty to choose between multiple options where such choices do not impact the safety of the finished project.
In short, the client has an idea for a thing to be built. The engineer takes the idea and massages it into a form that can be implemented with bricks and mortar (and wires in my case), and then documents this form on drawings and other documentation. The contractor then takes the engineering documentation and implements the idea in the real world.
It may be possible for a software engineer to specify established, tested software modules to be used in implementing the idea. Some of the glue between the modules may be able to be specified. Forgive the obvious lack of software development experience here, but examples might include “Programmer shall use GPSD for all GPS interface requirements. Programmer shall pipe modules together in a Unix shell script. Programmer shall use Python for all scripting. Programmer shall use the XYZ linking technique to link modules A, B, and C. Hash tables shall not be used.” Again, the programmer could be given liberty where such liberty does not affect the functionality and safety of the finished package.
I’m not a hacker (nor do I play one on TV), but I’m betting that this approach is not going to fly. The people who would be best able to develop such specifications are the people who would be best able to implement such specifications. Therefore there is no benefit to such specifications.
@Peter Davies
Depending on the bridge, $1 million may be far to little in engineering cost. I have been the lead electrical and instrument engineer on jobs where my budget alone was half that.
Oops! That should be “far too little in engineering costs.”
Mr. Bennett and I are in similar boats. What can I add? I’ve done all three, software (low level, unimpressive, but still software); hardware (quite a bit, some impressive, some not so much); medical devices (involving surgeons). Where to start?
First of all Mr. Bennett is correct. Engineered hardware is applied physics. Our computer is fixed. The rules don’t change, although context can. Tacoma Narrows is a perfect example. That bridge was fine from a static physical standpoint. Perfectly sound. Unfortunately the vortex shedding frequency at a certain wind speed of the road deck happened to coincide with a natural frequency of the structure….and from the looks of the video, not a simple mode shape. These things are very difficult to model and calculate, even today using very complex (SOFTWARE!!). Unfortunately the wind decided to blow at just the right speed and a very unhappy set of circumstances occurred.
Software: Applied Logic. Your rules change. The computer is still evolving. All 1s and zeros yes, but the instructions change. The rules change. It’s CHAOS!!!. Of course there are failures. Interdependent instructions and languages. Inventing the Rosetta Stone while the languages are changing.
Surgeons….hmmm….mechanics who work on self healing machines. I’ve seen some scary stuff. Some of these people are brilliant. Some, not so much. Some are artists, most are not.
In all of these things art exists. Clean artistic code is a thing to behold. Clean artistic design is the same. Clean artistic surgery….I hope any surgeon of mine is so gifted. All are the exception, not the rule.
Aater Suleman wrote: “Surgery is about fixing a complex system which already exists. It is not iterative as such and it is done in one go with –as you point out– very little risk of failure.”
I’m not sure I would agree that surgery isn’t iterative. I recently had corrective surgery on my left thumb. My hand specialist had performed that exact procedure more than a thousand times before (and, in fact, had performed it on my right thumb five years earlier). While it’s true that he only operated on my left thumb once, the fact that he’s repeated this procedure so many times on other people’s hands had a direct bearing on his success in my case. (The surgery took seven minutes and worked perfectly, restoring my thumb to full functionality.)
I work for a very big corp and well surprise surprise we still do Waterfall Dev. As a dev I get 60 page Requirements Documents on my desk which I obviously do not read (maybe skim a little).
The Paper cited by msb http://www.fastcompany.com/node/28121/print indicates that the software for the Shuttle that is so bug free costs $83.33 per LoC per year. Normal software, full of bugs, is developped with one hundredth that budget in the best cases.
Informatimago Says:
> The Paper cited by msb http://www.fastcompany.com/node/28121/print
Fascinating read. But I gotta disagree with this statement (which seems to be the theme of the whole paper):
“It may not be sexy, it may not be a coding ego-trip — but it is the future of software.”
I doubt that … for most examples of “software”.
Today the mantra seems to be “the cloud, the cloud, everything in the cloud”, but yet your typical web page, taken as a moment-in-time example of software, is just about the buggiest and most failure-prone thing in modern life. Offhand I can think of few things that I’d tolerate having a failure rate like what pours forth from a web browser.
I’m coming around to what many are saying above that software is so buggy because we want it that way.
The shuttle software is remarkable and likely to stay that way.
Oh boy…this debate again ;)
If I have two pieces of software – A has “1 error per kloc” and B has “5 errors per kloc” – which is the better piece of software?
If programmer X contributes 10 lines of code to a project, and programmer Y contributes 100 lines of code, who has been the most productive?
—
There is plenty of code written to standards worthy of “engineering”. A great deal of it is squirrelled-away in places that are out of the limelight. It is not uncommon to discover that this software is formally constructed (to varying degrees) and related to systems with the potential for catastrophic impact on lives and fortunes.
Contrast this with the vast oceans of fartware in the marketplace – where failure is, at most, irksome.
When the stakes are high, and consequences dire, care and attention are valued.
Unlike the physical realm, binary doesn’t decay, rust, waterlog, splinter or dissolve. A well-written, constrained piece of software can have a lifespan that vastly exceeds that of a bridge. The ‘maintenance’ that is needed (other than bug fixes, of course) is often due to changes in the environment/context or manner in which it is deployed. Also, nothing in the physical realm can match the phenomenon of software being “built once, available anywhere”…imagine if you could instantaneously duplicate a bridge at near-zero cost – could you really just plonk it down wherever you like?
The complexities of the ‘virtual’ world are such that, IMNSHO, software engineering is orders of magnitude more challenging than ‘classic’ engineering. Given the vastness of the domain, I’m impressed we’ve managed to wrangle as much general discipline as we have.
If engineering rigor is what you want from software, either pay lots of money for it or be prepared to study very hard.
Michael Hipp Says:
> but yet your typical web page, taken as a moment-in-time
> example of software, is just about the buggiest and most
> failure-prone thing in modern life.
Right on Michael. And there are several reasons for that. One of the most important is that the client side code is all written in JavaScript which is, in my opinion the worst programming language in common usage (except perhaps COBOL, but I don’t know much about COBOL.)
Not only is the language lacking many of the core mechanisms that any reasonable programming language should have, but it is riddled with subtle problems where behavior is counter intuitive. Even worse, the API is documented in an extremely handwavy way, so that you never really know what you are going to get unless you test it in twenty different environments. Not to mention it is lacking the most basic of capabilities. For example, there is no easy way to get the timezone without some extremely dubious hacking.
You can’t build a skyscraper on quicksand.
With some rare exceptions, fly-by-wire systems for example, the consequences of bad code don’t seem to me as serious as bad engineering or bad medicine. Both M.D.s and engineers are said to “practice” their professions, and the term is accurate.
Modern engineering is based on empirical experience going back thousands of years, and until fairly recently mathematical modeling was not used – you built it, based on everything you knew, and then it worked (most of the time) or didn’t, in which case you hopefully learned from your mistake. There is a fairly significant element of art still involved – “if it looks right. it is right., and in my own field I’ve noticed that the designs developed primarily through computer assisted design are often not as pleasing or functional as those shaped by the hand and eye of a skilled designer.
I think it’s all much worse than this :) The problems in software are closer to being problems of political economy. It is not that proper or optimal outcomes are too hard to find. It is that the organizational incentives are pointed away from finding them.
This is aided and abetted by the changing shape of the firm. People within a firm are not as much “gain elements” as they once were – you’re more expected to be part of a governance structure that steers decision making more than a producer of widgets. This means firms have a “thinner layer” of production to add, with more decision overhead ( because all firms more or less act as integrators of other firms’ complex products ). You’re left with statistical process control of governance. That means that you’re bottlenecked on instrumenting the firm itself.
Rigor requires constraint of scope, and we’re mostly out of that. FOSS may or may not aid an abet this process; I can’t quite tell. FOSS tends to copy the model used for the kernel, and that’s completely dependent on some pretty radically non-market factors. Meanwhile, non-FOSS products are disappearing rapidly, unless they can metastasize themselves as command and control critical.
Software is not engineering, because programmers are not willing to take responsibility for their code the way engineers have to for engineering work. If software devs had to personally seal their code, including taking professional (and in some cases legal) responsibility for mistakes, then software development would become like engineering in a hurry.
Of course, most programmers will cite things like complexity, or blame the client requirements or myriad other possibilities in order to avoid their own responsibilities.
Problem is that many of you guys are still trying to equate the few thousand freeway bridges around the planet with all the android apps and the vast multitude of little projects on freshmeat.
If, instead you looked at the many, many thousands of little bridges, just here in the pacific northwest, that span a creek, and have a half dozen vehicles a day, on a big day, cross them, it would be more equivalent. Those tend to either be *way* overbuilt or pretty shakey — or were overbuilt fifty years ago and are now scary to cross.
If you look at linux, the gnu compiler, apache and python — just to pick four that I know something about, and compare them to a big bridge, or skyscraper, the quality is equivalent. In either case, they don’t fall down with very rare exceptions — one can depend on them to ‘just work’ until you get to the edge cases.
Software engineering isn’t significantly different than mechanical engineering when you back off to the meta engineering level. Even buildings will evolve over time — if you have ever had to work on the wiring in a large, older building, you would look at some of the kludges that get into software in a much different light. Don’t even ask me about the wiring on an older semi trailer that hasn’t been maintained by a single shop — I could tell tales.
The most significant difference is that the good software projects are expected to evolve over time — Eric’s tales of gpsd illustrate that, as do the other projects I mentioned. A bridge or a building has to be built in one go, then evolution is much more expensive. But happens anyway. Also, unlike a physical structure, a good software project has a potentially unlimited lifetime — do you have any real guess as to how long linux may go on? I wouldn’t bet against 100 years.
Couple of comments, mostly random.
1) “Software doesn’t decay. Its assumptions do.” Don’t remember where I first read that, or if the source was quoting someone else. Real-world structures go through *much* smaller changes in circumstances (mean vehicle count per hour, average weight per vehicle, etc.)
2) Everybody knows what a bridge looks like, and once the requirements are complete (soil surveys, cost analysis, etc.) they are relatively stable. Bridge builders don’t get a ‘change order’ half-way through that says, “Connect to the west side of the river at these three additional points, and add a helipad to the east tower, and light-rail tracks to the lower deck.
Engineers do not just worry about designing something that works, say a bridge, but they worry about designing a working bridge at a specific budget that can actually be built after a public bid process. After 40 years in this industry, I’ve never seen software folks try to have either real budget or a real time schedule. Thus to me, software is not engineered.
The reason is simple: we have no idea how to measure software. The standard business school quote is: if you can’t measure it, you can’t manage it. At least the bean counters try to manage by counting costs. The linked article mentions the stupidity of our current software metrics, yet we still use KLOC as if it means something.
The follow quote has been on my website’s home page for nearly 20 years, since I tried to get a PhD in how to estimate software (or why we suck so badly at it):
When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.
Popular Lectures and Addresses, Lord (William Thomson) Kelvin.
More like: “Connect to the west side of the river at these three additional points, and add a helipad to the east tower, and light-rail tracks to the lower deck, and can you have it email me whenever I get to the other side?”
Furthermore, bridges aren’t subject to Zawinski’s Law.
@Morgan:
> Furthermore, bridges aren’t subject to Zawinski’s Law.
Are you sure about that?
On a more serious note, the Jonhn Sonmez article somewhat puts the cart before the horse, treating cause as effect. I’ll get to that error in a bit, but first I want to call out some of the commenters who made good points.
To echo msb and Jim Hurlburt and probably a few others, software certainly can be engineered; it’s just that in many cases, for reasons good and bad, it isn’t. Jim Hurlburt has a few more cogent insights, such as “Software engineering isn’t significantly different than mechanical engineering when you back off to the meta engineering level. Even buildings will evolve over time.”
Several people echo the truisms about discipline maturity:
Gerry: “As to his bridge building example, I’d say the main difference is maturity.”
jmg: “I posit that what helps her accomplish her job [of bridge building] is the thousands of years people have been building physical structures and the accumulated wisdom and knowledge base that provides.”
Michael Hipp also gets close, but unfortunately mistakenly restricts the context to patents: “What would a bridge look like if it had to be designed from scratch such that it held no superficial similarity to any other bridge in the world.”
Hunt Johnsen expands a bit on Jim Hurlburt’s bike shed analogy: “With some rare exceptions, fly-by-wire systems for example, the consequences of bad code don’t seem to me as serious as bad engineering or bad medicine.” Basically, a lot of little software systems can let in some wind and rain, and it doesn’t really matter, just as it doesn’t matter with some real systems.
On the flip side of things, Mike E points us at another part of reality: “Civil engineering certainly has had its share of fiascoes.” But this was already pointed out by Monster in the first comment (on the Tacoma Narrows bridge), and then summarily dismissed by esr as “not fair” because those sorts of failures are rare. In fact, Monster was correct, and the analogy is quite fair.
Engineering can be done well or badly. Doing it well usually costs more in time and money, but as Jessica Boxer points out, doing it too badly will also cost in time and/or money. There is a sweet spot, and it varies by project.
So, what is the difference between software engineering and bridge engineering? In particular, if we take open source sharing of knowledge as a step forward, what is the difference between open source software engineering and bridge engineering?
One difference is that, in software, we very seldom build the same bridge twice. We have a star-trek style replicator that lets us build the same bridge over and over, so we don’t waste man-hours on the exact same problem. Our replicator also lets us put in change-orders, so we can replicate something with minor tweaks, just adding the labor for the tweak.
So when Sonmez says “But, here is the problem. We build much more software than we do bridges.” his second statement is true, but that doesn’t automatically lead to his first statement.
To the extent that we have more unqualified software practitioners (you don’t need an engineering seal) he could be right. But open source and the marketplace in general tends to weed out this sort of issue. No, the real reason we build more software than bridges is because of that replicator. The same people don’t build the same bridge — they build a different one. And that’s not a problem, but it does mean that we are constantly solving problems we haven’t solved before.
This is why the Tacoma Narrows bridge is a good example — even in the physical world, when confronted with a problem we haven’t solved before, we don’t always get it right. So software engineering isn’t really different, just practiced by a lot more people and covering the range of problems roughly corresponding to engineering nuclear plants all the way down to putting up a crooked bookshelf, or a tomato cage in the garden that will topple over in 30 mph winds.
As far as the waterfall model goes, modified waterfall models are used successfully in real life software projects in several places (e.g. the space shuttle example that msb pointed out, where feedback is allowed, but spec changes are a fairly big deal). The waterfall model was designed to mimic physical engineering, and it does — to a point. The difference is, in physical engineering, most of the feedback comes from lessons learned on previous similar projects whereas with software engineering, a lot of feedback happens during the build process.
The unmodified waterfall model fails because it assumes no feedback during the project. Note that the lack of feedback on physical projects is a good thing — “We should have poured that 200 ton column 2 inches to the left, and then this would have aligned better” isn’t going to cause any changes, so you really do have to get more right up front in the design phase.
But the equivalent in software — “move this button to the left a bit and color it purple” — is not damaging, in isolation, because throwing away code is much easier than throwing away a 200 ton column. It’s merely the undisciplined feedback that can bog things down.
Finally, to expand a bit on an observation that Glenn (NotReynolds) made: “Software doesn’t decay. Its assumptions do… Everybody knows what a bridge looks like, and [the requirements ] are relatively stable. ”
In software, we take it as a truism that if the user can do anything that makes the program exit when he didn’t explicitly ask it to exit, it’s a bug. We are much harder on ourselves than we are on physical engineers.
When (real engineers) design a bridge that is such a tempting way to end it all that 2000 people have needlessly died (and studies of 515 people who were stopped from killing themselves show that it was really a temporary bad spot in their lives — 94% went on to not commit suicide), we don’t call that a bug. And we don’t call the $50 million “patch” a “bugfix” — no, it’s a feature.
And, the idiots like Rohan Verghese, who said “Software is not engineering, because programmers are not willing to take responsibility for their code the way engineers have to for engineering work” both completely ignore the Space Shuttle example, where the programmers do take responsibility, and also have nothing to say about the engineers on the Golden Gate bridge who cost thousands of lives.
Now, you can make an argument that the engineers are not responsible for the suicides. But there is nothing in that argument that doesn’t also apply to someone who is doing something with a piece of software that the programmer didn’t design it for, and despite all the hand-wringing by the engineering union/cartel who salivate at the prospect of absorbing software into the list of disciplines that require a license, even counting the Therac-25 and the Prius, fewer people die each year from bad software than die by jumping off this one bridge.
But if I “replicate” my program on a different system that dynamically links in different libraries, have I really built the same bridge, or have I built a slightly different bridge? When you write software for general-purpose computers, running on various OS releases, service packs, and updates, with a crazy quilt of differing hardware and the accompanying drivers, do you ever really have identical bridges?
> do you ever really have identical bridges?
Perhaps not. But you still get a lot of mileage out of the replicator…
@Monster:
BTW, if someone physically replicated a real bridge (using a replicator or by just stealing the design plans, whatever), and put it in an inappropriate site (soil too expansive for the abutment design) and the replication failed, no one will blame the original engineer. In software, however, “that was pretty stupid of you not to test it on a 64 bit system” is a common sentiment in some quarters… Personally, I take this to mean that software best practice is, in at least some areas, far ahead of civil engineering.
Where I work, I have the crappiest cobbled-together piece of Python (that is slowly being refactored) that is an integral part of building some chips. It would fail most “software engineering” tests. There are very few unit tests — the production data running defines correctness. Things that are checked are checked multiple places in multiple ways. If an error is found, the program aborts, sometimes without a useful (except to me) diagnostic.
But…
It greatly helps in producing correct, working first time, silicon, by acting as an impedance matcher between the way that the system engineers on the project prefer to enter some data (spreadsheets, blech), and what the chip design tools need to see. Basically, it compiles register definitions.
Now, where I used to work, I had plenty of time and built a much fancier version of the same thing. This one will get there eventually, but the basic crappy version I already have is already so useful that I am reminded of one of my favorite sayings: “If a job’s worth doing, it’s worth doing badly.” If I get run over by a bus, it’s probably currently unmaintainable by just about anybody else, but OTOH, it’s already saved more time than it cost to build, and there is sufficient information there for someone to build a new one “correctly” — just add time and money.
Matt said:
> It’s not that hard to teach a business-domain expert to implement his knowledge in this year’s fad programming language — certainly nothing like as hard as it is to teach someone who’s spent his life head-down banging out code about how your business actually works.
Well, in fact, that depends on how your head works, Matt: I’m a passable coder — I’ll never be one of those Mythical Man Month superstars by any stretch. But I spent 20 years primarily as a systems analyst for doing custom systems pretty much from the ground up, for small (5-50 person) companies.
Nearly every one of those systems outlasted the company; one is still in use by the company that bought them out. Most of them needed no substantial changes except tax calculations, after the shakedown period. The one that took the longest, that’s been running since about 1991 or 2? Took me 6 months: 3 months analysis, 1 month coding, 2 months testing and shakedown. Most took substantially less.
Everyone thinks their business process is so complicated and sui generis, and, IME, most of them are mostly wrong.
Many designers are mediocre coders; *lots* of programmers are *miserable* designers — to FOSS’s loss, among other things. (But that FOSS is a *code-ocracy*, not merely a meritocracy, is a rant way off topic for this thread.)
Oh, BTW: defining the Big Dig as “a few miles of highway” may qualify for the Handwave of the Millenium. This one, not the last.
Nice counterarguments. I agree on that he didn’t argue for why it wasn’t engineering, and that he should have.
He’s still right though. 36% [according to Chaos Report] SW projects fails. I’d say that would be about the same ratio a mediumly skilled surgeon has on his patients.
I agree he should have argued, and not just allowed loose statements to fall through. Though he is right. SW isn’t engineering, if it was it would have been logically possibly to ‘proof’ the probability of a SW project to fail. Which both me and you know is impossible today, unless your SW development crew is *outstanding* …!
Still some devs are apparently ‘geniuses’ and virtually never delivers ‘broken projects’, while some deliver broken projects piled on top of other broken projects.
If it was ‘engineering’ we’d be able to say stuff like;
* It failed because of x
* It failed because of y
etc…
Today the best arguments we have is
* It failed because the devs, or the project manager wasn’t good enough
Which makes it obey to a much larger extent by ‘non-engineering-principles’ than it does by ‘engineering principles’ …
I’d say [like many before me] that it’s more like ART in fact. Since the rules that makes art ‘succeed’ seems to be the same rules that makes ‘SW engineering’ succeed …
Please notice Eric that even though I don’t think SW is engineering, I think it should be. But currently we’re at least 50-150 years away from making it become so, unless some radical change occurs, such that for instance people would start reading about my thoughts about architecture and the O2 architecture …
Software has already turned into an engineering discipline, with the advent of Test Driven Development and a series of methods and tools for controlling aspects of the process. The knowledge is still not very widespread, but this will change
That’s about as wrong as you can get, at least in the context of this discussion.
There are pieces of COBOL code that were written in the 1960s that are still in use today, and that have never needed so much as a single word to be changed. And there is one hell of a lot of COBOL code out there. ESR himself referred to it (in “The Art of UNIX Programming, I think,) as a vast, dark ocean.
In fact, I’d not be surprised to learn that COBOL programmers produce some of the highest-quality software in existence… and, unless the numbers have changed significantly since the early 2000s, their average salaries reflect that.
It’s not about the language itself, it’s about the engineering culture that surrounds it and how that culture feeds back to the vendors that supply that culture.
–Shannon
>In fact, I’d not be surprised to learn that COBOL programmers produce some of the highest-quality software in existence… and, unless the numbers have changed significantly since the early 2000s, their average salaries reflect that.
No. The high salaries merely reflect the increasing scarcity of people who can program in the language. They’re not coupled to code quality for the same reasons they’re not coupled to code quality in any other language.
Software has already turned into an engineering discipline, with the advent of Test Driven Development
I haven’t had such a good laugh in quite a while. Thanks.
I’ve used similar approaches with a team of developers, but to think this is engineering is fooling yourself. Shell Oil had a testing department that would have probably agreed with the sentiment regarding TDD. My boss asked me to assist them. I came up with my own methods and found code Shell found to be bug free to be riddled with bugs. How? Because I came at the problem from a different perspective. I was looking for specific data that would corrupt the system based on my decades as a programmer and had no problem finding and documenting them. Adding those cases to TDD, which they did, didn’t slow me a bit. The program got progressively more solid. But at no time was it engineered.
Software development and engineering require different types of people (while there is some temperament equivalences they are ultimately distinct) suggesting that you may be able to include engineering principles in software development but never quite get there. All programming is fundamentally equivalent from APL to Z80 machine language, but not on the level that programmers think. Style is distinctive for every individual programmer. Often elements of that style is also substantively important. Computer science departments don’t have a clue from a working programmers perspective. Otherwise, language development would be a lot farther along.
Magic bullets turning software into engineering is a goal of many that is likely to disappoint for a long time. Getting into specifics would turn this comment into a book.
Part of the problem is that Americans are profligate with the term “engineering”. In many countries, until you graduate with an engineering degree and put in the requisite hours of apprenticeship, you cannot legally call yourself an engineer. (Canadian engineers have a weird, Freemasonry-like ceremony they go through upon being certified, a rite of passage which impresses upon them the gravity of their profession.)
Point being, engineering is something different from craftsmanship. What separates engineering from craftsmanship isn’t complexity, it’s the consequences of failure. Engineered designs are repeatable, work correctly within well-defined tolerances, and are designed to minimize the loss of life, limb, and property. In many of the countries mentioned above where enginer is a legal term like doctor or lawyer and not just a buzzword, enginers are criminally liable if their systems fail. Until it’s possible to completely, mathematically characterize the operational parameters and characteristics of software, and the results of its failure before the code gets written, software is not an engineering discipline; it is a craft.
Most software development does not meet this standard. Most software development doesn’t have to. Some software does, and the term “software engineering” becomes appropriate. I’d like to see more places adopt software engineering — maybe then there would be fewer “LulzSec hacks the planet” type headlines, but as long as software craftsmanship remains economically attractive (read: cheap) it will be preferable to software engineering outside a narrow range of high-integrity systems like real-time control of physical systems (cars, planes, spaceships, medical equipment, etc.).
That said, why did you slag off Ada so grievously in Jargon, only to drop it from the File as irrelevant? Ada is more relevant than ever, and it is perfect for what it does. Its purpose is to keep the programmer “on the bounce” in Mobile Infantry jargon, all while enabling the construction of sophisticated systems without getting more in your way than is strictly necessary. A language of Ada’s type is essential for software engineering which is probably why hav=cket=rs find it so distasteful. It’s how grown-ups program.