Sometimes good terminology, by making a distinction that wasn’t easily articulated before, can be very clarifying. I was in an IRC conversation about software engineering with A&D regular HedgeMage earlier today and found myself inventing a term that I think may be useful: the ground-truth document.
The context was this: HedgeMage has observed a lot of haphazard practices at software shops that have to deal with hardware interfaces. Very often these interfaces are poorly documented by the hardware vendor, with serious gaps and ambiguities in what little description they give you.
To cope with this, S.O.P. for most shops is what HedgeMage described as “observe what it’s doing, then throw inputs at it and observe what you get back, and try to puzzle out what the patterns are”. Which is OK, except that it’s also normal to go through this process while trying to write production code.
And that is a bad, bad mistake. The result is often code that sort of worked, once, but is buggy and unmaintainable because nobody actually remembers what their assumptions were at the time it was written. They’re baked into code in cryptic ways, and trying to fix problems is terrifying because it’s so hard to tell when a change will break an undocumented assumption.
There is way to avoid this kind of mess. It’s to write down your assumptions before you write code, and treat that document as the authority of which the code is an implementation.
Later in the IRC thread, HedgeMage explained this to someone else by saying “So given the example of the hearing device from my interview question, [the document] would show all the control codes we’ve been able to pass to the device and what the device does in response to those codes, but would leave out the three that the company insists are there but that the device doesn’t actually respond to.” I added “It would list those as ‘documented, but no response'”.
At the earlier point in the discussion when I first advocated writing one, I was referring to this thing as a “design document”. But then I realized, and said, that calling it a “design document” is a problem. Programmers often associate that term with waterfall-model practices in which they’re expected to implement a bloated specification that’s wildly out of contact with reality. The point of the kind of document I was trying to describe is that it’s totally in contact with reality and not trying to describe or mandate anything else.
Here is an example: AIVDM/AIVDO protocol decoding. It describes the behavior of Marine AIS radios; I wrote it as preparation for coding the GPSD project’s AIS driver. It isn’t exactly or completely a hardware-interface specification, and some of its claims are derived from standards documents and not yet tested – but the point is that it tells you which claims have been tested and which have not. It also tells you where the observed behavior of AIS doesn’t match the standards.
Casting about semi-consciously for a way to distinguish this from a “design document”, I found one. What this is, is a “ground-truth document”.
The thing about ground-truth documents is that they don’t make promises, don’t erect requirements, and don’t talk about the future. They’re just the facts, ma’am. They describe what is, warts and all. Mine evolved into the best single reference on the AIS protocols anywhere, and has since been used as a spec by at least three decoder projects other than GPSD itself.
The practice that goes with this term is simple: always put your ground-truth document together before you start on production code (test tools to reverse-engineer the device are not production code). Maintain it with the code, treat it as the authority for how the code should behave, and when the code doesn’t behave that way treat the divergence as a bug. When your knowledge about how the device behaves changes, change the code second; change the ground-truth document first. (Of course you have it under version control, so you also have a history of your knowledge of the device.)
This a form of knowledge capture that will save you immense amounts of pain, hassle, and rework over the entire life cycle of your project. For even greater gains, write your ground-truth document in a form that can be machine-parsed and then generate as much code as you can directly from the specification tables. (Yes, GPSD does this. So does the X windows project).
The other thing not to do (besides starting on production code too soon) is to entangle the process of writing the ground-truth document with the process of writing the specifications for your software. Wishes, plans, and hopes don’t belong in this thing.
Ground-truth documents can also have other, more political uses besides knowledge capture. Having one can help you hold a balky vendor’s feet to the fire, or short-stop an attempt to pass the buck back to your team when it belongs elsewhere. “Yeah? You say that transfer should run at 30MB/s? Well, here’s exactly what happened when we shipped it the control code for high-speed mode.”
For best effect in this kind of situation you hand the vendor your test-jig software along with the ground-truth document in which you recorded the results of running it. (Yes, this is another reason to write your test tools well before you start on the production code.)
In extreme cases (and yes, I’ve seen this happen) you can wind up documenting things about the hardware that the vendor’s engineers as a group once knew but have partly or totally forgotten. This is good. It’s great negotiating leverage.
I broadcast this term and concept so that software development teams can use it to rethink their processes and do better work. Have fun with it, and stay safe out there!
I’ve seen variants of this problem all over, and it’s not unique to coding. Years ago, I worked for a bank, and the VP of the area I worked in went on to better things, eventually becoming Co-CEO of a Fortune 500 company. While he was CEO, he had a bright idea that brought his company large increases in revenues and profits, and a higher stock price. But his bright idea rested on underlying assumptions about the state of the economy. When that state changed, his bright idea became a money-losing anchor dragging on the firm, and the board lost confidence in him.
I wondered after the fact if he was fully conscious of the assumptions he was making, and what he planned to do if things changed. I came to the conclusion the assumptions were unconscious, and he never devised a Plan B because it didn’t occur to him an exit strategy might be needed. I wondered how many organizations had stumbled and sometimes failed entirely because they never said “Okay. What assumptions are we making here? What do we do if things change and those assumptions are no longer true?”
Were I sentenced to be a CEO, I think I might insist on just such a “ground-truths” document when a proposal was made, before even starting to think about whether it was a good idea to do it.
I’ve learned from years of troubleshooting and figuring out how to make things work —
The documentation is always incomplete.
The person who wrote the documentation will never be using the language quite the way you do.
It is frequently wrong to some degree.
It is not at all uncommon for it to outright lie to you.
Good documentation of a piece of hardware or software is a joy to behold.
It is also, very very rare.
On the other hand, being able to cope with the above is why we get the big bucks and interesting jobs so it’s not *all* bad.
Jim
@esr –
You also need the same document if your software exchanges data with another piece of software, for which the communications protocol is not formally specified, or where there are significant divergences from the spec.
Real world example: my very first programming job was at a company that made a (DEC LSI-11 based) gizmo that talked HASP protocol between IBMish mainframes and VAXen. They were trying to build a Motorola 68000 single board computer that would replace the ’11 and fit into a UNIBUS slot in the VAX. It turns out that what was documented for HASP wasn’t exactly what was passed on the wire. (Part of this might have been due to the fact that our mainframe was actually a CDC-7600.) When it came time to transliterate the PDP-11 assembler to M68000 assembler, it took quite a bit of effort and not a few swear words to make the code work, as opposed to doing what was in the specification.
If you can point to an RFC (or an IBM “red book”) and say, “Your code is broken, go fix it!”, that’s one thing. Otherwise, you had better look at what is really going on, and not rely exclusively on the FM (which might not be so “fine”, but more “f##ked”).
What does “Sometimmes” mean?
> Otherwise, you had better look at what is really going on, and not rely exclusively on the FM (which might not be so “fine”, but more “f##ked”).
That is the biggest problem with the response of RTFM; even if there is an actual manual, it’s probably “f##ked”, most of them seem to be.
When I read that, I thought, I have heard that parts of MS Windows are like that – code that can’t be touched because no one is sure what it does and/or how it does it.
I don’t know if this ever was true or if it is true now.
Ground-truth documents can apply to software.
Nevertheless, generating a ground-truth document for a version of Windows along with Internet Explorer might…
– take almost forever to do
– end up with a document that was extremely large
I would be surprised if any project as old as Windows didn’t have dark corners in it that no one understands. There are parts of the Linux kernel that are similarly poorly understood, the main difference there being that since the source and history is public, you can (as a last resort) track down the person who wrote it and ask them about it. And there are enough eyes going over the Linux source that there’s usually some number of people who understand all of it, though the number may be very small. I’m told that there are pieces of gcc and bash that are similarly inscrutable.
In any case, doesn’t WINE sort of constitute a ground-truth documentation of the Win32 API?
Yeah, in a way it sort of does.
OT: do you have anything to say about the Apple vs. Samsung verdict?
Been there, done that. In my case it was an error in the simulator for a Motorola hc11e microprocessor that agreed with the Motorola documentation, but not with the “Ground Truth” behavior of the actual chip. In this case, RTFM was part of the problem.
The in circuit emulator acted like the actual chip did, but didn’t match the simulator. The simulator matched the Manual, but the manual had an error in it – I have no doubt that the engineers intended the chip to work like the manual, but it didn’t.
I called the vendor of the ICE, (filled with gotcha indignation) and was told, politely but firmly, that their tool coincided with “Ground Truth”, (they re-tested it just to make sure) and to take my complaints to the simulator vendor.
The simulator vendor looked at the documentation, looked at an actual chip’s response, and fixed their simulator.
In the end, it didn’t really matter who was at fault. Every mistake/error/bug/oops reveals a way to get closer to the Ground Truth, and it is in everyone’s best interest to move in that direction. Nobody that I talked to was affronted that I doubted their tool’s correctness. Everybody wanted to get as close to the real thing as they could, and worked toward that goal with good will and a bit of patience.
I certainly didn’t mind being the one to help find this bug – it was a little exciting – like finding a bug in the compiler or something. Passing the information along to the people who could fix things didn’t cost much, even in the days before the internet. Knowing that I could rely on the ICE was nice, too. At that point I was pretty much done with the simulator (Software – virtual machine emulation of the actual chip), but fixing one bug might have helped the next guy to come along.
In my own small way, I helped make the world a better place – one where the simulator more closely matched the actual chip. What more can you ask?
Can’t ground truth documents simply be called “specifications” or is there a problem with that word?
>Can’t ground truth documents simply be called “specifications” or is there a problem with that word?
Yes, in part that it has the same association with waterfall-model idiocy as “design document”. But also, a ground-truth document isn’t a “specification” because it doesn’t mandate anything. It just describes what is.
Now, if you add a sentence that says “Our software must speak this set of request/response operations” then you have a specification. But you’ve also changed the nature of the document in a way that is unnecessary and unhelpful.
Thanks. If I understand right, “specification” tells you what to do and how to do it, while a ground truth document simply places the known facts on record, in relation to the behaviour of hardware/software. In other words, ground truth document is actually a form of reverse engineered document. Am I right?
@billswift: sometimes the best answer to “RTFM” is “WTFM!”
@hari: if you’re familiar with the terms, think of it as the difference between “descriptive” and “prescriptive” work in linguistics.
Yeah – I think the key aspect is that a ground truth document is about what is, not what is to be done.
If the user requirements change, this should have no effect on a ground truth document.
Years ago, I inherited (and pretty much rewrote) an IBM 3270 terminal emulator.
The most broken, fucked up piece of code in there was valiantly trying to follow a description of how the 3270 worked in the IBM manual. The IBM manual was 100% accurate — IBM made REALLY GOOD manuals. But it described the behavior of the hardware, including lots of really obscure corner cases, not the DESIGN of the underlying hardware.
Instead of trying to fix the existing code, I reasoned that the underlying IBM hardware we were attempting to emulate should really be pretty dirt simple — this was back in the days of 7400 logic, and reasoned that the emulating code should also be pretty dirt simple, if I could only extract a model from the description.
I extracted the model, and turned around 200 lines of assembly language into about 15.
By your definition, it would appear that the “ground-truth” document was actually embodied in the IBM reference documentation. It described to a tee what the hardware did. But that description was insufficient to do useful work until a model that matched the ground-truth was also developed.
I have always intuitively done this when writing code to deal with a system that is poorly documented or poorly understood. Once bindmogglingly simple example was a shell-wrapper I wrote to make Chrome or Chromium on Linux work correctly with Java Web Start programs. It turns out the IcedTea’s ‘javaws’ program doesn’t work with some JNLPs. So I wrote a little test-jig that forced the ‘-verbose’ flag, logged each and every argument , environment variable, etc. that Chrome passed to it and logged all of javaws’ output. Then I documented which JNLP’s worked and which ones didn’t, and what error messages I got.
I then wrote a little text file that explained all of thiis. (I’m not going to post it here.) I then took various stabs at trying to get IcedTea to work with the JNLPs that didn’t work, eliminating each complaint. It turned out that some of these Java webstart programs either depended on “quirks” in the way javaws handled the JNLP file (such as the URL having a raw ampersand in it, which doesn’t conform to strict XML standards) or had classes that were different between OpenJDK and Sun/Oracle Java (some of the class names were the same, but Oracle Java 1.6 had methods in those classes that OpenJDK 1.6 did not.)
Also chrome/chromium renames JNLP files that already exist in the way Internet exploiter does — by added a ‘ (#)’ before the .jnlp filename suffix. There’s no way to turn off this behavior short of editing the code in Chromium, so the shell wrapper I ended up finally writing just makes sure that there are no spaces in the filename, replacing them with underscores and passing the result to Oracle’s ‘javaws’ binary. *sigh*
But that ‘ground-truth’ document guided the final implementation of the kludge I put in place to make it work. Maybe sometime I’ll try to see if I can actually fix OpenJDK to make it work.
“Specification”=”ought”
“Ground Truth”=”is”
See also: “the map is not the territory”.
One way to write the ground-truth document is as a
difffrom the formal specification, so that it can be used to patch the spec and completely document actual behavior. The advantage of this is that it explicitly lays out to the people who failed to implement the spec exactly how they’ve fallen short. Later, if they either fix the implementation or rewrite the spec, yourdiffgets smaller, ideally becoming 0 bytes.Engineers have had ‘ground truth’ documents for years. They are known as ‘engineering notebooks’. A truly great engineering manager is one who can get his engineers to share what they have inside them.
>Engineers have had ‘ground truth’ documents for years. They are known as ‘engineering notebooks’.
It’s not quite the same concept, though there is some overlap at the edges of these forms. An engineering notebook describes the thinking of the person who wrote it, while a ground-truth document describes some particular thing the engineer is analyzing. It certainly is the case that many ground-truth documents originate as jottings in an engineer’s notebook, but fully-developed ones have lives of their own and (not infrequently) multiple contributors.
” It’s to write down your assumptions before you write code”
Just to check, it’s your assumptions about the behavior of the hardware?
>Just to check, it’s your assumptions about the behavior of the hardware?
Most often, yes. As others have pointed out, writing ground-truth document can also be useful near the interfaces to software.
“…the vendor’s engineers as a group once knew but have partly or totally forgotten.”
@Brian Marshall:
“When I read that, I thought, I have heard that parts of MS Windows are like that – code that can’t be touched because no one is sure what it does and/or how it does it.”
I can’t speak to Windows, but I once inherited and maintained a C application like that. No documentation whatsoever, hardly any comments in the code, maybe 30,000 lines total. I was half of the 2-person team that rewrote large chunks of it, and I personally wrote a detailed user guide and a programmer’s interface guide (it was a 3D graphical server that included a client library so that other applications could drive it).
However, it was loaded with global variables, including the arrays (with complex cross-pointers between them) that held most of the modelling data, and we didn’t feel up to the level of rewrite that would have been required to change fundamental data structures. That would have amounted to starting over.
I clearly remember there was one piece of code only a few lines long (maybe 5?) that recursively called itself and worked on those global variables. It was completely incomprehensible. We tried and tried and tried to reverse-engineer what it did so we could replace it with human-readable code, and finally gave up. Every attempt we made to change anything in that short stretch of code would blow up when run.
Ultimately, we put a very prominent comment just above it along the lines of “DO NOT CHANGE THE FUNCTION BELOW” and left it that way in the greatly-improved next release.
>>ESR : Most often, yes. As others have pointed out, writing ground-truth document can also be useful near the interfaces to software.
+1, and thanks for that important topic.
I had once to make a full remake of an existing piece of code(36 years of age, very badly written cobol, looking nearly like assembly, and working perfectly). First thing I did was a study of the code & the functionning elements. I made then something like the “ground-truth document”, just I called it “reverse-engineering specification”(and everyone liked the name).
Though, the real test was when I made the new software to replace this flawless but unmaintenable thing. As it had an exotic output, at a format that was not reused(output switched to XML), I had to create a little comparator software to compare the functional output of both the old thing & my new toy(and I tested upon a full year of production, not to miss any existing case, event the very rare “cats & dogs” insurance product, that had a single customer, its product manager). Then, I found a lot of new things that reverse engineering did not give.
Finally, the real ground-truth document was version one of the new software : absolutely identic in behaviour to the old thing(besides the one bug I did correct, as the old thing did allow the 31st of June), and commented & documented enough to be understood quickly. Well, I hope I made it readable enough, I went for other adventures afterwards.
@ Cathy
I don’t know how other folks here feel about this, but… recursion is so elegant, so sweet… and will basically turn a bit of code into a demon.
I am not talking about a daemon – code that is invoked or comes alive as a result of some event. I am talking about demons – little things that come from Satan – evil, horrible things.
I believe that anything that can be done with recursion can be done with with a loop (or maybe two nested loops). It isn’t as elegant and generally takes a few more lines of code, but the loop way has one advantage – humans can understand it and change it – it is not code from the underworld.
re: recursion
Recursion is often illustrated as a way to calculate a factorial. This is fine, since a factorial is essentially a simple recursive concept and the recursive code to calculate one is simple and obvious.
I wrote an extremely simple compiler sort of thing once for a program in which the geophysicist could write snippets of code to run against an array of data. Knowing nothing about this, I just did it from first principles. After converting a line of code into tokens, I kept removing operators (in the correct order) and adding instructions in my pseudo “machine code”. I could have used recursion until all the operators were gone, but, as I recall, I used a loop to find the highest-priority remaining operator – it just made the code more obvious – the prime directive with virtual memory (in VAX days) or now when I just bought a 2GB stick of memory for 15 freaking dollars!!!
But recursion in logic to manipulate variables until the thing decides it is done? Good luck touching it.
In the real world, the appropriate opportunities to use recursion are few and far between (outside compiler design, maybe searching trees/graphs – of course, a piece of code can be represented as a graph in a compiler).
(I may claim to not care to “understand math”, but I love code.)
Both calculating factorial (where tail recursion can be replaced by simple iteration) and Fibonacci numbers (where you need memoization to not have exponential time) are actually _bad_ exaples of *using* recursion.
But some algorithms are naturally recursive, and non recursive solution actually emulates recursion with hand-made version of stack. I think both quicksort and towers of Hanoi are among those algorithms, and of course tree travelling.
I agree. I said that calculating a factorial is often used to illustrate recursion, by which I meant, teach the concept. In practice, iteration is a better approach.
Yeah… and they both can also be done with nested loops. These two problems are so inherently recursive that recursion may be appropriate. But both seem to me to be just about as complex as recursion can be before it turns into code from the far side of the River Styx.
So… I agree that some problems are naturally recursive. There may even be problems more complex than the towers of Hanoi that are so obviously recursive that recursion is appropriate.
Nevertheless, I still suggest that, in practice, when solving problems someone is paying you to solve, recursion is almost never the solution and is frequently the problem if it is attempted.
I swear I was taught something like this at college. It went roughly like this: there are “well structurable problems”, where you know what the desired end result is, like fixing a broken hair drier, in this case you can use “hard” mathematical methods. And there are “badly structurable problems”, basically open-ended problems where you don’t even know exactly what you want, in which case you go for an iterative process: you make a root hypothesis of what you want, implement it, and based on the results modify the root hypothesis.
Which may work awesome in open-source volunteer projects, but in the business world where the root hypothesis = a specification, which is part of a development contract which must be paid when it is implemented even when it turns out to be not really useful… it can be a huge problem.
This is why I am in favor of employing developers at end-user businesses instead of using subcontracting/consulting companies. My life is so much easier since I am working for end-user companies and not at consulting companies… As an employee I can make mistakes, and there is none of that usual “OK now you fix it for free” “No, we won’t, we charge by the hour” “You must provide warranty for your product, I am not paying for something for work that did not deliver the expected value” “No, we sell a development service, not a product” never-ending debates that make life at consulting companies such a stressful hell.
There is already a term for this sort of documentation in construction and surveying (and likely other fields): as-built documents. They document a thing as it was actually built, which inevitably differs from the original design in some way.
Hmm… I don’t have to deal with other people’s code much, and frankly don’t really code much. The only time I’ve used recursion outside a college class was traversing a file directory. Recursion feels like a natural way of traversing an unbalanced tree of unknown depth.
“Nevertheless, I still suggest that, in practice, when solving problems someone is paying you to solve, recursion is almost never the solution and is frequently the problem if it is attempted.”
That sound you hear is all the Lisp hackers grinding their teeth…
Even in lisp, if you can use a higher order function instead do that. And most implementations have special forms that behave a lot like for/while to use when iterating over a numerable data structure. f
Functional data structures helps a lot in the reasoning, vs global data structures with shared state. Shared state also makes code far less testable and hard to make isolated changed as the assumptions just fracture and ricochet all over the place.
@ phlinn on
I agree – recursion is appropriate for traversing a tree because the problem is so obviously a recursive activity – Jakub Narebski made this point when he said:
.
Otherwise, I consider recursion to generally be something beautiful that you learn about in college. Consider it part of your college requirement for “humanities” in your curriculum.
@ LS
I gather that you don’t agree…
I have never learned Lisp. From something ESR said long ago, I understand that it is worth learning just to get a totally different perspective on programming. Years ago, I was interviewed by a company that was writing their commercial software in a variant of Lisp. I gathered that it was being done by, and under the close supervision of. one Lisp wizard. In what was then a Fortran/C/C++ (and, ick… COBOL) city, this sort of struck me as a dangerous situation. But they didn’t hire me, so I didn’t pay attention to how long it lasted.
Excuse me while I have a little look in Wikipedia….
Oh, yeah… invented by John McCarthy… CAR and CDR… Steve Russell on an IBM 704… LISP machines… Lisp compilers written in Lisp in 1962 by Tim Hart and Mike Levin at MIT… garbage collection… it has been a long time since I last thought about what I did know…. I remember when Common Lisp was introduced.
Yeah… Lisp is part of the great history of computing and the history of recursion.
In any case, in modern software development, the Prime Directive is: “Make it Obvious”. Recursion…. just make it obvious; if this is difficult to do, it sure as hell won’t be obvious to the next person that has to work on the code.
@Brian Marshall: To each his own, in the matter of programming languages. I have always disliked C, yet I have used it more than any other HLL. (Back in the 80’s, a programmer was quoted in Byte Magazine as saying, “C is a disease.”) OTOH, I don’t really know Lisp, but from what I have seen of it, it is obviously a work of genius. I would never use that phrase to describe C, or any of its children.
there is already a term in the construction industry for this: it is called “as built specifications” or “as built plans”.
@LS
” I have always disliked C, yet I have used it more than any other HLL”
Some say C is the lowest HLL. Others say it’s the highest LLL, and have gone so far as to call it roughly equivalent to a “universal assembler”. Whatever way you look at it, it gives the programmer the lowest possible level of control without getting bogged down in the details of the processor architecture. (And judicious use of conditional compilation directives allow those details to be tucked away, isolated from the truly portable code.)
@LS:
> (Back in the 80?s, a programmer was quoted in Byte Magazine as saying, “C is a disease.”)
Yeah, back in the 80’s, C was a disease. But C didn’t suffer from NIH, and between accepting lots of good ideas from other languages, and the compilers warning you when you might be doing something stupid, it got a lot better.
@The Monster:
> Some say C is the lowest HLL. Others say it’s the highest LLL, and have gone so far as to call it roughly equivalent to a “universal assembler”.
My frequent statement is “C is the new assembler, and Python is the new C.” By which I mean, of course, that I used to drop down from C to assembler when I needed performance, and now I drop down from Python to C (or Pyrex/Cython) if I need performance.
@ LS
I haven’t programmed professionally for a number of years (and I haven’t coded for my own use for a number of days).
I loved C and C++. I read Bjarne Stroustrup’s first C++ book, but really learned the language and how I approached it by reading the “Gang of Four” book – Design Patterns, about 5 times. At this time, C++ was almost as elegant (in it’s somewhat more complex way) as C.
I also bought and read the third version of Stroustrup’s book. By that time, at least from Stroustrup’s point of view, the Design Pattern philosophy became a little less important and the standard library that could do everything including your taxes became more of a focus. It was like the emphasis shifted from object-oriented and “encapsulate that which changes” to making maximum use of this library. There was also the evolution of the complexity of templates.
The prime directive is: Make it Obvious.
For some reason, when I think about this whole subject, what runs through my mind is…
Oops, missed an angle bracket
For some reason, when I think about this whole subject, what runs through my mind is…
… Chuck Berry
The teaching of GoF should be rated as a criminal offence: it mutilates the mind beyond recovery.
Design patterns are tropes; that is all they are. Without understanding tropes you do not understand fiction; but the goal of a good writer is to transcend the tropes of his culture and medium. So it is with programming. If the programming community at large had taken the same approach to design patterns as TV Tropes had taken to their analogues in popular fiction — and included exhausive crowdsourced lists of patterns from a variety of domains and languages as TV Tropes has done — we’d have a much healthier software ecosystem. As it is GoF is treated as a prescriptive work which enshrines certain patterns as ones that “should” be used in certain contexts in order to have a “good design”. It’s nonsense; and it’s led to a few uncomfortable moments for me where I was asked “Did you use the suchandsuch pattern?” by another developer, as if the merit of my idea was gated on the yes/no answer to that question. Probably many times more such moments for other developers who work on more enterprisey stuff.
In scientific method, the ground-truth document is called a model. As long as it is useful at the current level of detail and correctness, it is a good model. When the model no longer serves its purpose, you have to improve it.
Assume a spherical cow…
@ Jeff Read
You seem to be in the wrong blog.
The specific design patterns described in the GoF book do describe some common patterns and show some useful approaches – individual programmers (with any sense) recognize they are but patterns and that solutions can be implemented in many different ways.
You may have noticed that I said that what the GoF taught me was some details of a language that was new to me and “how I approached it”.
Further down in my comment, I referred to:
In ten words or less, that is what I got from the book.
If other people want to treat Design patterns like some sort of Ten Commandments, that is not my problem and it is certainly not my approach.
@ Jeff Read
Ya gotta problem with, say, iterators? Or Facade? or any of dozens, some of which are in GoF and some are described elsewhere.
They are patterns… you see them in reality – some folks draw attention to them.
Eric: Speaking as a former specification-writer and software-tester at a Fortune-500 company, I can tell you that you would help a lot of programmers if you published some version of this article in a platform-independent, peer-reviewed journal commanding the respect of middle managers. ( Software Practice and Experience comes to mind.)
These days, the culture among in-house software project leaders is that project paperwork is for telling implementors what to do, not for making them (let alone their successors) understand what they’re doing. The programmers who tell them that this is a destructive path are correct on the merits, but weak politically. So by arming the programmers with an authoritative reference that the middle managers are violating known standards of good practice, you could do a lot of good. (Just a suggestion.)
>( Software Practice and Experience comes to mind.)
Looked like a good idea, until I got to the part where they want me to surrender copyright on my work to the journal publisher. Fuck that noise.
In my above post, replace “understand what they’re doing” with “understand the world they’re doing it in”. While my statement as written was correct, too, it missed the point of this thread.
Ah, bummer.
@ Thomas Blankenhorn and anyone that can help
re: Peer Reviewed Journal about Software Practices
I think that your idea is good and important. I don’t know how common it is for peer-reviewed journals to want copy-right, but we need to check into this further.
The Linux Journal certainly isn’t appropriate, but when I wrote an article for them (and was paid!), I kept the copyright – all they wanted was “First Serial Rights” – ie. they don’t publish stuff that has already been published. They wanted to publish an earlier article that I had written, but, alas, I had already “published” it on my website, so it was published in the Linux Gazette instead (which doesn’t pay). (When I first got into Linux 1n 1999, I was really gung ho.)
I mention my experiences partly to blow my own horn (of course), but also as a warning to others – if you want something published, don’t put it up on your website or blog. If you want feedback, publish the fact that you have written something and have people ask you for a copy – that way it isn’t already “published”.
If anyone can find an appropriate journal that doesn’t want the author to hand over copyright, now is the time to speak up.
Re: Peer Reviewed Journals and Copyright
I see that my comment, above is behind the times…. ESR has already written a new Post about this problem. I should have noticed but I was working from oldest posts to newest this morning.
Re: Prime Directive for software: Make it obvious
There seems to be a subset of Perl coders who like to write code that looks like a core dump.
This isn’t even acceptable for software that is only to be run once against a set of data.
You don’t want to delete the code or you will have no record of what was done. And if the code still exists, someone in the future will either:
– want to try to run it, or,
– want an explanation of what it does.
Not good either way.
Brian Marshall,
I would recommend against the use of Perl altogether for production code. From an outside perspective, Perl’s operational semantics is riddled with little poofy clouds that say “and then a miracle happens…” You have to exercise enormous discipline to write maintainable Perl; and even then you have to be privy to a bunch of oddball terminology and language idioms that don’t exist outside of Perl’s memosphere. (For example, “grep” means something different in Perl than you’re used to, and it’s something that has a perfectly serviceable name in other languages. And that’s one of the easy ones.)
If you want to “make it obvious” (and you almost always do), consider Python; for large compiled software projects where it HAS to run fast, consider Ada.
@ Jeff Read
I agree. I don’t use Perl and agree that it is not appropriate for production software. It doesn’t “do what I say”, it does (a fairly good job of) “do what I want”, but like you say, it is invisible magic – not good.
As I have said before, I haven’t been paid to code for a number of years. Almost all of the coding I do now is processing text files, for which I use awk. You may recall when I made an Open Source project of the cutest little double-entry bookkeeping system based on two text files and a variety of awk scripts.
There was a flurry of interest when Ada was first introduced. I was horrified – it looked to me like a perfect example of what you get if you let a committee design something – it does EVERYTHING. The interest seemed to wane except for DoD work. But I gather that it has become more common for use outside the DoD. It does force you to specify exactly what you are doing – it is verbose but, in the big picture, it does “Make it Obvious”, so… it is probably a good thing but I am just glad that I personally don’t have to use it.
From what I understand, Python is a great language, and if I had any real use for it, I would definitely get into it. Now, about 99% of what I do can be done nicely (and obviously) in awk – for the remaining one or two percent, I use simple C++ (although I haven’t actually had a reason to do so for… 2 or 3 years).
Sorry about being rude about design patterns – you and I know that they are just patterns that can be addressed in a variety of ways. It is not the fault of either of us that some people turn them into THE WAY.