I just sent mail to the Battle for Wesnoth developer list titled “Why I want to confiscate every dev’s * key”. The points in it probably deserve a wider audience.
I want to confiscate every dev’s * key because while converting the Wesnoth repo I’ve just fixed about the hundredth comment that does this:
------------------------------------------------------------------------------ Fossil-ID: 29314 * Document weapon/second_weapon addition and changes on RC imagepath * function ------------------------------------------------------------------------------
The obvious thing that is wrong with this is the second ‘*’; the bare word ‘function’ is not a list entry.
The unobvious thing that is wrong with this is the first ‘*’. This entire comment should not have been written as a bullet item. Because there aren’t any other items in it! There’s no list here!
A change comment like this is OK:
------------------------------------------------------------------------------ A summary of what I did to fix a random bug * This is the first thing * This is the second ------------------------------------------------------------------------------
A change comment like this is not OK:
------------------------------------------------------------------------------ * This is a random thing I did. ------------------------------------------------------------------------------
That leading ‘* ‘ is just a hindrance to readability, a visual bump in the road. When your comment history is over 50K commits long this sort of small additional friction matters.
The following is also *not* OK:
------------------------------------------------------------------------------ * This is one random thing I did. * This is another random thing I did. * This is a third random thing I did. ------------------------------------------------------------------------------
Why is this not OK?
Well, the obvious reason is that there is no summary line tying all the items together. This matters more in git-land because so many of the tools just show first summary lines.
The less-obvious reason is that if you write a comment like this, your single commit is almost certainly doing too much. You should break it up into several commits, each with one topic that becomes a single (non-bullet) item in its own change comment.
The next time you are tempted to press your ‘*’ key when writing a change comment, stop and think. Is it just going to be noise? If so, stop and slap your own hand.
More generally, give a little thought to what the shape of your comment says about the shape of your commit. Is it trying to describe too many different and unrelated changes in one go? Does it have a proper first summary line that will minimize the overhead of reading it for someone six months down the road?
There’s a programmer’s proverb: “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. Code for readability.”
That applies to comments, too. At the scale of this project [8 years, over 55K commits], such details really matter.
A former coworker had the quote above on his cubicle wall at work. The guy in the next cube had a quote attributed to Brian Kernighan: “Debugging is twice as hard as programming, so if you write code as cleverly as you can, by definition, you’re not smart enough to fix it.”
But pointers to pointers to pointers in C are ok?
But pointers to pointers to pointers in C are ok?
All problems in Computer Science can be solved by another level of indirection — Butler Lampson
— Foo Quuxman
Ditto what Glenn said.
The geeks who long to use every feature provided by the language can turn out code that is nearly impossible to debug. I once found the “++” operator overloaded to fetch the next line from a config file, just because the developer wanted to play with overloads.
The virtue of self restraint is rare (in any field) but some sort of restraint is desirable.
Maintenance is harder than development.
While I’m unqualified to speak about the change-comment mangling, I would add that the format Eric recommends is also considered good English composition: Lead with a short sentence to summarize, then bullet-point the details. (For one reference, see Bryan Garner, Legal Writing in Plain English, section 34, which makes a similar-enough point to apply here.) It’s not the most important point in the world, but may be worth knowing, especially if a sizeable share of developers are foreigners.
So, isn’t Kernighan’s implied recommendation to code at about half your cleverness limit so you’ll be able to debug it? Sure, you might be bored out of your skull during the planning and coding phases, but it’ll get interesting again once it starts to compile…
for(int i = 0; i < 8, i++);
{
if(i == 7)
{
//Why is the code in here never running!?!?!?!?!?
}
}
Two days to find the bug, I kid you not.
The “function” in the first example is probably because of some text editor “helpfully” inserting the * at the beginning of the second line, and the poor chump in question either not noticing or assuming it’s part of some coding style their editor is programmed to uphold. I’ve seen it too many times.
I think I had a similar bug baffle me for awhile, Terry, once upon a time. I feel for you man.
@Terry:
That’s the sort of thing that makes me very puzzled when people complain about whitespace being significant in Python. It really is a feature…
My usual similar goofy trick is to forget to give a register a width in Verilog. It’s interesting — when I’m looking at code, if the register width is declared, my brain will usually almost subconsciously verify that it is wide enough for what is going in it, but if it’s not declared, I might not notice it. Since the default register width is 1 bit, that can be problematic.
Terry: one of the semicolons is a comma, and there’s a semicolon after the for() invocation. Do I win anything? ;)
Yup. The semicolon that’s a comma was a mistake in my typing it for the comment. The semicolon after the for() is the real bug. I think I had my head on the desk for about fifteen or twenty minutes after fixing it. Your prize is, let’s see (rummages around) a clock battery for an EPDM 701 mainframe ;)
@Terry and Patrick Maupin
Another solution that doesn’t require Python’s conflation of rendering with syntax, is to eliminate semicolons by designing the grammar such that there are no ambiguities without them.
Python’s conflation violates the end-to-end principle on the internet, since it forces all client programs and users such as browsers and HTML programmers to respect Python’s raw text whitespace interpretation. Braces don’t suffer this end-to-end slavery, and interested renderers could with a simple algorithm hide the braces and display reflowed with consistent indenting.
I think I had my head on the desk for about fifteen or twenty minutes after fixing it.
This kind of bug is like the Programmers Groin Attack, *everyone* in the audience grimaces in remembered empathic pain.
— Foo Quuxman
Terry: Heh, ironically, I’ve had your typo as an actual bug before. And it took a long long time to figure out why my code wasn’t doing what I wanted it to. vgrep is hard, sometimes, when you “know” what you’re seeing, and so your brain sees it even when it’s not there…
It would be really useful to be able to turn down the autocorrect sensitivity on my visual cortex, actually. :)
Second level digression: I wonder if reading (for example) YouTube comments, with the required autotranslation out of lulspaak and tyop and l33t and txtUl8r, actually makes it harder to see such code errors by retraining your brain to autocorrect more and ever more. I have thought before that reading some of those comments was actually making me dumber by proxy, now I’m seriously contemplating the possibility.
JustSaying: That’s the first reasonable objection to Python’s syntactically significant whitespace I’ve ever seen. The usual objection is “eww, significant whitespace, icky!”.
However, I would point out that anyone who’s showing code in HTML should enclose it in <pre> and </pre> tags, anyway. Otherwise, it gets mangled in other ways.
> The usual objection is “eww, significant whitespace, icky!”.
I suppose it should be possible to write an algorithm which renders or translates Python’s layout or braces as the other, even between preferred bracing conventions.
> anyone who’s showing code in HTML should enclose it in <pre>
Poorly programmed forum software, mail clients, social networking sites, sometimes drop or don’t provide an option for the pre tags or the user has an overriding style sheet, etc.. I am not sure if this is an significant concern. Just pointed it out. I guess my stance is that idealistically I’d rather place it safe, then write a translation algorithm to make it beautiful for my preference.
Don’t Python users disagree about how many spaces should be used for indenting? So still there remains inconsistency of layout preferences.
It only matters within one source file, and projects tend to adopt their own conventions for indentation.
@JustSaying:
I think you mean HTML’s mis-handling of whitespace violates the end-to-end principle :-)
(After all, if a simple space can be mangled, what chance does a video stream have?)
Sure. It’s just obviously not enough of a concern for anybody to bother seriously. It’s not an internet problem per se, so much as an HTML not handling code well problem, and it is extremely easy to find sites that render it properly in pretty much all browsers. It’s mainly a problem for things like this wordpress blog, where you _can_ make code indent properly with the right pre tag, but it’s difficult to remember how if you don’t do it very often.
The canonical number is 4. I think some groups were using 2, but don’t think that’s popular any more. PEP 8 is really a pretty good style guide.
In (my) real life, the far worse issue is sharing code with non-Python types who think tabs are OK. FWIW, I thought tabs were stupid (because some people and code editors thought they meant any arbitrary tab-stop, and others thought they meant 8 characters) long before I encountered Python, and I used to do a heck of a lot of assembly language programming (which is usually done in a columnar fashion). Perhaps those events primed me for the emergence of Python.
>The canonical number is 4. I think some groups were using 2, but don’t think that’s popular any more.
Confirmed. I’ve looked at a whole helluva lot of Python code from different projects and never seen anything but 4-space indents. In the distant past some of those spaces were actually tabs, but the Python dev group waged a successful campaign against them.
(Note: I was at one time peripherally part of the Python dev group, and Guido has given me a standing invite to come back, but I was not involved with that campaign.)
Python the interpreter doesn’t care about what you use for indentation as long as it’s consistent within a source file; it can be a tab character, it can be three spaces, etc. But yes, most projects adopt PEP 8 to a large extent, and the 4-spaces rule in particular tends to be followed more universally than the rest of it. (PEP 8 is a really good style guide, though, and it transcends the intended scope of CPython’s code itself.)
@Mike Swanson:
Sure. The problem comes about when someone starts editing preexisting Python files, and they have their editor configured to a) use tabs; b) not show them the actual tabs vs. spaces and c) not convert all preexisting spaces to their tab style.
The good news is, it fails miserably enough right away that they call me. The bad news is that they call me. In (my) real life this is a much worse problem than the hypothetical not-being-to-extract-meaning-from-a-Python-web-snippet-due-to-bad-layout problem.
The canonical number is 4.
The emacs python mode uses 4 as default, *and* automaticly maps the tab key to do the Right Thing.
Was worth learning emacs just for getting this right.
— Foo Quuxman
Generally I find that I come across more instances of the ‘let’s put everything on one line’ school of exasperating commit message writing.
@dak180:
I’m much more apt to be guilty of this than the other:
svn commit -m “I did a whole bunch of things here, like (a) and (b). Oh, and I rewrote the entire parser because it was broken.”
I think I was trained by early VCSes to always do the commit with a comment if I wanted the comment in the repository, when more recent VCSes will almost always launch an editor for you if you don’t explicitly add the comment.
My impression is that most programmers view commit comments as just another form of documentation, and therefore something they don’t want to do except under duress.
Not universally true, of course, but very common.
@Michael Hipp
> …documentation, … something they don’t want to do except under duress…
Although too much verbiage violates K.I.S.S. and 80/20 principle of diminishing returns (because I have to wade through it too and unnecessary complexity is an indicator of insufficient application of unifying and simplifying reductionist thought), yet I find that strategically chosen (terse one liners if suitable) documentation helps me work faster because it reminds me more quickly WTF was I thinking.
A little off topic, but speaking of Wesnoth, are you aware of and do you approve of the four dollar android app that rips off Wesnoth?
>A little off topic, but speaking of Wesnoth, are you aware of and do you approve of the four dollar android app that rips off Wesnoth?
The only version I knew of before you brought it up was the free one. I don’t think I’d say the pay version “rips off” Wesnoth, because nothing in our licensing terms precludes what that dev did. Anyone else is free to do likewise, and if he’s charging more than his labor for the port is really worth someone will undercut him.
I wrote “>Anyone else is free to do likewise, and if he’s charging more than his labor for the port is really worth someone will undercut him.”
Someone should. Turns out the port is incompetently done; even the pay version won’t load campaigns.
ilcylic
It would be really useful to be able to turn down the autocorrect sensitivity on my visual cortex, actually. :)
Oh, god, that is such tempting but subtle transhumanism. The alternative (possibly better) would be to be able to turn on the proofreader function, which makes errors jump out visually.
In case you’re wondering, this is a thing vim does, which is probably to blame in the majority of cases you’ve encountered. It’s an attempt to allow easy editing of comment blocks that read like this:
/*
* some stuff
* some more stuff
*/
But it doesn’t do it A) only with languages that use /*this*/ style of comment or B) only when in the context of such a comment.
And I see I’ve missed the entire point. Well, in defense of the people writing these, if it _looks_ like the changelog file has a formal syntax (and in many cases it does, at least enough for editors to syntax-hilight on), and it looks like that syntax includes “entries must consist of a bullet-list of things changed” (because other entries have done so), the path of least resistance is to write your entry in the same format as preceding entries.
As for “your single commit is almost certainly doing too much” – this probably ties back to when a single changelog entry would be made per _released version of a project_, rather than per commit in an easy-to-use VCS.
Ah, another reason to use emacs, which is pretty smart about detecting bullet lists and indenting successive lines correctly.
1) Make a language more suitable than C that does not use the asterisk in its syntax. Write or rewrite a popular operating system in it, so that programmers are finally forced to use it.
2) You are nitpicking over “grammatical” errors, albeit they are in an artificial language. Such errors are all too common in peoples’ use of natural language as well. Open source software developers who contribute to major projects tend to be perfectionists (I have tried to be a perfectionist myself). Other people, who constitute the majority, do not care as much.
3) Make a better way to mark bullet points in programming languages, given the age-old obsession with pure text syntax in such languages.
Grammatical errors are still errors; being concerned about them is not “nitpicking”. If you think it is, you have no business programming computers. They don’t “know what you mean, not what you say”. GIGO.
By putting the apostrophe after the “s”, you’re talking about how entire peoples use language, not how individual people use it. That completely changes the meaning of the sentence. When an entire people makes a rule of a usage pattern, it becomes the standard of their language, not an error. Arguably at that point that people then has a different language from another people who use a similar language.
But you probably consider the above to be “nitpicking”, which is why you’re so careless in your apostrophe placement in the first place.
Use of a markup language like HTML or XML with explicit semantic significance for the structure would be an interesting choice here, but existing usage is so well established that it’s unlikely to catch on.
I am lucky in that orthographic errors (“spelling” errors) in my native language, Polish (the language with more declensions than Latin ;-), e.g. misspelling of “u?ytkownik” (“u\.{z}ytkownik”) (pl. user) as “urzytkownik” stand out visibly for me…
I’m arguing with a friend about whether using a comma where a semi-colon belongs is a mistake which might be made by an experienced programmer. He finds it inconceivable that an experienced programmer would do that, and I think he’s overestimating how common his level of accuracy is.
>I think he’s overestimating how common his level of accuracy is.
So do I. I certainly qualify as “experienced” but I still make that sort of mistake, though quite rarely. Once every 2-3 years maybe.
@esr
>So do I. I certainly qualify as “experienced” but I still make that sort of mistake, though quite rarely. Once every 2-3 years maybe.
Indeed, everyone makes mistakes. What is inconceivable is that a decent compiler and development system would not detect 90% of the typical error modes and warn of them, and equally inconceivable that an experienced developer would either ignore or turn off those warning messages.
(Of course by inconceivable I mean “nuts”. It is conceivable because people do it all the time, but it utterly escapes me why programmers would turn off error detection in their programs, even if there is a small amount of noise.)
In regards to the original example, I totally get that we all make dumb mistakes, I do it all the time. What I don’t get is how you would not find it in five minutes by stepping through the code with a debugger.
Is it possible for you to update this page since the content is pretty old (like suggesting AFL over Apache 2) although lot of it is still relevant.
http://www.catb.org/~esr/Licensing-HOWTO.html
Thanks.
@The Monster
Regarding your comment about grammatical errors.
I neither know whether you are a friend or acquaintance of ESR that you became so vehement in your/his defence, nor am I interested in starting/continuing a flame war with you. I may not “have any business programming computers,” but I do have 20 years of experience in programming.
Regarding your anger about the misplaced apostrophe, I frankly admit that it was a genuine mistake that I did overlook and should not have made, although, with all due respect, I refuse to admit that I am always “so careless” in my “apostrophe placement.” One of the main points raised by ESR in his article, however, was, in my opinion, “nitpicking” (you are welcome to look up the meaning of the word since you seem to be so angry about it) because he did mean it; it was not just a mistake, and that is what is not restricted to programming languages or programmers. People in general do make such mistakes in their day-to-day use of natural language, which you can verify by visiting any of several Internet forums.
Jessica: “What is inconceivable is that a decent compiler and development system would not detect 90% of the typical error modes and warn of them, and equally inconceivable that an experienced developer would either ignore or turn off those warning messages.”
The problem is that compilers aren’t as smart as programmers. Those warning messages are a nuisance when you’re trying to do some things, and having a 40-minute build blow up because you did something that the compiler disagrees with but is a reasonable thing to do (like an assignment in an if statement) is highly annoying. So is upgrading your compiler and discovering that it has found something new to complain about that your code just happens to do, a lot. In cases like that, turning off compiler warnings is a natural reaction.
@ Saurav Sengupta
The Monster expressed concern about grammatical errors; he didn’t indicate that he was angry in any way.
[sarcasm>I am sure he appreciates your permission to look up the meaning of “nitpicking”.I like your avatar – a snow flake.</snarky]
hmmmm… I had an end-sarcasm fake-tag, a blank line and a begin-snarky fake-tag which WordPress ate.
Personally, I don’t like auto-indent, syntax-based coloring or any other “smart” editor behavior. I want my brain trained to write and see the code.
@BRM: ” If you think it is, you have no business programming computers.”…”But you probably consider the above to be “nitpicking”, which is why you’re so careless in your apostrophe placement in the first place.”
This is what “The Monster” said, and in the ordinary use of language to express thoughts and feelings, this shows anger or rudeness.
Case in point: Since I was typing my last comment on a phone, I made a grammatical error and also ended up splitting the comment in two (I also could not figure out how to delete or correct the comment). Now that it does contain a grammatical error, you can see that this phenomenon does occur and that it is not restricted to programming languages.
@ Saurav Sengupta
I can see how you could interpret The Monster’s comment as being rude, but I still can’t see how you could interpret it as expressing anger.
@BRM on Tuesday, February 19 2013 at 4:52 am:
As I had mentioned previously, if anyone wants to insert pseudo-tags into their posting, you have to do it like: “<helpful>Hope this helps</helpful>”, which I achieved by typing & l t ; blah, blah, blah& g t ; (of course, without the spaces in the markup).
As an exercise for the HTML masochistic, think about how I had to actually type that previous line. (HINT: it contained the “&” string several times….)
Writing comments in code is very much like writing an autobiography. You don’t want people to understand the details of your sordid little project so you try to dress it all up with flowery language and quaint (but usually indefensible) justifications. The truth is, most programmers write code the way they run their lives. That is, quietly trying to hide the fact that they’re human just like everyone else. Therefore, code comments are analogous to explaining yourself to the school superintendent. Of course, to your larger point, banning the use of one character would only encourage other bad habits much like banning liquor or manga. The law of intended consequences rules supreme over all other axioms.
@Jay Maynard
> The problem is that compilers aren’t as smart as programmers.
FWIW, I disagree with this. For sure programmers are smarter at some things but compilers are definitely way smarter than programmers at other things. For example, compilers are MUCH better than programmers at DFA. This realization actually makes me write code differently. In the past if I had a local variable that had a value calculated by some nest of logic I would always give it a default value at declaration. I now almost never do that. Why? The bogus assignment at declaration confuses the DFA analysis from detecting a fault in your nest of logic where you don’t determine the correct value.
To give a super simple example:
int i; if (predicate1 || predicate2) i = 1 else if(predicate1) i = 2;
This little nest of logic has a bug where I forget what to set i to if !predicate1. The compiler detects it immediately. It isn’t too hard to see the bug here, but with a more complex program these sorts of bugs are VERY hard for programmers to find. Yet DFA finds it instantly.
> Those warning messages are a nuisance when you’re trying to do some things,
It is a matter of programming hygiene. As you build the software you eliminate the warnings, even if it means expressing yourself in a way that is a little clearer to the compiler. Clarity is not a sin.
Of course it is a matter of philosophy. The above example is wrong. I never use short variable names like i. I give it a real name to make my code self documenting. I know most people here disagree, but I guarantee you would find my code extremely easy to understand because of that. For me, clarity is vastly more important than concision or putative “efficiency.”
> having a 40-minute build blow up because you did something that the compiler disagrees with
I find this scenario unlikely. When I write stuff I add small bits to the program and do partial compiles and links, I suspect you do too. So as long has you are disciplined in removing the warnings as they come up (which is to say you verify that the potential errors the compiler identified are not in fact errors at all) this situation doesn’t arise.
FWIW, I am also impressed, I don’t remember the last time I saw a program take 40 minutes to compile. Perhaps you work on gigantic systems, but I work on big ones and they rarely take more than ten minutes to compile. You might consider an SSD :-)
All my code compiles with zero warnings at the highest warning level. It makes a lot of difference to my bug counts. FWIW, the fact that you are telling the compiler redundant information like explicit typing is one of the reasons I much prefer statically typed languages to inferred type languages. The more information you can push into the compiler about your true intentions the more likely it is to find problems.
@Eric: I guess I’ve found something else for us to disagree on. I’m a big proponent of tabs. What else would you expect from somebody who actually *likes* C++?
@Jessica: I work on a product where a NULL build takes ~2 minutes. A trivial change (think space removal in a comment) takes ~8 minutes and a full build takes ~30 minutes. This is on a build farm backed by 64 blade servers (except for today where it is apparently taking 2 hours).
We are very big on having all warnings be treated as errors, we have a ratcheting mechanism for lint errors, and Coverity problems all must be addressed in short order. I will agree with both Jay that breakage due to warnings is really annoying, and yourself, that code cleanliness goes a long way over the long-term. Every compiler upgrade takes weeks to do because we have to clean up a lot of stuff just to get the code to build. Still, it’s probably the best overall plan for reduction in technical debt we have.
@ John D. Bell
<gratitude>Thanks, John</gratitude>
@ Saurav Sengupta
Somehow, our culture has deemed it “rude” to discuss errors in spelling, punctuation, and grammar. Since you referred to such as “nitpicking”, it is a reasonable inference that you agree with that notion. Personally, I think it’s inconsiderate of one’s readers to casually make those errors which require them to ponder “what did the author mean to say there?”.
Example: The common English expression “All X are not Y” is probably intended to mean “Some X are not Y” but the literal wording cannot have any meaning other than “no X are Y”. I simply move the negation where it belongs: “Not all X are Y”. To the average person, this is probably “nitpicking”. The average person has no business programming computers.
If you wish to consider my remarks rude, that is your right. But I stand by my assertion that people who take the position that it is “nitpicking” have no business programming computers.
You can type
rm -rf . /and think it rude of me to correct you. But when your computer obediently deletes every file in every mounted writable filesystem, you might think that’s far ruder.It is impossible for a computer to “know” when you write a syntactically valid statement that happens to not mean what you thought it meant. It’s possible for certain common error forms to generate warnings, but there will always be less common errors that won’t be caught.
>If you wish to consider my remarks rude, that is your right. But I stand by my assertion that people who take the position that it is “nitpicking” have no business programming computers.
Monster is quite right. While it is not always the case that people with sloppy punctuation, spelling, and grammar are inferior programmers, that’s the smart way to bet.
Patrick Maupin said: In (my) real life, the far worse issue is sharing code with non-Python types who think tabs are OK.
I my real life, I fortunately don’t have to deal with Python, and can thus use tab to indent rather than wicked, vile “spaces”.
Hit tab to indent one level. Hit backspace to remove the indent. No counting, no having to hit space 2/4/8 times (whatever “the standard” is).
(Reminds me of make(1) and the utterly insane way it can’t handle spaces-not-tabs.
The entire point of whitespace, coming from a printing and layout>/i> background, is that it’s filler, not meaningful. [In terms of communicative content; at the artistic level we’re talking a whole ‘nother ball-game].
Making it meaningful, ala make(1), or even moreso, ala python, makes me twitchy. I am perfectly willing to agree with Mr. Maynard that this boils down to “icky” and that at the pure semantic parsing level it makes no difference.
But aesthetics matter, too.)
@Sigivald:
Either you work by yourself, or you work with someone with the power to dictate how many spaces a tab is.
Even before I used Python, I would get files that were unholy messes, completely unreadable by human (though easily readable by compiler). Until I set the tab stop to whatever the original programmer used.
Unfortunately, not every programmer was that considerate. Some files had sections that were obviously tabbed at 4 characters and other sections at 8 characters.
Editors are smart enough these days that you can still have the tab key and backspace key do the right thing if you care to.
Sure. But I have seen examples of bad tabulation that far exceed the asterisks that are driving esr crazy, and rise to the level of making the file unintelligible for any sane human.
On this we agree.
On this, we don’t agree at all. Proper filler is much more than “artistic.” It would be true to say that the filler is redundant information, but you have to realize that almost all human communication is filled with redundancies that are actually useful, and that make the communication faster, easier, and less error-prone. Yes, we call that “beautiful” or “artistic,” but what we really mean is “I can look at this for long periods of time without going bat-shit crazy.”
But … but … I WANT Python’s conflation of rendering with syntax. And it’s not hard at all to preserve whitespace in HTML. And braces introduce another kind of slavery: making your braces match your indentation. Computers were made to count and match.
@Patrick: it was make’s insistence on tabs-not-spaces that caused me to make “invisible” spaces (that is spaces which don’t move the cursor over because they are followed by a tab) visible in Freemacs, and when the cursor is sitting on a space or a tab, the space gets rewritten into a visible character. Big dot for tab, little dot for space. AFAIK, no other editor ever did that, but BOY was it nice to have.
@Sigivald:
>The entire point of whitespace, coming from a printing and layout>/i> background, is that it’s filler, not meaningful. [In terms of communicative content; at the artistic level we’re talking a whole ‘nother ball-game].
Whitespace is meaningful at more than just the artistic level: It is *structurally* meaningful.
“Thequickbrownfoxjumpsoverthelazydog” and “The quick brown fox jumps over the lazy dog” aren’t much different communicatively or artistically. The whitespace is significant, though, in that it helps in the identification of structural elements of the text.
It just so happens that the roll whitespace has in Python is that of delimiting structural elements.
@The Monster, @esr: The first point that Eric had made in his article related to a way of bulltet-marking points in comments. It is always best to strive for perfection in anything you do but the gist of what I intended to say either missed both of you because I was incapable of getting it across or you were incapable of understanding it. In every art and technology form, the creator should always do everything to ensure that the result reaches as close to perfection as possible. I agree with Eric’s point that indiscriminate use of a particular tool (the asterisk in this case) can lead to problems but I, at least, do not generalise it to infer that everything that the programmer will do will be sloppy and with incorrect syntax. I do not consider that a mistake at some point means a mistake at every point. I do not know whether great programmers’ code compiles without errors the first time every time, but I am not as great a programmer as Eric or The Monster and code that I write does sometimes contain errors that either the compiler catches or that I examine and correct myself. I think that I too can safely say that if your brains are more like computers and less like those of humans to the extent that you find it hard to understand the meaning of comments containing small errors, then you do not have any business being human. I say “small errors” because the average human brain, which you seem to have fun in deriding, is infinitely more capable than a computer at sifting through text to decipher the meaning that the author intended to convey. It is no wonder that the “average” person finds everything from Unix to Linux difficult to approach. If the best programmers like you do not consider or never build for the average person, this is bound to happen. The irony of the matter is that Eric has himself acknowledged this in his book, “The Art Of UNIX Programming.” I believe that it is better to be a good human before or instead of being a great artist, which is why I stand by the “average” person.
Jessica: I regularly build a 1.3 MLOC C++ package with roots stretching back to 2002 or so. One change to the right header file or some debugging options, and you get a full rebuild. The build process has most warnings treated as errors, but there are some warnings disabled because the compiler decided something needed to be warned about long after it was done throughout the code. The package is also portable to Visual C++ as well as G++, and there are warnings turned off for each because the other compiler doesn’t consider them problems – and does consider the fixes for them problems.
And I fully agree that clarity beats efficiency every time. Even so, at some point, it’s a better use of programmer time to simply tell the compiler to shut up and accept the code.
Sigivald: Get a better editor. If your editor can’t translate the tab key to the appropriate number of spaces, it’s probably too basic to program in.
BRM: I do like syntax coloring and autoindent. Both make it easier for me to see what the code actually is, rather than what my brain might see it as – wrongly. It’s an assist with the mechanical aspect of programming, and that’s exactly what computers are there for.
Re: syntax coloring
I used to view green-on-black, eschewing the more modern white background. Syntax coloring was intolerable until I switched to a white background.
I also find that what you can tolerate visually may be age-related.
> I agree with Eric’s point that indiscriminate use of a particular
> tool (the asterisk in this case) can lead to problems
Indiscriminate *anything* causes problems.
Intention is not optional.
@William O. B’Livion, @The Monster, @esr, @anyone else who is more of an automaton than a human: I know that indiscriminate “anything” causes problems, but I like to talk within context. Anyway, I am sorry that I ever commented on this article. You win, simply because you never understood my point. Arguing with both ignorance and adamance is pointless. You are all obviously free to hold whatever opinion you see fit, just as I am; the only thing is that due to your extreme knowledge of programming and logic, you lack humility and have forgotten what it is to be human. My inbox has become flooded with messages from this thread. I will have to either unsubscribe or block messages from this thread.
>you lack humility and have forgotten what it is to be human.
Such a precious, precious little flower you are. And what nasty people we are for criticizing helpless widdle you. *sob* How will we live with ourselves?
Er, quite easily, actually. We have little patience for carelessness and sloppy thinking because the tools we have to work with don’t either – it’s an adaptive response. You are being mocked because you don’t merely engage in sloppy thinking, you implicitly defend it. You remind me of this idiot.
If you think this makes us un-human, you live a remarkably coddled and sheltered life.
@William O. B’Livion: Mistakes made by humans are not always intentional.
Eric and his blog really are “Armed and Dangerous”!
@Sigivald
> The entire point of whitespace, coming from a printing and layout>/i> background, is that it’s filler, not meaningful.
In a sense I agree, I also think the Python thing is nuts. However, the plain fact is that whitespace has huge semantic content for programmers. If you disagree then take a piece of well written C code, remove all the indents and try to understand it. Even worse, take the same piece of C code, remove the indents, and then add random indentation. It makes the code much harder to read.
However, I think the Python solution is suboptimal. What I would do if I were writing a compiler is that I would go with block markers like {} but the compiler would also validate the whitespace indent, and report indentation that did not correspond to the appropriate block indents as a warning. Basically, like C, but if you did the indents wrong you’d get a compiler warning questioning you, which would catch a lot of errors.
Actually, what I’d ideally do is have the errors and warnings from a build system work like spell check in a document: not only tell you there is an error or warning, but also give you two or three suggestions on how to correct it, and auto fix with a single click.
_Variable gps_prot does not exist, did you mean gps_port (click to auto correct.)_
90% of basic syntax errors could be auto corrected that way, and that would go a long way to solving the problems Jay mentioned.
> But aesthetics matter, too.)
Right but for more than reasons of prettiness.
Jessica, why make the programmer input the same information twice? We need to be decreasing the work involved, not increasing it.
@Saurav Sengupta:
> Mistakes made by humans are not always intentional.
You are correct. I make mistakes *all the time*.
Some things I do deliberately thinking the outcome will be different.
Some things I do without much thought (without intention).
My goal in life is to be *more* intentional, to have the appropriate level of thought devoted to whatever activity I am doing. Because then at least I’ve *tried* to avoid negative consequences.
I do not disagree with your position–I am probably *worse* a programmer than almost all here. I almost never venture out of the shell script ghetto.
My only contention is that attention and intention are critical to doing the right thing.
@John D. Bell > markup stuff
Great, now how do we insert links with text, or those nicely formatted quotes some people use? What markup language does this use, anyway? Is it really just HTML? test test
Ah, apparently it is just HTML. Or, at least, HTML plus converting line breaks into paragraph breaks. Whenever I encounter a comment box I assume it’s something like markdown or bbcode or wikitext, and couldn’t find any documentation… never thought to try HTML.
@Russ Nelson
> it’s not hard at all to preserve whitespace in HTML
But we don’t always have programming control over the client displaying the code. The end-to-end principle says for highest reliability we shouldn’t rely on intermediary nodes to be smart, and that what comes out the ends should be intelligible only to the end, i.e. the intelligence goes at the ends.
Braces are less likely to be lost by a dumb intermediary than whitespace.
And then your intelligent end client can render the (brace-delimited) blocks with a consistent style of indenting automatically for you (even removing the braces if you so desire), and differently for the other guy. You both will always see everyone’s code consistent with each of your personal preferences. This would (in theory) eliminate the need to preach and coerce a consistent style.
@Jon Brase
> Whitespace is meaningful at more than just the artistic level: It is *structurally* meaningful
For delimiting blocks, it adds no more Shannon information than braces, which is proven by the fact that an algorithm could automatically reformat to any preferred style.
@Jay Maynard
> Jessica, why make the programmer input the same information twice? We need to be
> decreasing the work involved, not increasing it.
For the same reason accountants do double entry book-keeping. For the same reason we spend lots and lots of time typing code for unit tests, or typing comments.
Typing a few extra characters increases the amount of work negligibly, and in fact usually zero because modern editors auto indent. Almost any extra amount of typing is worth it to eliminate bugs. Programmers spend several orders of magnitude more time debugging than they do typing. I can’t remember the exact metric, but McConnell’s “Code Complete” bristles with statistics like that.
Back when I wrote code and led programming groups, I really tried not to get into format wars. Arguments about where on a line an open curly bracket went was not productive time to my lights.
Indeed, one time one of the more rabid Knights of Whitespace checked out the entire code base of a large project, ran it through his own formatting and checked it back in.
I made him choose between backing it all out or a horsewhipping.
@JustSaying
> For delimiting blocks, it adds no more Shannon information than braces, which is proven by the fact that an algorithm could automatically reformat to any preferred style.
Since I am in a pedantic mood, let me point out this is not in fact correct. It adds no more “Shannon information” from the perspective of the compiler, but the information content is larger than that from the point of view of the code reader.
You cannot, for example, take my code, then transform it into a standard formatted example, and then translate it back to the original. Information is lost in the transformation. Not information important to the compiler, but information important to me.
Often, for example, in a place without coding standards, you can tell who wrote the code just by looking at the style.
Similarly, you can rip out all the comments in the code and the compiler won’t care, but you have definitely lost “Shannon information.”
@SPQR:
At one point, I built/maintained a formatter that kept my project pretty-printed the way I wanted.
In self-defense.
Because while I completely agree that “Arguments about where on a line an open curly bracket went was not productive time to my lights” I will also state most emphatically that gratuitous changes made to spacing is the worst thing possible in the repository.
I had people who used strange editors and obviously noodled around in them. I would get changed files where the _only_ change was whitespace added at the end of a few lines.
It drove me nuts.
So if they can’t act like adults and insure that they don’t make gratuitous changes, I’ll treat them like children and tell them “here, run this script before you give me the code back.” Then, of course, because they still acted like children, they wouldn’t even bother running the script half the time…
If you’re dealing with intransigent foreign contractors, try never to be “in charge” unless you have firing power and a demonstrated willingness to use it.
@JustSaying:
FWIW, I absolutely agree with Jessica. Even Shannon will tell you (well, would if he were alive) that you can get more information through a noisy channel if you have the right kind of redundancy. And, btw, there is typically a lot of noise between the screen and your brain, and even more between somebody else’s brain and your brain, when their fingers and keyboard are in the middle.
With English, we have redundancies in spacing, spelling and grammar. If you are a native English speaker, and the sentence “He like ice cream” doesn’t set off a warning in your grammar detector, you probably have no business programming. This, despite the fact that the “s” you expect on the end of “like” is fully redundant information.
It’s worse than that. He could, in fact, have much in common with ice cream.
This is good stuff. In the same vein, I’ve been telling people that “if you use the word `also’ while writing a commit-comment, then you should think about splitting that change into multiple smaller, better-scoped commits.”
I’ve been trying to find or pull together a good collection of `best practices for modern VC systems’ (both in general terms and for specific systems), e.g.: `summarise the change concisely in the first line of a commit-message’, `use topic branches’….
If anyone has links to any such `VC best practices’ guides, I’d love to see them. Most of what I’ve found so far are `getting started with ‘ guides that are really mostly focused on things like command-syntax and explaining the big differences between DVCS tools and Subversion or CVS, but which include a few good interspersed words about best practices….
@rozzin
> If anyone has links to any such `VC best practices’ guides,
You should check out ericsink.com. Eric runs a company that makes a scrappy little VC system in the Windows world, and consequently has lived and breathed version control for several decades. He always has interesting stuff to say on VC. I believe he also wrote a book on best practices.
I’m geeky enough to find it really interesting.
@JustSaying:
>For delimiting blocks, it adds no more Shannon information than braces, which is proven by the fact that an algorithm could automatically reformat to any preferred style.
My point wasn’t that it adds more information than braces do. The point was that it adds as much.
That said, from the perspective of a programmer’s brain, as opposed to the perspective of a computer, it does add more. This is a valid C program:
int main(){int i;for(i=0;i<10;i++){printf("Hello World!\n");}}
This is not, but would be parsed more quickly by a human being (and can easily be transformed into the other):
int main()
int i;
for(i = 0; i < 10; i++)
printf("Hello World!\n");
Stupid tab-eating comment form… (Thus demonstrating JustSaying’s point about lossage through a dumb intermediary).
@rozzin:
This is true. OTOH, it’s no good to strongly encourage people to take what they have done and then tease it apart into constituent components and commit those separately — unless you are fairly certain they will re-run the regression before each commit.
@Jessica Boxer
> It adds no more “Shannon information” from the perspective of the compiler, but…
> You cannot … transform it into a standard formatted example, and then translate it back …
> Information is lost in the transformation … information important to me.
If your style is not random and thus can be expressed in an algorithm, then indeed it can be transformed back by your algorithm which you can use to view every source code file written by any person. The white space adds no Shannon information because even it is random, it is not Shannon information.
@Patrick Maupin
> you can get more information through a noisy channel if you have the right kind of redundancy
Agreed. There is a benefit to saving the consistent white space in the file along with the braces. Realize this could even be automated by the editor needing only the braces to determine the whitespace, so it isn’t additional Shannon information.
And if the indented block is more than two lines, then the relative clutter of the braces is diminishing. It is also possible to eliminate braces for delimited statements (e.g. if-else, do-while, if-end, for-end, while-end), if the grammar requires that all one line blocks must appear on the same line as the statement that spawned the block.
And I have no qualms with Jessica’s point that some people may find they mismatched braces and/or indenting often enough that the compiler could the option to check for any style of consistently matched indenting and braces. Those who don’t need the feature, could turn it off. Or if the political cost is acceptable, the compiler could enforce a single style for consistent indenting, if we are most concerned about noisy channels.
> must appear on the same line as the statement that spawned the block
Or optionally single line blocks must be terminated with ‘end’ keyword in my example.
> if we are most concerned about noisy channels
Logical typo. I mean if we may want to enforce consistent style if we are most concerned about the existence of dumb ends, i.e. not performing the algorithmic transformation to each viewer’s personal consistent white-space and block delimitation style.
The noisy channel destroying white-space (thus losing block delimitation) is an orthogonal issue.
Perhaps I like the keyword ‘end’ more than ‘}’, because it eliminates the opening brace which programmers argue about whether it should go on the same line as the spawning statement, or on the its on line and then at an arguable indent level. And because ‘end’ is much easier to see, not only because it is longer (but not too long), also the programmer is unlikely due to aesthetics hide it at the end of a line of code.
I really like the Go language’s way of dealing with code formatting: they have a tool called gofmt which converts code to the One True Canonical Style. It comes with the compiler, it’s actively maintained, and everyone is strongly recommended to use it before every commit. The whole standard library is formatted this way, as is about 70% of code seen in the wild. Every major text editor that has Go code support also includes automatic gofmt integration. It has effectively killed the debate over minor details of code formatting.
Sometimes, I wonder whether the authors of Go really thought this would be a big win, or if they were just sick of getting into arguments about where curly braces should go, or how many spaces are in a tab.
I have always used blank lines to delimit logical blocks in code. Examples include:
– between class declarations in a header file
– between blocks of code that do some sub-task in a function
In light of the discussion, above, this kind of white space is different in that some of it can’t be reduced to an algorithm because it is not redundant information, such as my second example.
Comments?
@esr:
>You are being mocked because you don’t merely engage in sloppy thinking, you implicitly defend it. You remind me of this idiot.
You finally did make me both laugh at you and pity you. By calling me an idiot or likening me to someone who is an idiot in your opinion, you have simply confirmed what I said: that you do lack humility. Since you seem to have forgotten everything related to humans, let me remind you that humility is a human quality. I may be an idiot, but thankfully that is better than being as conceited as you.
> We have little patience for carelessness and sloppy thinking because the tools we have to work with don’t either – it’s an adaptive response.
By saying this, you have again confirmed what I said: that you are more of an automaton than a human. The tools you work with are just that – tools. They are mechanical or automatic systems. You, despite being born a human (superior to automata), take pride in being like them.
> If you think this makes us un-human, you live a remarkably coddled and sheltered life.
Believe me, I don’t have the least interest in sharing with you the ordeals that I have gone through for more than 15 years of my life because, being the precious, precious conceited automaton that you are, you will be incapable of understanding even one bit of it because it was related to being human, which you are not. Pity you.
Eric Steven Raymond. How many people have even heard your name outside the cocoon of “hacker” culture that you are a part of? You have made a website, this blog, which people like you visit and comment on your “anti-idiotarian” articles and that gives you as much pleasure as a narcotic gives an addict. I just happened to come across this particular article, but no doubt this type of “everyone other than us (hackers) is an idiot” philosophy is what you live by. Indeed, I do remember previously seeing similar stuff authored by you and the likes of you. I also happen to have read that you have been a fan of science fiction. Perhaps that is what inspired you to behave like a robot, but since that happened after you had come of age, when you should normally have had enough maturity to be able to sift through literature, it seems, from a neurobiological point of view, that it was you who lived a coddled and sheltered life where people used to mock you because you were “different,” and that is why you sought refuge in things that you thought or found to be uninteresting to or beyond the capabilities of “mere” humans.
I don’t know how much interest you have had in the scientific research done on human and animal brains, but I have always followed it very closely, as well as following all development in computer and robotic sciences. I am interested in all of science, not just a small part. For example, since I have been programming for more than 15 years, I have coded everything from algorithms in machine and assembly language to user interfaces based on ergonomics and cognitive science. Take your website (this blog) for instance. Apart from the icon in the search box looking like a copy of the one in Windows XP (which makes a bad impression), there is no way a visitor can easily discover what this or you are about unless the visitor happens to click your photo and visits http://www.catb.org/~esr/. On this latter website, the information is outdated, let alone the cramped and unfriendly (yes, unfriendly, even though you don’t care about friendliness for the user) layout. If you had cared about the average visitor and you no longer maintain the older website, you would have made the necessary arrangements for gathering information on this website itself.
You and hackers like you have always made the critical mistake of ignoring the world, the “average” person, at your own expense. Today, when Linus Torvalds publicly hurls insults at people, it angers them, and that is a natural reaction, not an idiotic one. You people have created a vast amount of open source and free software but what is happening now? The “average” person cares only about those parts of it that appeal to them by catering to their needs. Google makes Android user-friendly and the “average” person is happy to use it. Canonical makes Ubuntu user-friendly and the “average” person is happy to use it. But you know what? These companies, these people who build for the “average” person, may not be as clever as you but they don’t shout to the world, “Look at what we have built! You average people are so stupid that you are utterly incapable of even understanding what it is about! Look at you idiots!” This is what sets the average person apart from people like you. You have created and held on to your world of open source software but your pride, when it goes before a fall, is not likely to make you more well-known or friendly, or, worse yet, it may even let the better parts of what you have created slip from your grasp. Even scientists do better than you; at least they try to explain their theories to the layman in a way that the latter can understand. Until and unless you and your kind can understand the average person, until and unless you have empathy, you, sir, are not going anywhere outside your sheltered cocoon about which people don’t give a damn.
@BRM
> I have always used blank lines to delimit logical blocks in code
The prior discussion I think was referring to indented blocks with horizontal spacing, not to logical sectioning of code with vertical spacing.
I am not aware of much contention about vertical spacing, thus no need for consistent style.
The only problem I expect is incorrectly programmed clients which don’t render the newlines. This means that using braces instead of horizontal spacing is not going to help so much with the end-to-end principle (all the code squished onto one line) in this context (see below though).
One way HTML could have reduced this possibility is if there was an opt-out tag <nbr> before newlines that are to be ignored. But that would get quite noisy, so opt-in is <pre> is required to have newlines respected.
There could be a standard browser plugin that one could use to highlight code and right-click to choose “render preformatted”. It could even then apply your preferred style to different languages.
If we consider such a plugin running as a standalone application so that we can copy+paste from any kind of application (e.g. mail client), where the original (HTML) source is not available, then the braces will not have been lost (where the horizontal and vertical spacing was), thus a reason to favor redundant horizontal block delimiters. And I suppose such a plugin that knows the syntax of the language, could also be smart enough to make reasonable decisions of where to break lines to make the code readable.
@esr: Since I could not figure out how to remove my membership (probably because I’m an idiot), can you please remove my account from the membership of this website?
@Saurav Sengupta
I ignored most of the debate between you and your claimed antagonists, but I think Eric’s valid point is that to be good programmers so that our world functions reliably (probably the entire world relies on us believe it or not), we must have precise logic and syntax. That doesn’t stop hackers from expressing emotions and humility in other contexts where it is beneficial. Making this point so bluntly, is intended to serve YOU well. If someone was rude to you, they might have had a slip of mood, or perhaps they were not impressed by your logic and exposition up to that point. If you subscribed via wordpress, I guess you have to login to wordpress to unsubscribe. If you haven’t already done so, you may have to create a wordpress account with the same username and email to access the settings.
@JustSaying: I had tried performing all the WordPress steps you have mentioned. It did not work, but I think Eric has deleted my account anyway as requested, so problem solved. Regarding your other points, excessive pride in oneself in any situation is considered ill-mannered throughout the world in general (only the hacker culture maintains the contrary), whether the world depends on you or not. Let me quote a dialogue I once heard:-
Patient: “I have never seen God, doctor, but for me you have come the closest.”
Doctor: “Believe me, sir, it is far easier to be God than to be just a good human.”
I don’t think all programmers are as snobbish as Eric and his fellow hackers and it is a good thing too. Neither you, nor I, nor Eric was born with complete knowledge of all programming. Everyone has their learning days. When you forget those days and begin calling others “idiots,” you have already fallen in character far more than the idiot.
It is fine that the whole world depends on all of you for their survival, but you would do better to keep that fact to yourself because proclaiming it by calling those who depend on you “idiots” is obviously not going to raise your esteem in their eyes.
It might be admitted that esr has attitude, but likewise that he’s demonstrated more than adequately that there’s a reason for it: he can back it up. Whereas you, saurav, are demonstrating attitude without any such backup.
I tend to concur on the notion of strict spelling and grammatical structure as being essential to programming because, of course, the computer will not let an improper semicolon slide. Is language not language, whether English or C? The law is the law. But I have come to suspect that not everyone processes computer languages in the same way as human languages. I reserve the right to be grossly annoyed when you decide an apostrophe is a random decoration to throw out in warning of an approaching “s” but, reality probably won’t crash due to that error, today.
@Jessica Boxer on Wednesday, February 20 2013 at 6:21 pm:
> You should check out ericsink.com…. I’m geeky enough to find it really interesting.
I just did – <sigh> Another fascinating rathole for my time </sigh>. Thanks (i think) :-)
Also, please email me directly about Penguicon 2013 (couple of items you should find interesting) – ” J D B (at) systemsartisans (dot) com “
@ Saurav Sengupta
Man, you really don’t know when to quit while you are ahead. JustSaying was going out of his way to be nice to you.
@BRM: Thankfully, you have given me a reason to quit this argument because to continue arguing with the bunch of egoists here is simply hopeless. So much for online etiquette, and good riddance too!
@Saurav Sengupta:
Please tell me where you live and work so that I may show up at times of my convenience to derail your life and hurl insults at you. Preferably with a megaphone.
@Garrett: Thanks, but I am not interested in inviting you or your clan (I get the sarcasm of your words, but still). You are most welcome to hurl insults at me here if you feel that you have not done so already.
Eric’s and his companions’ point was that since they use automatic tools, they have themselves become like those tools due to “adaptive response.” My point was that the phenomenon of people making mistakes is not that uncommon, except outside the hacker culture. I did also claim that the human brain is much more adept at sifting through errors than computers. Hackers, however, are of the opinion that anyone who makes mistakes is an idiot and has no business programming computers. It is true that people who program computers should have a logical mind and should strive to attain as much precision as possible. However, I also stand by my assertion that programmers too are people and can make mistakes, just as doctors or scientists can do. Generalising that to infer that everyone who ever makes a mistake is an idiot is not the way I think. Being a mere human, I can and do make mistakes, correct them, and can also successfully make programs that work. But it is being able to understand the meaning of comments, etc., even if they contain errors, that sets the average human brain apart from those of the self-proclaimed hackers.
@zeph: Since you seem to be in need of proof as “backup” of what I had claimed (I will not say “what I had demonstrated as my ‘attitude'” because “demonstrating attitude” proves itself), you can visit http://www.inf.ed.ac.uk/undergraduate/reading.html and http://en.wikipedia.org/wiki/Word_recognition, though I am not sure if any of the people here will be capable of reading the malformed sentences shown in the former website, given their automaton-like minds (as Eric has himself claimed in response to my comment in this thread).
@Saurav Sengupta on Wednesday, February 20 2013 at 4:57 am said:
@The Monster, @esr:
> I do not know whether great programmers’ code compiles without errors the first time every time,
You completely miss the point. Code can be syntactically valid and therefore compile without errors and still not correctly express the idea the author intends. That was what I was getting at by “nitpicking” about your apostrophe placement. Many people simply do not grasp that there is a difference between “people’s” and “peoples'”, or worse, have some dim understanding that there is a difference, but just don’t care, because they believe the people with whom they are communicating will suss out what they really meant to say.
> I say “small errors” because the average human brain, which you seem to have fun in deriding, is infinitely more capable than a computer at sifting through text to decipher the meaning that the author intended to convey.
I’m not deriding human brains at all. I’m challenging the social convention that caring about spelling/punctuation/grammar is “nitpicking” and rude. I know people will make mistakes, and I make them myself. But the attitude I’m fighting is that they somehow aren’t even mistakes at all, because you can figure out the intent most of the time. I find that attitude particularly arrogant and inconsiderate of the many readers who will all have to waste their time deciphering, whereas if the author simply took a few moments to proofread, it would be unnecessary to do so.
>I believe that it is better to be a good human before or instead of being a great artist, which is why I stand by the “average” person.
Why is it that you think you’re a better human than I am because I “nitpick” about grammar, spelling, and puncutation, and you don’t?
@JustSaying
> If your style is not random
Everyone’s style is somewhat random, for example, how lines that are too long for one line are often formatted in a fairly random way. Evidenced, btw, back to OP, is that people when given free form do funky things with *.
But you are mostly right, I was just being pedantic.
> And I have no qualms with Jessica’s point that some people may find they mismatched braces and/or indenting often enough that the compiler could the option to check for any style of consistently matched indenting and braces.
Doesn’t have to require a very specific style, it can match multiple different styles and consider all acceptable. I’d give some examples, but I have no doubt that I can’t type something into WP to make it format the way I want.
Jessica: I completely agree that whitespace is meaningful to human beings as an aid to interpretation; I meant to disagree only with the idea that it should be meaningful syntactically as part of the parsing process.
(Which makes my comparison to printing-and-design contexts make more sense, I hope.)
@ The Monster, @Saurav Sengupta
Since we seem to have come full circle, here, I would just like to make the point that when I said:
I was referring to how you might interpret The Monster’s comment; I agree with The Monster and I don’t think that his comment was rude.
$DEITY knows that I make more than my share of errors posting to this blog, but at least I feel bad about it ;)
And unfortunately while better version control systems have an option to ignore changes in whitespace when displaying differences (diff), AFAIK there is no option of forcing to ignore whitespace when merging (or reformat automagically in background before merging).
Though I guess with Git’s pluggable merge strategies, merge drivers (file contents, per filename via attributes) and possibility to pass options to merge strategies (c.f. ‘subtree=’ option) it should be possible… {reads git-merge(1) manpage}… actually `
@Saurav
> excessive pride in oneself in any situation is considered ill-mannered…
> [hackers] … are of the opinion that anyone who makes mistakes is an idiot
The best response to my following reply is none at all. The next comment from you should ideally say something that is interesting to the participants.
What you incorrectly mistake for pride, is the group’s filter that tunes out the karaoke noise interfering with the goal of this blog to explore information and logic. A person who is so self-important that they pollute the blog page with off topic discussions about manners, is boring and obfuscates the interesting meat of the discussion.
Apparently no one was being rude, but when you insisted on turning this blog into your personal crusade against ill manners, Eric pointed out bluntly that you are apparently incapable of realizing following simple concept on your own.
Imagine you walked into a model airplane builders workshop and someone pointed out that you forgot your supplies, and then you disrupted the conference with megaphone shouting about how the participants are not tolerant of mistakes.
I don’t think most participants here are gloating about this, rather feeling embarrassed for you and annoyed that you can’t recognize that you are standing up in public with a the ill manners of a megaphone.
P.S. I did the same thing you are doing, but I adjusted. So you can adjust to, if you take breather and think about it.
@Sigivald
> Jessica: I completely agree that whitespace is meaningful to human beings as an aid to interpretation; I meant to disagree only with the idea that it should be meaningful syntactically as part of the parsing process.
I don’t think we are far apart really, but I think the point I am making is that if the information is relevant to human understanding it is a very useful piece of data to capture to determine if that human understanding is out of sync with the compiler’s understanding. Hence checking the whitespace against the actual parse tree is a way to find errors for the programmer, and that is a good thing.
There are lots of other things that are syntactically valid that look right to the programmer, but are probably wrong: assignment in a conditional for example, or, to get away for C, how about this piece of javascript:
if(items.Length == 0) $(“#errorFlag”).attr(“color”, “red”)
Looks good, right, but the grossness of Javascript takes the non existent Length property and returns null (of course it is called length, not Length.) Can someone define such a property? Yup, but it would be crazy to do so.[*]
Which is to say, it is a good thing for a compiler to analyze a program to a much stricter standard than the language allows, and where there are gaps, produce warnings, and, ideally, automated corrections.
This is especially so where there are classes of error that are extremely common, and block errors are very common, however, I am not aware of a single compiler that will warn for this sort of thing, even though it would be very useful and very easy to do.
[*] Footnote — I say it would be crazy to do so, but this is Javascript, Its “flexibility” has lead to some of the wackiest, brain numbing insanity I have ever seen in programming. For example, the jQuery library is a function called “$”? Obfuscated C contest eat your heart out.
@Jessica Boxer
> You should check out ericsink.com
OFF TOPIC: I am interested to contribute to a discussion about “Mobile Apps: HTML5 vs Native”, which is on that home page now, but that blog does not allow comments.
I think it is ridiculous that I can only code HTML and JavaScript in the browser (from a seamless webpage excluding plugins), and that the only GUI API I have is the DOM. And I don’t want Ian “Hixie” Hickson in charge of the APIs I have access to. Security can be controlled from the APIs. Note, I stopped following HTML5, so my contention may be outdated.
Jessica, I suspect you and I will never agree on the relative intelligence of the compiler vs. the programmer. All I can say is that you will never optimize the programmer out of the way, as you seem intent on doing.
Doug Gwyn was right: “Unix was not designed to stop its users from doing stupid things, as that would also stop them from doing clever things.” The same goes for compilers. The more you try to dumb the compiler’s expected input down, the harder you make it to do the really tough jobs.
@The Monster:
“Many people simply do not grasp that there is a difference between “people’s” and “peoples’”, or worse, have some dim understanding that there is a difference, but just don’t care, because they believe the people with whom they are communicating will suss out what they really meant to say… I know people will make mistakes, and I make them myself. But the attitude I’m fighting is that they somehow aren’t even mistakes at all, because you can figure out the intent most of the time. I find that attitude particularly arrogant and inconsiderate of the many readers who will all have to waste their time deciphering, whereas if the author simply took a few moments to proofread, it would be unnecessary to do so.”
Personally, I had already admitted to you that the apostrophe placement was a genuine mistake. Many more people do not even care that there is a difference between, e.g., “its” and “it’s”, etc. I had simply said that it is not that difficult for humans to understand the intended meaning. I did not say that it is actually OK for people to be careless about their writing/coding.
“Why is it that you think you’re a better human than I am because I “nitpick” about grammar, spelling, and puncutation, and you don’t?”
I am not a good human. (Stating the reasons here would become even more off-topic). Besides, I “nitpick” about precision myself and have grown up with it because most members of my family hold post-graduate degrees in English and science. In fact, I may be as much tuned to logical thinking, precision and rationality as you or Eric, especially given that I find it hard to bear even behavioural stupidity. What I meant was that I do not like to bluntly call people idiots or whatever for making a mistake. Euphemisms may not always be the best way to get a point across, but I have both seen sloppy programming and taught best practices to those same people who came up with the sloppy programming in the first place, so I have come to know how people react if you simply admonish them. I know what Eric is saying, but if I, personally, said it that way to whoever comes to me for help, that person would never discuss anything with me a second time (I have experienced this first-hand).
@JustSaying:
“What you incorrectly mistake for pride, is the group’s filter that tunes out the karaoke noise interfering with the goal of this blog to explore information and logic. A person who is so self-important that they pollute the blog page with off topic discussions about manners, is boring and obfuscates the interesting meat of the discussion.”
As you have pointed out, all discussion pertaining to my posts on this thread has been off-topic. I had not realised that Eric runs this blog to discuss the art of precision (only), so I would not like to comment any more on the point I was trying to get across.
“Apparently no one was being rude, but when you insisted on turning this blog into your personal crusade against ill manners, Eric pointed out bluntly that you are apparently incapable of realizing following simple concept on your own.”
There seems to be a difference of opinion in the definition of rudeness here but I was not trying to turn this blog into my personal crusade against ill manners. I think that I am capable of understanding the point Eric originally made. I have seen countless programmers do what Eric is rallying against, but what I said in response apparently never got across and the whole thing became a mountain out of a molehill.
> that person would never discuss anything with me a second time
Perfect.
> The next comment from you should ideally say something that is interesting to the participants
@JustSaying: Sorry, didn’t get the meaning of your last comment.
@JustSaying: The one in which you said:-
> that person would never discuss anything with me a second time
Perfect.
> The next comment from you should ideally say something that is interesting to the participants
He was politely telling you that your rants on manners are getting tiresome and that, if you have nothing else to contribute to the actual topic of discussion, you should stop typing.
I quit reading Saurav’s rants after the first one, except to note that at one point he was leaving…
@Saurav
>if I, personally, said it that way to whoever comes to me for help, that person would never discuss anything with me a second time (I have experienced this first-hand).
This is a feature, not a bug. If people are so thin-skinned as to be offended by the language I used with you, and choose not to seek help from me in the future, they are saving me the trouble of telling them it’s a waste of my time to help someone who refuses to be trained.
@Jay Maynard
> All I can say is that you will never optimize the programmer out of the way, as you seem intent on doing.
I agree with that, and I am certainly not intent on doing that. On the contrary, I want to eliminate the trivial mechanical work so programmers can concentrate on the difficult work. It is not different than the whole concept of using a high level language. Writing machine code is difficult primarily because you have to handle lots of teeny details. For example, how to write a loop that iterates over a collection or how to optimally manage a limited set of registers. These concepts are abstracted and made easier in a high level language.
Doing so allows the programmer to think about what she wants to achieve, rather than the mechanism for achieving it. Yon can certainly do things in machine code that you cannot do in C, but that is no reason to write your gpsd program entirely in machine code.
So ways to make languages higher level, to validate that programmers are not making mistakes are a good thing because they free up the programmer to think about the things they are good at thinking about, and have to think less about the things they are not good at thinking about.
FWIW, “clever” is not an adjective I would want applied to my code, I’d think of it as a criticism. “Plain”, “obvious”, “simple” and “clear” are what I am going for.
According to the definition of “rant” (http://oxforddictionaries.com/definition/english/rant?q=rant), @BPSouther, @Jay Maynard, the original article here by Eric fulfils that criterion.
@BPSouther: “He was politely telling you…” Politely? There are other such examples here, but you people really do seem to be suffering from some sort of psychosis or a misplaced notion of sarcasm or both (pity).
@JustSaying: “A person who is so self-important that they pollute the blog page with off topic discussions about manners, is boring and obfuscates the interesting meat of the discussion.” If this poor blog has been “polluted” by the “rants” of someone as “self-important” as me, Eric can, at any time, delete all my comments and all replies related to them. However, I will keep a cache of this page (thread) to show normal, healthy people the true face of the cult of hackers.
Anyway, BPSouther is right. Although any normal person would pity the mental condition you are suffering from, this long-drawn argument is really getting tiresome. It is pointless to argue with idiots because they are incapable of understanding anything in the first place, so ideally I should stop wasting my time trying to reason with a bunch of vain snobs like you and get back to real work. Thanks for all the replies.
>ideally I should stop wasting my time trying to reason with a bunch of vain snobs like you and get back to real work.
What, you’re not gone already? Hey, don’t let the door hit you in the ass on the way out.
> So (perhaps) you can adjust to(o), if you take breather and think about it.
Correcting my typos above and admitting that I was too optimistic.
@Jessica
> eliminate the trivial…validate…
> what she wants to achieve, rather than the mechanism for achieving…
> they free up the programmer to think about the things they are good at thinking about
Seems to roughly correspond to my explanation of the declarative property of programming languages.
Eric can, at any time, delete all my comments and all replies related to them. However, I will keep a cache of this page (thread) to show normal, healthy people the true face of the cult of hackers.
Why such fearlessness in the face of crushing authority! Not to mention the selflessness of exposing this cult. I bet you would insist on the truth if Goebbels himself were your enemy!
Watch out for the Black Hele^H^H^H^H^H Quadcopters.
— Foo Quuxman
>Eric can, at any time, delete all my comments and all replies related to them.
Far from deleting them, I’m thinking of preserving them in aspic with a caption that reads “How to be a self-important asshat: a study in unintentional humor”.
> Hey, don’t let the door hit you in the ass on the way out.
Don’t worry, it didn’t. There were too many of your asses shitting in the doorway for the door to even be closed.
> Far from deleting them, I’m thinking of preserving them in aspic with a caption that reads “How to be a self-important asshat: a study in unintentional humor”.
You are most welcome, but you are late. My own opinion of you is already documented in my blog (website link on this post). You really ought to have a scientific (biological name), something in the subgenus of asinus would do well, I think.
Oh sorry, I made a stupid mistake. I should have written, “scientific (biological) name” because poor you won’t be able to understand it otherwise.
Somebody in this thread was asking about “best practices” guide for working with version control.
Git sources contains SubmittingPatches document, where some of steps are not git as a project specific, nor git as a version control system specific (details in said document):
(0) Decide what to base your work on.
(1) Make separate commits for logically separate changes.
(2) Describe your changes well.
The rest of steps are more specific:
(3) Generate your patch using Git tools out of your commits.
(4) Sending your patches.
(5) Sign your work
HTH
I’m fascinated to see how deep this hole goes. I figured it was over a couple of pages ago, but I’m clearly lacking in imagination.
@Christopher Smith
If you want to see a *really* deep hole go check out Saurav’s blog about the events here; wow, just wow.
— Foo Quuxman
>If you want to see a *really* deep hole go check out Saurav’s blog about the events here; wow, just wow.
Amazing. Reading things like that puzzles me profoundly. What I don’t get is how it’s possible to at one and the same time so verbally fluent and superficially intelligent but at the same time so…hm, “stupid” doesn’t completely cover it, nor does “self-centered”. It’s like he doesn’t really get that there is a reality prior to his ego and his elaborate verbal constructions.
And he’s so blind to how he presents to others that he thinks that blog entry will accomplish something useful for him. As opposed to what it actually does, which is yell “I am a passive-aggressive grievance-peddling wussy hiding behind pseudo-scientific language” loudly enough to be heard from here to Sheboygan.
Ah, well. He’s not our problem, fortunately. Spare a little sympathy for anybody for has to interact with him day-to-day, though; the narcissism must get hard to take.
@Foo
How about no. I’ve seen my share of bizarre persecution complexes, TYVM. I just wasn’t expecting him to hang around, although since he apparently has a martyr complex to accompany it, perhaps I shouldn’t be surprised.
You are aware that this is a pretty good summation of a wide swath of academia?
>You are aware that this is a pretty good summation of a wide swath of academia?
Yes, I know. It’s also why being described as an “intellectual” sometimes makes me twitchy.
I like Thomas Sowell’s definition of “an intellectual” as someone whose work product is ideas. Some people (e.g., professors) are professional intellectuals, and some of those (e.g., professors of $VICTIM studies) produce nothing but fertilizer, but I don’t think there’s an inherent problem with being accurately described as someone who’s produced (meaningful and useful) ideas such as theories of software economics. (I do understand that the class “intellectual” gets tainted by its loudest members’ tendency to produce Frankfurt bullshit, but I think that’s a case of the 80% making the 20% look bad.)
@JustSaying
> Seems to roughly correspond to my explanation of the declarative property of programming languages.
Perhaps, but let me remind you that the end doesn’t always justify the means.
@Jessica
Are you interested to unbox that thought more? The declarative property is somewhat fuzzy and difficult to generalize in words when explaining the possible variants of the means. Perhaps I have not yet distilled the essence.
@Foo
That is funny and terse. Since Eric wishes his comments to be a case study, I will elaborate.
I suspect his paranoia is nearly an unconscious defense mechanism, i.e. insanity. He apparently couldn’t rationalize the initial discussion that upset him, so his emotions ran with finding an explanation that would classify us as abnormal. (I did this in the past but not on manners rather on the “right” to speak unintelligibly, yuk I cringe and shudder) Emotions are very dangerous, when they are not fact checked with rational thought. His reaction to my comment will be that we are emotionless automatons, incapable of social harmony. He felt if we would just participate in his self-important need for harmony (i.e. cater to his out-of-control emotions), then we would not be a threat (to him, and in his mind to social harmony in general) and he (and society) could be at peace. I suspect he is rarely at peace, because his requirements for peace are irrational. When we did not relent to his emotional insanity, it confirmed to him that we are as bad as his fantasy wants us to be.
Saurav could not rationalize that this blog reaches a large group of mostly very busy people who want to only read comments which are interesting to our vocation. He may be correct about his society. My assumption is much of humanity expects this emotional hand holding, i.e. irrational “harmony” in the workplace. It is why I could never be a manager. I think we are headed to world where most people work unsupervised online. The misfits are going to fall away economically. I will be observing their dysfunctional statism attempt to sustain irrationality by attempting to steal from the collective (of themselves). The next few decades may be a game “avoid the zombies”.
In hindsight, Monster wasn’t being pedantic, rather very astute. Saurav’s sloppy writing style indicated he was perhaps not based in logic and rationality, which are required for the software age. I will add this early warning system to my repertoire. I am so willing to give people second and third chances, because I never wanted to believe I was different than most people. And because as a leader in sports, I was able to inspire non-technical people to greater performance. I was able to explain physics in a conceptual manner to someone in college who said I was instrumental to him becoming an adjunct professor at Georgetown U. and a manager at SAIC. But alas, I must also learn to be more efficient at not wasting my time. The recent blog on why hackers shouldn’t hand hole disciples is insightful.
@JustSaying
> Are you interested to unbox that thought more?
Not really, it was meant as a kind of funny snipe. I like purely functional coding, though I haven’t done it for a while, or done more than trivial examples. If my point was anything, it is that saying that a declarative language describes the results only is all very well so long as you accept a pretty broad definition of “results.” Temporal process is often an important part of “results”, and the basic model of “these inputs produce these outputs” is a little fuzzy when you are writing an interactive piece of software.
Nonetheless, surely purely functional programming and to a lesser extent softer declarative styles are particularly relevant in the new multicore world. The tractability they have offers some really interesting possibilities.
And, on a very soft point, in the language I use most, C#, there is a soft functional component called LINQ, something that I find extremely expressive, but, unfortunately, lacking the clarity, simplicity and obviousness that I advocated earlier in this thread.
So there you go, I guess I did want to unbox a little. :-)
@Jessica Boxer
As far as I know, the fantastic hope embodied in the declarative property are largely unproven in the real world of general purpose programming. HTML achieves the tight coupling yet without Turing-completeness. Once JavaScript is added, the declarative coupling is lost. I wrote an insight recently on inverting the control for events with pure functions, thus sustaining my hope for a tight general purpose programming paradigm.
My limited experience with Python is that it has constructions which enable the programmer to express semantics more cleanly. In my opinion, this is fulfilling the declarative property.
> hackers shouldn’t hand hole disciples
I’m confused,
do you mean hand-hole – http://www.thefreedictionary.com/hand-hole – in which case , as you use it as a verb, I’d read that as ” … insert their hands in holes of disciples” ;
or did you mean hand hold ?
Sweet mother of Jesus…I was going to post a daft bit of black humor about stars and short Austrians, but the oxygen seems to have been sucked out of this thread by an utter dolt.
Ah well….you want my asterisks? MOLON LABE
;)
>I was going to post a daft bit of black humor about stars and short Austrians, but the oxygen seems to have been sucked out of this thread by an utter dolt.
What a shame. I’d like to read that.
>Ah well….you want my asterisks? MOLON LABE
Yeah, and while I’m at it I’m coming for your high-cap keyboard, too. If it saves just one lolcat…
Typo, saw it earlier, laughed. class HandInHole inherits CoDependence
Send all the black roflcopters you want, you’ll never take me alive!
Give me emoticons or give me death!
I with you force, he pry ’em from my cold, dead iRectum display touch.
I had decided not to post anything more here but there’s one thing that I was wondering about.
What I fail to understand about you people is, if I’m enough of a troll, an idiot and an asshole, why is it that you are being unable to stop discussing me and my posts? Why are you still unable to actually _ignore_ me? Just to be clear, though, I’m saying this not so that you stop insulting me (doesn’t matter to me any more now after all your contributions and the pingback article), but to actually understand your life.
You continue to operate from false premises.
You seem to think you somehow affect us in some important way? So that our “compulsion” to respond to your posts shows some lack of mental well-being on our part?
Er, no, we just like to abuse assholes such as yourself.
Or is it possible that you think you have a special right to not be abused? That you are important enough to be free from criticism? Perhaps we are infringing on your ‘free speech’ rights by attempting to suppress you?
No, you’re not that important. There is no right to not be criticized, though some extremely self-important people act that way. And if the last is true, your understanding of property rights is even poorer than your understanding of free speech rights.
Actually it might be amusing to go to your home or office, and once inside start screaming obscenities. Then, when you complain, ask why you’re unable to ignore me. Your answer to *that* question would be enlightening.
>Er, no, we just like to abuse assholes such as yourself.
Actually, that’s not my motivation, though I can understand the urge to claim that it is. Rather, I find the process of someone publicly making as huge an ass of himself as this guy is doing perversely fascinating. Partly due to the element of pratfall comedy added by his repeated insistence that he’s gone, invariably followed by attempts to salvage some sort of face with replies that try to be biting and snarky but just sound like whining. It’s like watching a really nasty auto accident happen – you know you should look away, but you can’t.
if I’m enough of a troll, an idiot and an asshole,
Oh I don’t give you enough doubt benefits to call you a troll, from the statements in your posts here and elsewhere I do hereby call you a blithering idiot who wants everyone to be perfectly shaped corn syrup coated cogs in $MACHINE
why is it that you are being unable to stop discussing me and my posts? Why are you still unable to actually _ignore_ me?
You’re frickin hilarious. Also a classic example of someone pulling out the loner==freak card
Just to be clear, though, I’m saying this not so that you stop insulting me (doesn’t matter to me any more now after all your contributions and the pingback article),
Why would you care in the first place? I don’t give a shit if you insult me, hell, I wouldn’t care is ESR insulted me even though I fit pretty well in the fanboy slot.
but to actually understand your life.
If your previous statements have anything to do with your thought processes you will *never* understand any of us.
— Foo Quuxman
@Greg
> “Actually it might be amusing to go to your home or office, and once inside start screaming obscenities. Then, when you complain, ask why you’re unable to ignore me. Your answer to *that* question would be enlightening.”
My last post was some time back and yet, since then, although I had posted nothing, you have been unable to stop discussing me (there have been several posts about me since then), that’s why I asked. If I were still “screaming obscenities” at you, that would validate your point.
> “You seem to think you somehow affect us in some important way? So that our “compulsion” to respond to your posts shows some lack of mental well-being on our part?
Er, no, we just like to abuse assholes such as yourself.”
In my question, I had already stated that it was not to stop you from “abusing assholes” like me.
@Foo
> “Why would you care in the first place?”
I don’t, as I said.
> “If your previous statements have anything to do with your thought processes you will *never* understand any of us.”
No, my previous statements do not have anything to do with my day-to-day thought processes. As I said, I do not care about this one-off event (I was just curious about you people going on about me even after I had stopped posting comments here). My originally commenting in this particular case (this article) was a one-time mistake. My ordinary thought processes are much more “tolerable” for both me and for the people who have to “interact with me day-to-day” (quoting Eric). Details are here: http://www.thepowerbase.com/2013/02/blogger-claims-eric-s-raymond-has-severe-mental-problems/#comment-809774766 (in my reply), if you care to take a look.
Ah. In that situation I just keep quiet and eat my popcorn. :)
But there’s a certain type of online presence that I’ve seen quite a bit of over the years (first got Internet access in the late 80’s) that I can’t help but abuse. They’re almost a troll – the real trolls you don’t feed – but not quite. They take themselves too seriously, and have too much emotional investment in the garbage they spew.
A real troll will say anything calculated to get a reaction, he doesn’t care about the actual content of what he’s only, he’s only in it for the reactions he can cause and the blood pressure he can raise. The emotional investment in their statements (they need to be RIGHT, always) gives you a handle on them, if you want to grab it.
>The emotional investment in [the pseudo-troll’s] statements (they need to be RIGHT, always) gives you a handle on them, if you want to grab it.
This is a worthwhile distinction, if we can keep it, but we’ll have to work to keep it. Popular usage is not consistent about that boundary.
What would you call the second category to distinguish them from true trolls? “Ranter” is a fine old word that leaps to my mind.
And I agree, Saurav Sengupta is a ranter. He doesn’t have enough self-awareness to be a troll.
Apparently I am very prone to perceptual set when proofreading, after revising text.
It’s pretty common and it’s an issue, particularly when writing code.
<snark>
just for the LOLz…
today’s Sinfest comic
</snark>
I have to say, I don’t really think of it that way. To my way of thinking, a ranter is just someone who tends to be a little peevish who wants a little validation. He doesn’t necessarily argue, he just has a complaint.
I think of someone like SS as being on the ‘know-it-all’ spectrum (but not in a good way). Someone on that spectrum is the type of person that would enter someone else’s conversation to offer a correction, or additional facts. We’ve all done that here, to some degree.
But there are variables that qualify a ‘know-it-all’s behavior. Some of them are behavioral failure modes- they can be belligerent, overbearing, self-righteous, passive aggressive, self-pitying…. to various degrees at various times. (Those of us who aren’t narcissists try to avoid all those.) Too much of any of those and you’re more or less a would-be intellectual bully of some kind- which is what SS came here to do.
Not really rigorous, but it’s my own little system. (Yes there are other pathological types of k.i.a., like the braggart, the one-upper, the name-dropper, etc).
>But there are variables that qualify a ‘know-it-all’s behavior. Some of them are behavioral failure modes- they can be belligerent, overbearing, self-righteous, passive aggressive, self-pitying…. to various degrees at various times. (Those of us who aren’t narcissists try to avoid all those.) Too much of any of those and you’re more or less a would-be intellectual bully of some kind- which is what SS came here to do.
You make good points here, but you don’t solve our terminological problem – what label shall we use to distinguish these “intellectual bullies” from true trolls?
Actually, I think you’re pushing me towards my wife’s suggestion: “orc”. Despite the appeal of continuing the Tolkien analogy, I was going to reject it because I didn’t want to bundle in the implication that ranters are actually evil. But your implicit argument that SS is best modeled as a (failed) variety of intellectual bully is persuasive and somewhat reframes that distinction for me; attributing evil to intellectual bullies does not seem inappropriate.
Maybe what we need here is a tripartite distinction: troll = manipulative shit-stirrer in it for the lulz, orc = intellectual bully, ranter = mainly just peevish or broken.
Bearing in mind the reason for having such terminological distinctions – so we can associate different countering behaviors with them – what do you think? Other regulars, please jump in on this.
> I used to view green-on-black, eschewing the more modern white
>background. Syntax coloring was intolerable until I switched to a white background.
For tailing logs I use a green on a very dark slightly transparent window. This allows me to have several overlapping windows and be able to scan them all at a glance.
For coding, I find a manila or beige background (#FFFBBB) is the easiest color on the eyes that doesn’t cause any of the highlighted text to disappear. I usually don’t notice how much a pure white background bothers my eyes until I switch to the beige background. It’s like having a splinter pulled out.
> I also find that what you can tolerate visually may be age-related.
++1 …and more so for sound/noise.
> “Ranter” is a fine old word that leaps to my mind.
“Ranter” covers the blowhard end of the spectrum but doesn’t sufficiently convey the passive/aggressive, martyrdom aspects. I’m not sure there is a word that does.
> Actually, I think you’re pushing me towards my wife’s suggestion: “orc”.
You couldn’t hurt an orc’s feelings.
I think SS is genuinely insulted and hurt. Like JustSayin, I can see a little of my younger self in him, which makes this a little painful to watch. I’m sure he’ll want nothing better than to see this whole thread evaporate once he matures a bit and gets over himself.
I was going to reject [orc] because I didn’t want to bundle in the implication that ranters are actually evil.
Perhaps goblin? It has somewhat goofier connotations and the “inherent evil” part is much less pronounced
— Foo Quuxman
>Perhaps goblin? It has somewhat goofier connotations and the “inherent evil” part is much less pronounced
Mmm…maybe. I’d prefer not to risk collision with Jeff Cooper’s usage of “goblin” – though it’s in a somewhat different domain.
Wikipedia correctly notes “Jeff Cooper, creator of the ‘Modern Technique’ of firearm handling and self-defense, commonly referred to adversaries as ‘goblins’ in his commentaries.”. But the article misses Cooper’s implication – that the goblin is genuinely evil, having severed himself from the trust community of humankind, and should therefore not be thought of as human but rather shot in the same spirit as one would put down a rabid animal.
The basic MO of ‘going someplace, shouting criticism or correction, establishing local intellectual superiority browbeating the indigs as needed, then leaving’ is just a hit and run from a flamer, that classic Internet staple. With the narcissistic drama queen bits added… well I’m very sensitive to that type but never gave it a label, because I always just internally associate it with the person who defined the archetype for me- call it a ‘Barb’.
Oh and meant to add that I like the tripartite distinction. In my mind, there is definitely a type of person who is really just looking for someone to say “damn that sucks, I can’t believe you have to put up with that” and then they’re happy as a clam. ‘Ranter’ is as good a label for that type as any.
Narx? (short for narcissistic and the double-think of social harmony in Marxism)
Narcs is already used for drug law enforcers.
@SS
We demonstrated we have emotions and are not automatons. After a shocking event such as auto accident, there is an emotional gasp (refer to the prior humor about oxygen sucked out of the thread) among the onlookers. Since our injuries were only time lost, the hackers tossed humor to validate sanity analogous to checking the body for shrapnel and giving the thumbs up. Discussion proceeded to analyze the taxonomy of dysfunctional personalities in group discussion, to enable more efficient future labeling, thus maximizing the value of the time already invested. The point is that hackers navigate the world rationally, not because they don’t have emotions, rather they solidify emotions in a factual foundation.
Luke, mankind must repel the automatons until our last kiss :-*
SS, english grammar is ambiguous and requires information external to the language text for resolving meaning. Most computer languages are designed to be deterministic context-free, which usually means they are unambiguous. The dangling else is an example of a grammatical ambiguity and why we don’t want it in computer languages.
You were just wasting our time with your emotional reaction, because you did not grasp the unarguable technical fact.
From the disciples blog:
Greg noted the overconfidence effect, which is a form of ignorance.
Ignarx?
I’m not too happy with using this word in that context, because a “ranter” is clearly “one who rants”. There’s nothing wrong with a good rant; properly executed it can be a work of art.
Of course, art is rather subjective. I thought that pointing out the ramifications of a misplaced apostrophe was an artistic way of showing how sloppy spelling/punctuation/grammar can lead to problems with a system that actually follows the rules of the language in question (rather than relying upon the human brain’s ability to guess what the real intent was). Whenever the reaction to correction is “Well you know what I mean!” I respond with “No, I don’t know for sure. I may think a particular interpretation is far more likely, but precisely because you’ve expressed yourself this way, you’ve left the possibility open that you really do mean what you’ve actually said.”
And if it’s tough for humans to figure these things out, it’s exponentially tougher for us to write code to handle the exceptions to the exceptions to the rules.
This is, quite simply, the best argument for Python’s syntactic use of whitespace ever. (And those who have looked at C code and gone bonkers trying to get it to look right in their editor should immediately recognize why.)
How about ‘mule’ (as in stubborn)?
Unlike a troll or even a person who is conscious of their ignorance, the most general trait is that SS was unaware that he was ignorant of the facts, which enabled his other reactions. He was genuinely convinced of our nitpicking and thus rudeness, and his motivation wasn’t narcissism.
The synonyms for ignorance don’t capture the self-unawareness.
How about an ‘oblivion’?
@JustSaying:
Except that they really do by connotation. You can say somebody “doesn’t know all the facts” or they are “uninformed” or even they are “ignorant of the facts,” but when you say they are plain “ignorant,” that (at least around here) usually implies Dunning-Kruger in action.
Many people (including me, on many subjects on many occasions) are happily oblivious. No need to conflate them with the subject under discussion.
> Except that they really do by connotation
Disagree.
Ignorance != Dunning-Kruger.
Ignorance + self-unaware == Dunning-Kruger.
> Many people (including me, on many subjects on many occasions) are happily oblivious
Happy because you are aware that you are ignorant of the subject matter, thus you don’t go off on obliviously ignorant rants.
Oblivion only indirectly implies self-unawareness because it refers to the surroundings. Measurement of self requires surroundings.
How about a ‘Krueger’? (Freddie Krueger horror character)
ignorance of subject != Dunning-Kruger
ignorance that self is ignorant of subject == Dunning-Kruger
I agree that unqualified ignorance captures both cases, but I would prefer not to conflate the two cases.
How about a ‘Krueger’?
Kruegerite?
— Foo Quuxman
First, a couple of references that I haven’t been able to find: one was from a discussion of Turing tests– a claim that anger was the easiest (most mechanical) emotion to simulate. The other one is an old quote to the effect that if you have a great many slaves, you will become somewhat like them in order to be able to communicate with them.
Getting back to an earlier phase– Saurav, how do you manage to program without being precise about spelling and syntax and such?
For that matter, your text generally has good spelling and grammar. I’m not going to guarantee that you didn’t make any errors, but nothing jumped out at me, and I don’t think people in this thread would have been able to resist mentioning it if your writing was sloppy. Do you think this makes you more like an automaton?
Accurate spelling and the like are mechanical skills, but they don’t mean people who have them or value them are automatons.
People in general are affected by insults. You can hope that your insults will draw attention to the issue you’re angry about, but it’s a very unreliable strategy because the insults might get more attention than the point you’re trying to make. At that point, you can say that people shouldn’t notice insults, which works about as well as any other should-ought-to-trump-is argument.
I believe that people are affected by insults because their reputation has a large effect on their quality of life. More on the subject
On usenet, I would occasionally manage to improve the manners of someone who was habitually insulting. All it took was only replying to content. It was an ungodly amount of work, because I had to keep passing the Turing test (original thought rather than reflex) while pretending it was easy.
“Habitually insulting” isn’t the same thing as ranting or being a troll. I’ve never been able to do anything useful with trolls. They are admittedly more self-aware than people who are insulting but not trolls, but they’re addicted to cruelty and they know it. This is not an improvement.
I don’t have a suggestion for terminology, but “orc” doesn’t do it for me. Tolkien’s orcs weren’t especially bright, and most of their aggression was physical. Also, they were part of a large organization.
Might find some inspiration here…
http://redwing.hutman.net/~mreed/
Might find some inspiration here…
The “Innocent Abused” looks promising
— Foo Quuxman
After knowingly wasting a small portion of my life skimming this thread to see what all the fuss is about…SS just seems sorta, y’know….like a
http://redwing.hutman.net/~mreed/warriorshtm/lamer.htm
I am posting this despite Eric Raymond’s displeasure (http://esr.ibiblio.org/?p=4824&cpage=1#comment-395739) and because, even after my last post here, this thread has mostly been running off-topic (http://sauravsengupta83.blogspot.com/2013/02/hacker-mentality-or-internet-sub-culture.html).
@Nancy Lebovitz:-
> “Saurav, how do you manage to program without being precise about spelling and syntax and such?”
From http://esr.ibiblio.org/?p=4824&cpage=1#comment-395675, “I had simply said that it is not that difficult for humans to understand the intended meaning. I did not say that it is actually OK for people to be careless about their writing/coding.”
I do not manage to program without being precise about spelling and syntax. See my reply to this comment: http://www.thepowerbase.com/2013/02/blogger-claims-eric-s-raymond-has-severe-mental-problems/#comment-810726635.
Now, take the points I originally made:-
1) Make a language more suitable than C that does not use the asterisk in its syntax. Write or rewrite a popular operating system in it, so that programmers are finally forced to use it.
2) You are nitpicking over “grammatical” errors, albeit they are in an artificial language. Such errors are all too common in peoples’ [sic] use of natural language as well. Open source software developers who contribute to major projects tend to be perfectionists (I have tried to be a perfectionist myself). Other people, who constitute the majority, do not care as much. [There is a misplaced apostrophe here which was immediately pointed out.]
3) Make a better way to mark bullet points in programming languages, given the age-old obsession with pure text syntax in such languages.
Eric’s article was about an opinion against misusing the asterisk to make awkwardly/wrongly punctuated comments. Here is a break-down (explanation) of my points:-
1) The asterisk is used most by C and C++ programmers, who have become habituated to it, and most operating systems have been coded in C. Had there been better, more integrated commenting systems incorporated into (early) programming languages, we could have had a better state of affairs.
2) Although automatic tools can and do cause problems with syntax, reading a comment, even if it is not written properly, should not cause the same amount of trouble to a person, because of the human brain’s capabilities to decipher the meaning of statements with errors (also, comments are only for people to read, not for most tools; if an interpreter can ignore/modify them, so can other tools). Again, however, “it is not actually OK for people to be careless about their writing/coding.” I used the word, “nitpicking” because this type of erroneous commenting, although quite common and quite irritating, is still not as irritating as or, worse yet, nightmarish to debug, as erroneous code interpreted by the compiler, which too, I have seen even “experienced” programmers write.
3) Again, I think we can do better in language design to integrate comments nicely into the code with the help of syntax (it will require experimentation and research, of course, and as part of my work in creating my own programming language, I am doing that as well).
> “Accurate spelling and the like are mechanical skills, but they don’t mean people who have them or value them are automatons.”
You can have and value them, and it is a good thing too. For example, in answer to your question as to whether writing English well makes me an automaton, many people actually have likened me to an automaton because of my tendency to be so precise as to reach as close to perfection as I can. However, having gone through a lot of life and having experienced a lot of the ways in which people interact (outside the hacker culture, that is), I believe that it is not a good idea to prioritise accuracy and precision at the expense of the emotions involved in communication. I had chosen the word, “automaton” mainly because of point 2, as explained above, and Eric said that since they work with automatic tools, their thought processes have become tuned to work like that because of an “adaptive response.”
> “I believe that people are affected by insults because their reputation has a large effect on their quality of life.”
It seems that you are referring to my insulting Eric and others here because they are the people who have good reputation in their community, whereas I am a nobody. This quote of yours also seems to hint that this thread is currently mostly concerned with coming up with a properly insulting term for people like me in an attempt to protect perceived effects on reputation, given that my own article was highlighted on the Internet by The Powerbase website (http://www.thepowerbase.com/2013/02/blogger-claims-eric-s-raymond-has-severe-mental-problems/). If, however, it is the other way round, it does not matter to me, as I have already said here and elsewhere.
> ““Habitually insulting” isn’t the same thing as ranting or being a troll.”
Maybe, but at least I am not in the habit of insulting people, ranting, or trolling (more details in my reply to http://www.thepowerbase.com/2013/02/blogger-claims-eric-s-raymond-has-severe-mental-problems/#comment-809774766 and to quote from it, “The tone of my blog post resulted as a natural reaction to the “battle of words” with Eric and his team and in hindsight I can see that I could have done better, but if you read it correctly, it does not say that Eric and his fellow hackers are doomed to become criminals or schizophrenic, nor does it contain pseudo-scientific terminology as Eric has claimed. It merely draws parallels.”).
> “Tolkien’s orcs weren’t especially bright, and most of their aggression was physical. Also, they were part of a large organization.”
I am not especially bright either (certainly not as much as the hackers) and neither does my physique allow me to be physically aggressive (not that I am interested in being so anyway!). Lastly, I am not part of any organisation, and especially, although I have been a FOSS enthusiast since 2006, I have never been a part of the hacker culture, which, as I have said elsewhere, led to my primary mistake of posting a comment on this blog seemingly in disagreement with this article, when I should have known that since hacker culture is different, such a comment had the potential to be misinterpreted.
My usual response to “You know what I mean!” is along the lines of:
You put care and attention into programming computers. When you talk (or write) you are essentially programming people. Don’t they deserve at least as much care and attention?
Krugmanite?
Almost had me thinking he was just an oblivious prig until here. Definitely still a paranoid martyr.
Christopher Smith: “Definitely still a paranoid martyr.”
Can’t believe I wasted so much time here. The idiot already doesn’t know what paranoia is. And if anything was affected by your misplaced notion of martyrdom, it was my real life, which I unfortunately neglected trying to reason with such a large lot of pure idiots like you. You might just finally go over the edge if you come across a real troll. Anyway, carrying on your nonsensical, perverted humour and your idiotic competition for coming up with taxonomy is your own pitiable choice. Good luck with it. (Eric Steven Raymond will probably again come up with some foolish statement about my declaring that I am gone for good, and then think of it as some sort of witticism he has conjured up.). What a lesson learned!
Honestly, I can’t believe you guys are still ragging on Saurav. FWIW, I don’t agree with the basic premise that sloppiness is an indicator of bad programming. It can be, but I am sloppy at lots of things and still get the job done. For example, I am horrible at long division, it requires WAY to much attention to detail for my tiny mind. Nonetheless, I never do long division because I have tools to do it for me. Computers are really good at all that carry this bit, borrow this bit, subtract, move the point stuff. Me? I suck at it. But I am way better than a computer at know what GUI would work for a human. I believe the mot juste is “symbiosis.” I believe some Scottish guy called Adam Smith blabbed on about the benefits of specialization, which seems to fit rather nicely here.
Better to focus on the big picture and let the nitpicky tools find the micro mechanical problems. Better to let your compiler find minor typos than spend too much time straining on it, and consequently loose sight of the forest for the trees. Humans have a limited mental capacity. Why fill it up with semi colon placement, and “four spaces not a tab” minutiae?
Of course it is better to be non sloppy than sloppy, but is it not a death blow, and there are great tools to fix it. Of course if you turn off all the warnings from your compiler, or don’t build unit tests, or don’t put an assertion framework in your code, or ignore the stuff your compiler spews out to you at the end of a build, then you are not using the tools fully and won’t get the benefits.
Again, it is why I don’t like duck typing and non typed languages. You are removing redundant information from the compiler and so it is less likely to be able to find errors.
I’d grant you that Saurav seems a little bit of a whiny butthead, but really, who isn’t when a host of smart people start ragging on them? He was just trying to get into the tree house, and made a mistake.
>if you read it correctly, it does not say that Eric and his fellow hackers are doomed to become criminals or schizophrenic
It doesn’t say that literally.
But then exactly what message were you trying to get across with rhetorical questions like these
immediately followed by this conclusion
@Just Saying:
>Unlike a troll or even a person who is conscious of their ignorance, the most general trait is that SS was unaware that he was ignorant of the facts, which enabled his other reactions. He was genuinely convinced of our nitpicking and thus rudeness, and his motivation wasn’t narcissism.
>The synonyms for ignorance don’t capture the self-unawareness.
>How about an ‘oblivion’?
How about “homo sapiens”? In my experience, this is a fairly widespread human flaw.
This likewise applies to Saurav’s accusing esr of being unhuman: The behaviors he’s criticizing do more to prove that esr is human than that he isn’t. To be human is to be an insufferable ass.
@Jessica
>I don’t agree with the basic premise that sloppiness is an indicator of bad programming.
It’s not the sloppiness itself. We all slip up and make mistakes from time to time. It’s the attitude that considers concern with “grammatical” errors to be “nitpicking”. That is then followed by the “oh, you know what I mean” excuse, based on the idea that “it’s just a comment” (and therefore for humans to read, not for computers to read).
In this particular case, ESR is actually talking explicitly about human readability of the comments, but I looked past that to the eventual goal of machine parsing of comments to extract summary information, and I assume anyone else who understands the Tao of Unix would be thinking along those lines.
@The Monster
> It’s the attitude that considers concern with “grammatical” errors to be “nitpicking”.
But I think he is right. I am loathe to dig through the comment thread, but my memory is that his error was a misplaced apostrophe. I don’t think that such an error prevented any honest person from understanding his point, though I suppose a pedant could parse it differently. It is almost an aphorism in Internet commentary that a thread has gone sour when the discussion is over the form rather than the substance of the comment. That seems to be where that ended up. So much so that I have no idea what the substance of his comment was.
> ESR is actually talking explicitly about human readability of the comments, but I looked past that to the eventual goal of machine parsing of comments
Right and that is ultimately the problem here: the Tao of Unix. Namely the whole principle of promiscuous inputs, strict outputs. The problem here is that the check in comment that was a problem violated an unwritten rule of formatting, put in place to allow effective machine parsing. Which is to say it is a community of assumptions rather than a parsable, piece of data.
It would not be a problem at all if the comment system demanded input in a standard markup language, like markdown, then the bullets could readily be parsed. But Eric’s engine assumes certain things about the text, and the commenter didn’t make the same assumptions. The dissonance is over a different set of expectations and assumptions, not foolishness on either part.
And this is indeed the Tao of Unix — many unwritten rules about text formatting, and the associated claim that the text is free form, which it plainly isn’t.
>The problem here is that the check in comment that was a problem violated an unwritten rule of formatting, put in place to allow effective machine parsing. Which is to say it is a community of assumptions rather than a parsable, piece of data.
This is seriously and multiply wrong.
The comment violated a cloud of implicit but well-established rules about writing for human eyeballs (that what I was mostly complaining about). It violated a git rule that had nothing to do with ‘parsing’ of the comment except in the trivial sense that git log and gitk summaries only display the first line. And it wasn’t my engine making the assumptions.
And nothing here has anything to do with what you call the Tao of Unix – you got even that wrong, you should have referred to Postel’s Principle instead.
I LOVE LAMP
@esr
> This is seriously and multiply wrong.
Undoubtedly, but don’t you love being wrong? It is the matrix of learning and growth.
> The comment violated a cloud of implicit but well-established rules about writing for human eyeballs
Perhaps I live in a cave, but I am not aware of such rules, in fact that sort of asterisk boxing seems pretty common to me. Perhaps you are confusing “well-established” with “well-established amongst my group.” Probably not, I’m sure it is me who lives in a box.
> It violated a git rule that had nothing to do with ‘parsing’
Which rule? You mean the “I only show the first line of your commit comment” rule? Seems to me that the first line of his comment, while not perfect, is adequate to communicate his commit changes. But I deal with programmers who frequently don’t put anything in their commit comments, so I don’t really trust them anyway. In the ideal world you’d put more than a title in there, certainly for substantial changes, but perhaps the first line should be a complete title.
> And nothing here has anything to do with what you call the Tao of Unix – you got even that wrong, you should have referred to Postel’s Principle instead.
I was referring to Monster’s invocation of the Tao of Unix, and I think it does, at least the last paragraph does, namely the idea of small separate modules, readily joinable by a common format — text. My point is that that free form text depends on a number of assumptions as to its format, assumptions often not clearly stated, and that newbie Unixers are supposed to learn by osmosis. Something that also leads to lots of extraneous and promiscuous parsing. And you know what happens if you are promiscuous, right? You might get a really nasty itch.
Me? I prefer explicitness.
>Perhaps you are confusing “well-established” with “well-established amongst my group.”
No, I mean well-established everywhere. See this comment – even lawyers know these rules.
> No, I mean well-established everywhere. See this comment – even lawyers know these rules.
Confused. That might apply to your second point, however the thing that raised your ire against the humble asterisk was the the guy who split his sentence with asterisk boxing. That is what I was referring to. But perhaps I am being ditzy.
Just as an aside, I have had the misfortune of reading many legal documents. I don’t think lawyers should be considered a standard for good English, certainly not clarity. Their words are often deliberately ambiguous, and when they are not, they are often layered with hidden meanings that only lawyers really understand. Which is to say “Plain English for Lawyers” is an oxymoron.
>I don’t think lawyers should be considered a standard for good English, certainly not clarity.
Which is why esr said “even lawyers”.
He’s saying “even the most willfully unclear people on the planet recognize this rule of clarity”.
@Jessica Boxer and SS
Conciseness matters, especially to programmers.
is needlessly verbose as compared to:
Notwithstanding SS’s long-winded Yellow Brick Road illogic that removing the asterisk from C, or forcing some markup language in C for comments, would somehow impact lack of common sense above.
SS continues to demonstrate the Dunning-Kruger effect.
Dunbbell?
@Jessica
> But I think he is right. I am loathe to dig through the comment thread, but my memory is that his error was a misplaced apostrophe.
He expressed the attitude that ESR was “nitpicking” about the asterisk. In doing so, he misplaced an apostrophe, which I used to demonstrate how the literal meaning of what he wrote was changed by such a “nitpicking” error. And while I thought when I wrote my reply that he meant to put the apostrophe before the
s, I am one of those humans, smarter than computers. A computer program has no way to know when “peoples'” really means “people’s”. I know you’ve argued for the compiler emitting warnings about common errors that compiler writers can reasonably detect. I agree that it’s a good idea to try to do this for certain particularly nasty cases, but it is impossible to catch them all, and there’s a risk of “crying wolf” and rendering the warnings meaningless if too often the programmer really did mean to say what he said.To reiterate: it’s not that he made the error placing the apostrophe; it’s that the very use of the word “nitpicking” to describe an error in syntax or symantic markup indicates a cavalier attitude inappropriate for computer programming. Again, consider:
rm -rf . /I think anyone who has used *nix knows what the above is intended to mean. Unfortunately, some people might not realize what it means to a shell. I don’t want anyone who thinks it’s “nitpicking” to worry about that space between the dot and slash anywhere near a computer for which I’m responsible.
One of the reasons I occasionally gripe about the lack of a “preview” feature here is that I like to use it to reduce the frequency with which I inflict errors on the readers. I guess “preview” functions like your warnings in that regard.
Speaking of Preview… HTML fail on the <i>
@JustSaying
> Conciseness matters, especially to programmers.
It might matter to many programmers, but that doesn’t mean that it should matter. Concision and clarity are often at odds, and the good programmer always chooses clarity over concision.
To be clear, sometimes concision increases clarity, in which case it is the right choice. But clarity should always be the goal. Programs are too hard to understand to allow ourselves the luxury of “cleverness”.
Some algorithms are complicated and subtle, though not many. How terrible to amplify that difficulty with needlessly hard to understand implementations!
I know that many people still use “i” as the variable in their for loops. To me that is the very archetype of bad programming choices. I have expressed that view here before and been shot down, but I still have yet to see anything approaching a good reason why.
for(int i=0; i<l; ++i) s += a[i].v;
for(int accountNum = 0; accountNum < numAccounts; ++accountNum)
total += accounts[accountNum].value;
Is there any doubt which is clearer? What possible reason is there to choose the first over the second?
(BTW I predict a total pratfall here as WourPress munges the formatting of the second, but even if I don't get my point across, perhaps my utter fail will at least amuse you a little :-)
So, yes, I really could not resist the temptation rejoin, depite whatever ESR says. Happened to glance over the thread now a couple of days later and here I find Jessica Boxer disagreeing with the lot. And besides, how is this discussion still so dumb (apart from the name-calling)? Such a long list now! Are you really not getting the point?
@kn:-
> “exactly what message were you trying to get across with rhetorical questions like these
So, how much of a distance is there between a hacker and a criminal?
How much mental degradation is needed for a hacker to finally cross the line and become anti-social (given that they are unsocial from the beginning)?
immediately followed by this conclusion
Personally, I don’t think it is too safe to get involved in any sort of discussion with a hacker and you really cannot be too careful around them”
First part, I was drawing parallels, which means that certain characteristics in a person can motivate him/her to move from hacking to crime, but it is logically inconclusive that every hacker will become a criminal. Second part, demonstrated by this thread’s off-topic discussion itself, and although you can say the same of me, the reasons will hopefully become clearer in the upcoming replies. One example by Jon Brase can immediately be quoted, though – “To be human is to be an insufferable ass.”
@esr:-
> “The comment violated a cloud of implicit but well-established rules about writing for human eyeballs (that what I was mostly complaining about). It violated a git rule that had nothing to do with ‘parsing’ of the comment except in the trivial sense that git log and gitk summaries only display the first line.”
Where are these rules established? For which language(s)? I can see different commenting styles in K&R and Stroustrup. After learning from diverse sources, all programmers don’t even use the same code indentation styles, let alone commenting styles. Also, what about shebangs and line-spacing comments found in countless places:-
#!/bin/sh
#
# This program does X
#
Or
/*
* This is xyz
*/
Both of these are pretty common. How do you handle them in git? Besides, everyone doesn’t use git. What style of commenting will ensure satisfactory results in all versioning systems?
@JustSaying:-
> “Here is a list:
* item 1
is needlessly verbose as compared to:
item 1”
Here is an actual list:-
* Item 1
* Item 2
* Item 3
compared to:-
Item 1
Item 2
Item 3
The second one is more concise, while the first one is easier to read because it provides visual cues, or markers, to guide cognition and interpretation within the brain.
@The Monster:-
> “He expressed the attitude that ESR was “nitpicking” about the asterisk. In doing so, he misplaced an apostrophe, which I used to demonstrate how the literal meaning of what he wrote was changed by such a “nitpicking” error.”
Wrong in that I did not express (the attitude) that ESR was nitpicking about the asterisk. I expressed (tried to) that ESR was nitpicking about a comment, an entirely different matter (more on this below). Right about the rest.
> “…it’s that the very use of the word “nitpicking” to describe an error in syntax or symantic markup indicates a cavalier attitude inappropriate for computer programming.”
Wrong. The “nitpicking” was about comments. For most programming languages it does not make sense to say that there is a syntax error in a comment, and in most programming languages comments have no semantics for the interpreter/compiler.
> “I don’t want anyone who thinks it’s “nitpicking” to worry about that space between the dot and slash anywhere near a computer for which I’m responsible.”
Right. The example you have quoted, rm -rf . /, is interpreted and executed by the computer, so it requires extreme care and caution, whereas comments are parsed/understood not by the computer but by people, and because people are smarter than computers, you were able to understand my meaning despite the misplaced apostrophe, as you claimed – “And while I thought when I wrote my reply that he meant to put the apostrophe before the s, I am one of those humans, smarter than computers.”
I don’t assume that you interpret a comment like a computer interprets code, and I don’t think anyone else does either. Eric’s problem was partly about git’s behaviour, but partly it was also about how he or you or someone else will have difficulty understanding the intended meaning. For me, as long as I can relate a comment to the code, I find no serious impediment to my understanding the meaning, which is why I posted my initial comment in the first place, which led to all the pandemonium.
@Eric:
Instead of Orc, might I suggest Kobold:
http://en.wikipedia.org/wiki/Kobold_%28Dungeons_%26_Dragons%29
@Jessica:
May I suggest that you have managed to confuse two items in your example: loop variables vs. persistent variables (there’s probably a technical term for what I’m missing). To rewrite your example:
for(int i = 0; i < numAccounts; ++i)
total += accounts[i].value;
In the first example, critical information had been one-lettered (numAccounts, total, accounts, value). In some ways, the use of 'i' provides a nice indication that it has no other real meaning. Using accountNum makes me wonder if we are actually looking at every account which could possible exist (or that accountNum is a valid number which if converted to text would be the identifier that would be put on cheques, etc.), as opposed to just every item which happens to be in the array.
Spiraling nitpick
Erosion of soul and time
Tepid Aspercreme
@Garrett
> Using accountNum makes me wonder if we are actually looking at every account which could possible exist (or that accountNum is a valid num
I don’t understand what that means.
> In some ways, the use of ‘i’ provides a nice indication that it has no other real meaning.
To me it means you are too lazy to think about what it actually means. This example is small, and not hugely important, but to me, as I say, is the very archetype of lazy concision over careful clarity. Where it becomes really important though is in loops inside loops inside loops, where you have to remember what i, j and k mean. Good variable names are a way of documenting your code.
However, and to address the discussion I had with Jay Maynard on this earlier, here is a perfect example where giving up a little control and using a higher level construct makes things considerably better. If you language supports it, a better solution to this would be:
foreach (var account in accounts) total += account.value;
Or using something like C# Linq:
total = accounts.Sum(account => account.value);
(If you are unfamiliar with the syntax, the stuff in the brackets is a lambda function.)
I really shouldn’t buy into this conversation but…
I think you miss the point. Eric’s point(And JustSaying’s example) isn’t that the example given isn’t concise… it’s that it implies that there’s a list when there really isn’t. Writing a list of one item is the opposite of clarity(did they mean to put more things in and accidentally submitted too early?). Writing a list of two items where the second item is a continuation of the first item is even worse (maybe “function” is actually meant to be a legitimate item in the list?).
@JonCB
> I think you miss the point.
That is always a possibility.
> Eric’s point(And JustSaying’s example) isn’t that the example given isn’t concise
I was making a more general point about the aphorism that JustSaying offered as a maxim for programmers. It is wrong, and toxic, so being the helpful soul I am I thought I’d chime in.
> Writing a list of one item is the opposite of clarity
I don’t agree that that is universally true, though it might be true in this particular circumstance. My “Urgent To Do list” has but one item on it, but it is not misleading, there is useful information both in the item and in its list-iness. Just to be clear that item is NOT post comments of Eric’s blog, which shows how screwed up my priorities are.
Nonetheless, the whole foundation for this argument is the assumption that an asterisk designates a bullet point marker for a list. This often true, but certainly not always. They are frequently and commonly used as decorative items, as is plainly the purpose in the example that brought such fire and brimstone on the humble shift 8 key.
And really, who can possibly object to a little decoration to cheer the place up a little, assuming that it doesn’t detract in some significant way. I bet you don’t even have any photos on your desk ;-)
The “i” for the for loop ;-) is good choice for programming mathematical formula. Mathematicians have a convention of using i,j,k,l,m,n for matrix (array) and vector (table) indices.
@Jessica Boxer:
To Garrett’s point about loop vs. persistent variables, let’s take these examples of yours:
> foreach (var account in accounts) total += account.value;
> total = accounts.Sum(account => account.value);
In Python, something similar might be:
total = sum(account.value for account in accounts)
Which is fine, and I certainly wouldn’t shoot anybody for writing that. But most of the time, I write more like this:
total = sum(x.value for x in accounts)
To me, it is actually clearer that x is a throwaway than that account is a throwaway. Not only that, but the exceedingly short Hamming distance between “account” and “accounts” makes it, IMO, much clearer to use “x” than to use “account.”
@Jakub Narebski
> The “i” for the for loop ;-) is good choice for programming mathematical formula.
This is obviously a major outlier, and would indeed be appropriate. I might add that it is an illustration of the same dysfunction in mathematics, but I probably shouldn’t go there, because there are some real mathematicians here.
@Patrick Maupin
> but the exceedingly short Hamming distance between “account” and “accounts”
But account and accounts are entirely different types so any errors would immediately be surfaced by the compiler when using an explicit type system. But if your type system quacks, you might have more problems of this kind.
@Jessica Boxer:
It’s not about the type system. It’s about readability. I’m talking about Hamming distance for my eyeballs. “account” looks very similar to “accounts” and I have to look twice and think about it. I have no such issue with “x”, thus I find “x” clearer in this instance.
But this would change if it were not a one-liner. On the fifth or sixth line of a loop, “account” would be much clearer than “x”.
(There is also the issue of, for the sake of standards and readability, not wanting to have a single statement exceed 80 characters — a limit which is quite low for some one-liners, especially if they are already indented inside a loop inside a function inside a class.)
@Jessica Boxer
> Concision and clarity are often at odds
Agreed, I did not write that only conciseness matters. Note the word needlessly in my prior comment.
You are conflating semantic clarity with naming. To achieve both semantic clarity and conciseness, use the functional paradigm.
I prefer a syntactical sugar (for my new language under design) that uses type inference to determine a lambda function is intended and orders the function parameters alphabetically, thus more concise.
Scala’s underscore sugar would be incorrect because the assumed parameters order would be transposed, but we can fix.
Note how in Scala the ‘return’ is unnecessary. The entire function is an expression whose value is the return value.
@SS
You don’t realize that your latest comment was irrational? Dunbbell for sure.
The traits of a hacker have nothing to do with criminality. Hackers roll up their sleeves and find solutions to their problems, rather than waiting for someone else to do it for them. That is essentially what hacking means.
This blog had nothing to do with comments in programming languages. It was about not making bulleted or numbered lists containing only one item in comments for commits to source version control programs. It is about the english language.
You did not even realize that Eric’s point was about not making bulleted or numbered lists containing only one item.
But not left?
(surely you won’t get this humor about not using a less ambiguous word, e.g. ‘correct’)
@Jessica Boxer
I replied above before reading your subsequent comment.
Your list title is probably redundant with the filename.
I believe Eric’s point was that the only text in the commit comment was the announcement of a list of changes containing only one bulleted item. Thus any list title was redundant to the implied purpose of a commit comment, which is to list the changes. In absence of a list title, the asterik for a list containing only one item is noisy or ambiguous decoration (did it mean multiplication or needless bullet?).
@Patrick Maupin
Python conflates the generality of fold with the specific lambda function that sums, or say it inverts the control making the lambda the caller. It can be thought of as syntactical sugar that is equivalent to the following.
Fold is a general ordered enumeration that computes a single value. The type of computation in this example is a sum of the elements.
Which of the above is more quickly understood?
I suppose the Python, because the inversion-of-control places the desired result as the command, i.e. to sum. But the Python syntactical sugar is not sufficiently general to handle all cases of folds, thus we still need the functional paradigm for outliers.
Using partial application, we can get closer to unifying both approaches.
Realize the first line above would be a standard library function so we are only comparing the clarity of the second line to the Python sugar.
In Why Python and also a 2012 user group presentation, Eric argues that the single most important property of a language is the ease of recalling the meaning of the code in the future, i.e. semantic readability a.k.a. the declarative property.
@JustSaying:-
You seem to be arguing just for the sake of argument. Well, at least I can humour you.
> “The traits of a hacker have nothing to do with criminality. Hackers roll up their sleeves and find solutions to their problems, rather than waiting for someone else to do it for them. That is essentially what hacking means.”
It seems that you have studied enough criminology or biology to know the exact neurobiological similarities and differences in the brains of highly motivated people (and done so with an MRI scanner, probably). (As an aside, criminals don’t “wait for someone else” to solve their problems for them.) Although you have conveniently omitted the last part of my statement, “it is logically inconclusive that every hacker will become a criminal,” I think that you, at least, are surely not dumb enough to miss the fact that external stimuli provided to motivation can bring about bizarre results. Most hackers do ignore these external stimuli and stick to their intell… er… (ESR had expressed discomfort…) …creativity.
> “This blog had nothing to do with comments in programming languages. It was about not making bulleted or numbered lists containing only one item in comments for commits to source version control programs. It is about the english language.”
???
esr: “…I’ve just fixed about the hundredth comment that does this”
When did this article become about the English language instead of about comments, apart from the fact that people were talking about comments that are being written in English? You are right that it pointed out “lists” that logically contain a single item, but really, “this blog had nothing to do with comments in programming languages”? It also pointed out comments containing genuinely and logically different items in a list. The whole thing was about how you write the comment itself, not about the English you use in doing so; a natural language does not have anything to do with any version control system whatsoever, unless I am unaware of some sort of high-AI VCS that parses English as a natural language.
> “Eric’s point was about not making bulleted or numbered lists containing only one item.”
Of course. But I was not replying to Eric, I was replying to you. What you were talking about in your instance was a different matter. It was about demonstrating conciseness, which has nothing to do with lists as such (it encompasses a much larger scope). You used a one-item list for your demonstration, I used a multiple-item list for mine.
> “But not left?
(surely you won’t get this humor about not using a less ambiguous word, e.g. ‘correct’)”
As I said, you seem to be arguing just for the sake of argument. Actually, I did consider writing “correct,” but then thought that being such smart people, you would understand the context. “It is about the English language”, you see.
And we can eliminate the ‘, 0’.
Since we are discussing semantic clarity and conciseness of programming, can anyone provide an example of the idiomatic Python code for performing a fold (e.g. sum) on an iterable while simultaneously (in one traversal of the enumeration) creating a new iterable with the same elements converted (e.g. from integers to strings)?
I’m interested to compare the conciseness and clarity of a generalized functional approach employing the State monad and Traversable.
Correction:
I will preempt and see if anyone can express more cleanly in Python? Note the first line is a reusable library function, thus won’t appear in user code. The Python code should compete with the third line below.
This returns the tuple.
(3, List('1', '2', '3'))The formalism.
My prior example may (if + is overloaded) have needed a type declaration on the reusable library function only.
The difference from Python is no more unit tests for the invariants that are type checked. The hope is that with well designed libraries, the type declarations mostly don’t appear in user code and the expressive resonance is close or better than Python. Statically typed functional programming for the masses.
@Patrick Maupin
> It’s not about the type system. It’s about readability.
In reading you of the context of the singular on the left of the operator, which is usually enough. I do it all the time, it is much clearer that way. I have done both ways, you should totally give it a try some time.
In regards to line length, I agree that is a problem, however, I think constraining yourself to 80 characters is a mistake in the world of modern super wide screens. “Tools” includes more than software tools. As an example, any programmer who does not have an SSD and two monitors at least 19″ is inadequately provisioned in my opinion.
But of course there are exceptions to every rule.
> JustSaying
> You are conflating semantic clarity with naming.
Not really, we are talking about concision in general, including naming, but I agree it goes broader than that, and it is the semantic fog of C++ that has made me turn away from C++ for better options.
> To achieve both semantic clarity and conciseness, use the functional paradigm.
Sorry, that is only half true. There is plenty of opportunity for obscureness in the functional paradigm. There is certainly a beautiful elegance to the composition of functions, but it certainly can be hard to read. You example is perfect:
> total = accts.fold( acct, total -> total + acct.value, 0 )
You have to think for a while about what that means. The C# version is easier:
> total = accounts.Sum(account => account.Value)
But that is because it happens to have a function called Sum (which I am sure Python either has, or would be easy to write.) Even here the lambda is a little obscure, and often the for loop is in your face simple, and might be a better choice. However, when you start chaining them:
> total = accounts.
> Where(item => item.LastName == “Boxer”).
> Sort(item=>item.DateOfBirth).
> Skip(5).
> Take(3).
> Sum(item => item.Value)
It gets way more obscure. A properly commented procedural algorithm often is much clearer here.
For another example, have a look at a few Unix shell scripts with multi step pipelines. It is elegant for sure, but often extremely hard to decipher.
Needless to say WordPress ate my formatting.
@JustSaying:
That depends on what you consider a primitive to be. In Python, “sum” is a primitive that takes an iterable. That doesn’t conflate anything with anything, and is orthogonal to the iterable you insert into it.
But most of what I write are usable library functions, and it is easy to write such a function that makes the library client readable :-)
That’s nice, but one of the things I like about Python is the complete and utter lack of barriers between “user” code and “library” code.
OK, but it’s not a good example, because if I have a tuple containing (‘1’, ‘2’, ‘3’), I wouldn’t bother creating the 3 in front of it, because that’s simply the built-in function len().
A better example would be where we started, which is to return the sum as well as your tuple, e.g. (6, (‘1’, ‘2’, ‘3’)). OTOH, I don’t understand why it’s important to do the addition for the sum and the conversion to a string all in the same loop, especially if you have a functional paradigm, where two loops could easily be scheduled together by a smart compiler…
A high-resolution screen is a must (see Dell’s faceplant with their “Developer Edition” ultrabook with 1366×768), but you can only look at one monitor at a time–you only need two if you aren’t adequately provisioned with an OS with hotkeyed desktops. ;-)
@Jessica Boxer:
I need more coffee or something, ’cause I didn’t really understand this
It’s not about the width of the screen. Why do you think books and newspapers have columns that are narrower than the width of the paper? It’s a readability issue.
In Python, 80 columns is also normative. Especially if you have code checked into a public repository and you don’t want viewers seeing random line breaks at weird places, 80 is the right number. But I think we both agree there should be some number, and this means that there will be some set of coping mechanisms to get the line size under that number. Changing the number just changes how often you need a coping mechanism.
@Jessica Boxer
That is syntactical sugar. Let’s compare apples-to-apples.
Is it semantically more obvious (i.e. declarative) than the FP’s sugared version above? I find the sugared FP version makes more sense than the C# when read literally, i.e. “fold the accounts into a sum of account.value”. How do I read your C# literally in a way that makes sense (“sum the accounts on account for the account value?”). The Python below reads literally, “sum the account.value for (all) account in accounts”.
Disagree, your chaining example is very declarative and semantically clear, although unless the compiler is smart enough to combine all of those orthogonal operations into one traversal of the iterable (generally a complex deforestation problem), for performance I might be tempted to put all logic (except the sort) into one imperative function (i.e. employing state for the skip and take) using a Traversable and State monad. But this would make the logic less declarative, counter to your claim.
I am placing my code between <pre> </pre> tags.
I stated “it conflates the generality of a fold”. And for apparently no comparative advantage in readability (see above). The Functor, Applicative, Monad, and Traversable are based in category theory, which enables mathematically proven composability that ad hoc primitives can not. Scala is running into limitations and weird bugs (“corner cases”) with their (non-category theory) ad hoc collections library, as Scala users push the limits of composability.
I provided an example of such composition, as follows.
This is not only demonstrating performance, but more importantly the generalized composability of operations.
The loops can’t be run in parallel on multi-core if the one of the operations requires the result of the first. And AFAIK otherwise combining them is generally a complex problem of deforestation in strict evaluation languages (i.e. Python, Scala, and mine), which I mentioned that Traversable and the State monad can resolve.
@Jessica Boxer:
The implicit map (or, implicit Select if you only speak LINQ) in the C# Sum, and a lot of other LINQ functions, has always bothered me a little bit. Why isn’t it accounts.Select(account => account.value).Sum() ?
In my prior comment, I forgot to attribute the last two quotes to @Patrick Maupin.
@Patrick Maupin
That’s nice too, but it comes with the tradeoff of writing unit tests for mundane types that could be tested at compile-time, losing enforced S-modularity (cf. Wadler), and when only required sparingly I find that documenting my types in reusable APIs aids in understanding the API. What I don’t want is many type declarations cluttering the semantic logic. Diverging from C++, C#, and idiomatic Scala, I prefer type declarations separated from the function parameters (similar to Haskell), so that the semantics are not cluttered (the type declarations can be occluded without blocking the semantic code).
I am not sure which will end up being superior. I don’t think we’ve had a sufficiently well designed typed language yet. Haskell has global type inference so you rarely need to write types, but Haskell has other issues that prevent us from comparing to Python. There will be no way to get global type inference with subtyping (Haskell doesn’t have subtyping– no lazy language can…).
I have thinking about if what I’ve learned about category theory could be incorporated into a unityped language (e.g. Python), and whether I would prefer to go that direction than a typed language. I asked for the idiomatic Python code, to learn the state-of-art in Python now.
It will be more useful to discuss if there is something real to compare to in the market.
Patrick Maupin on Thursday, February 28 2013 at 12:24 pm said:
> Changing the number just changes how often you need a coping mechanism.
Indeed, as does using Fortran derived variable names like i.
And sorry about the other thing, I had too much coffee and my brain finger interface failed.
@JustSaying:
Yeah, but things can be so general they are impenetrable. I understand this drives your desire to put some things in libraries; but most programmers like to be able to easily comprehend the libraries they use, as well.
I saw above, and like Jessica, I disagree. If it was all about power and generality and not syntax, we’d have all been programming lisp 20 years ago, and half of us would still be doing that and the rest of us would be in a Haskell derivative by now.
And string theory enables reasoning about physics that Newton never even dreamed of. Nonetheless, a lot more real-world work gets done with Newtonian physics than with string theory. Have you considered that perhaps your brain is wired differently than the average programmer’s? If that’s the case, you might be very productive, but at the same time you might (I’ll reserve judgement for now) also have a hard time getting your ideas widely accepted. Maybe you’re the current generation’s Oliver Heaviside.
References:
http://lambda-the-ultimate.org/node/4660#comment-73672
https://groups.google.com/d/msg/scala-language/PIhmEeMg-Ms/F4iler2zbsIJ (read to end of thread)
@JustSaying
> That is syntactical sugar. Let’s compare apples-to-apples.
Well yah, but Python and C# are both syntactic sugar on top of machine code, so if it is better it is better. What you might say it is a special case that is specially handled, and I’d agree.
> total = accounts.fold( sum account.value )
>Is it semantically more obvious
Not to me.
> How do I read your C# literally in a way that makes sense (“sum the accounts on account for the account value?”). The Python below reads literally, “sum the account.value for (all) account in accounts”.
Oh you big cheater!!! No it doesn’t the Python reads “fold sum account values” which is definitely a huh.
The C# has that exotic French like syntax, where you put the object up front, and refer to it by a pronoun. Par exemple, «Ces romains, ils sont fous.»:
The accounts, sum them using item.value
See, classy and exotic. You gotta love that.
> Disagree, your chaining example is very declarative and semantically clear,
I manage programmers and these sorts of chains utterly confound them. I might add they are also pretty hard to debug with the tools we typically have. I think they are quite lovely, but often impractical.
> although unless the compiler is smart enough to combine all of those orthogonal operations
FWIW, Linq does do this, though obviously it has to block on the sort. As a matter of fact, often times, if it is reading from the database, the work isn’t even done on the client, it is translated into suitable SQL and run on the server to take advantage of the indexes and extra data the database contains.
> I am placing my code between
tags.
Ah, now this information is worth the price of admission.
@Patrick Maupin
Seriously, do you think for an average programmer with no prior knowledge of either, the Python syntax below is easier to comprehend? I can’t see any significant difference in comprehension of the two below. Could it just be you are already more familiar with Python?
I have been able to make and explain the State monad simply. And I can likely make it even simpler. But that is not the point here.
Behind the Python syntax is also some complex code somewhere. If Python’s sum syntax can be a standard that no one needs to look at the source code for, then so can my sum function. If Python’s sum source is consider user code, then so can my sum’s source. Apples-to-apples comparison please.
I have written no user code on this blog page that looks anything like the complexity of Haskell, so the reference to Haskell’s complexity does not apply.
@JustSaying
> so the reference to Haskell’s complexity does not apply.
What I know about Haskell (apart from a playing around with it) is that I once met Simon Peyton Jones at a conference. He is one of the nicest men I ever met, and, although having quite a few years on me, is quite adorable. Nonetheless, he is also so super smart that I felt like a talking dog.
I’m thinking if he, Eric and I ever got together, all I could do would be to drool uncomprehendingly, and perhaps offer to make the tea.
@Patrick Maupin
I understand the benefit of maximizing how many people in the community can comprehend all the code (including the compiler and the built-in syntax source code). But not all user code programmers understand compiler concepts. There is no such thing as one level of conceptual complexity that can apply to all coding.
If Python’s syntax is easier for those specialized cases that it handles well, then layer a library for it on top of the FP library. But don’t toss the FP capability (Eric states that he has been adamant about this w.r.t. existing FP features, even though Guido wanted to toss it).
It is not a mutually exclusive proposition.
I am attempting to make FP conceptual layers even more accessible to more of the community. Sometimes they will be the better solution, and in those cases, those who are not qualified (and don’t aspired to be) should not be looking at the code.
Agreed, something as inpenetrable as Scala or Haskell will not work. It has to be something that doesn’t make people’s head hurt when they try to move up a level in their abilities. When first learning those two languages, my head hurt for months. I still haven’t mastered Haskell.
@Jessica Boxer
They can write each step as a separate assignment statement for easier comprehension. Then the debugger can break between them, so you can look at the intermediate changes to the iterable.
Or let them write it as non-declarative, imperative spaghetti.
I think their rigidness is because they come from the historical imperative world, e.g. C to C++ to C#. I did too, then I jumped ship. As we can see, the Python syntax is much more declarative, so the future is declarative. I bet if we teach newbies the simplified FP way of thinking, they will absolutely detest imperative spaghetti, because humans would much rather state the result they want (e.g. Take(3) elements) and some long-winded imperative code where the semantics are obfuscated in the extraneous details. Older programmers have gotten so accustomed to the for and while loops, that they feel more comfortable with them, but it isn’t natural to newbie who would rather say, “do this” and not “here is how to do this”.
>Seriously, do you think for an average programmer with no prior knowledge of either, the Python syntax below is easier to comprehend?
FWIW (coming from a below average programmer – I’m a sysadmin with only a couple of hundred hours of formal training in Visual Basic and C++ several years ago)
I could almost immediately understand
total = sum(account.value for account in accounts)
but it probably helped that Patrick already related it to a common for loop
otoh,
total = accounts.fold( sum account.value )
made no sense to me until I
a/ read what ‘fold’ means
b/ realized it would have to do something similar to that for loop
c/ tried to read it as a object.method(params) – like statement
with lots of help from various comments.
so I’d say the former is more accessible or easier to grasp.
@JustSaying:
Yes, there is, and no.
reduce (similar to fold) has been in Python almost forever (since around ’93, I think). sum() was added in v 2.3, a decade later, because that was the primary use case for reduce, and because real-world use (and answering questions on newsgroups) showed that reduce (and map and filter) were voodoo black magic for a lot of users.
In Python 3, reduce was moved out of the core into the functools module, where the functional black magic now lives…
@JustSaying:
Umm, not really. I mean, yeah, sure, but only because it’s a C function for speed:
def sum(thelist): total = 0 for item in thelist: total += item return totalEminently understandable, not voodoo black magic at all.
Sure, but the Python sum function (unlike anything relying on reduce/fold) maps neatly to concepts that people probably knew intuitively, but certainly learned in grade school.
Real-world usage has shown that the concept of iterables is extremely easy to learn and teach, but fold/reduce hasn’t worked out that well. You’re right that it’s not rocket science, but the sum function I showed above is a viable template that semi-skilled programmers can use to write their own iterable consumer functions. In this case, the only thing really hidden inside the language is the iterator protocol, which has proven to be, from the average programmer’s viewpoint, conceptually simple yet powerful.
Just for the pedants in the crowd, the actual Python sum function allows a starting value (and the parameter name is different than the one I chose), so the equivalent Python would probably be more like:
def sum(iterable, start=0): total = start for item in iterable: total += item return totalIn any case, the point remains that this is a very simple function, and all the concepts (for loop iterating over an iterable, default parameter values, augmented assignment) have been battle-tested for years now.
And although fold has been around for awhile, not only has it not made much headway in popular languages, but it (well, the variant known as the reduce() function in Python) has been demoted from a built-in function to a library module that hardly ever gets used (relatively speaking), while the (admittedly not as general purpose) sum function came out of nowhere to become a lasting part of the standard library.
@Patrick Maupin
You are missing the field “.value”.
def sum(iterable, field, start=0): total = start for item in iterable: total += item.field return totalAnd that is not much different than pseudo-code below for a fold function.
def fold(iterable, callback, start=none): total = start for item in iterable: total = callback(item, total) return totalThere is no impenetrable increase in conceptual complexity. You know the latter is more general at least because the callback decides which accumulation operation to use, e.g. sum.
@kn
Okay so we agree the syntax is not more verbose, and as I anticipated, there is a legacy training barrier at preferring the concept of looping over the concept of fold.
Fold is quite easy to learn if you learn it first before learning looping, because it has numerous natural advantages. However, if you learn looping first, then your mind is handicapped by irrelevant imperative noise until you lose this unfortunate training bias.
“Iterable fold” is more declarative than “for Elem in Iterable” because it does not mention the irrelevant extraneous details of how to imperatively accomplish the result (i.e. the mind doesn’t have to maintain an irrelevant implementation concept of looping) and contains more semantic information– reduce the members to a single value (by chaining a callback on each member). Although a loop is more general in what it can accomplish, this makes it less semantic and thus less declarative– analogous to why we don’t program the detailed rendering of the bold <b> tag in HTML.
For example, readers should realize that even data types such as a disjunction of a value and `none` (or `null`) can be folded. Looping is not the implementation method because there is either one or no members (callback is not called if `none` or `null`). This makes it much easier to code seamlessly without boilerplate. The reasons for preferring the more declarative FP model domino in the code like this.
Such special case sugar that conflates with looping can be included to appease the old school programmers. This incurs the cost of not deprecating an inferior paradigm. I suppose this could be mitigated with a bi-directional translator between the sugar to the equivalent FP code, so then viewers can toggle which form they prefer to read.
I see an opportunity to teach a billion new programmers correctly. I believe that we are going to see a billion people become programmers by 2033, out of economic necessity. I base this on the repeating 78 year real estate market cycle, that peaked in 2007 and should bottom in 2033 (with a slight bounce first until 2016), with robotics and automation causing billions to be technologically unemployed.
@JustSaying
> I see an opportunity to teach a billion new programmers correctly.
Sorry to pop your bubble dude, but those billion new programmers are learning JavaScript. Which admittedly has some functional aspects, especially with jQuery, but they are more than offset by the horribleness that is JavaScript.
Does anybody know what these Javascript statements actually do? No cheating by testing it out…
To preempt a vacuous argument that the above is a semantic loop that terminates after the first iteration, a class could implement Foldable (preferred name instead of Iterable) on a plurality of members fields of the class, i.e. not organized as a data structure that can be extracted with loop control.
Edited to replace the offending word with “suboptimal” that caused the prior comment to go into moderation queue.
@Jessica Boxer
I can teach fold in JavaScript. Your correct point is most teachers are not. We see that relative measures of the uptake of computer science is woefully underperforming the other sciences that require equivalent levels of IQ. Thus arguably (instead one can blame debt-sustained economics) we won’t get even a million programmers in the current model. I did some research and discovered that all of the top-rated free online education for utter novices are suboptimal. I had some insight from trying to teach HTML on a whiteboard to some people who know nothing more than how to use facebook. They struggle with even understanding why the angled brackets are not used as plus and minus. I did let them try the free online learning resources, and they did not get it (there were too many learning gaps in the process). Thus apparently the market is wide open for someone to do a more effective set of lessons.
>Edited to replace the offending word with “suboptimal” that caused the prior comment to go into moderation queue.
For the record, I have no idea why WordPress’s spam filters consider this a bad word. I wouldn’t.
> are not used as plus and minus
Typo: greater-than and less-than.
@Jessica Boxer
> FWIW, Linq does do this
AFAIK, this is not true in general case of chaining FP. Perhaps the LINQ grammar is non-strict a.k.a. lazy evaluation and allows only constructions which are reducible to foldr and build. See section 2.2 of Cheap Deforestation for Non-strict Functional Languages, Andrew John Gill.
Also, the prior sections show some examples demonstrating the superior declarative property of functional versus loop imperative programming.
My prior reply to you about teaching FP in JavaScript is in the moderator queue.
@JustSaying:
> You are missing the field “.value”.
No, I am not. That is not part of the sum implementation.
If I wrote sum(x.value for x in accounts), the given sum implementation works perfectly. The magic is in the compiler understanding that “x.value for x in accounts” is a generator expression, and thus an iterable.
This misunderstanding of how you write sum() in Python completely invalidates the rest of your reply where you try to show that the mental complexity of dealing with sum() is identical to that of fold(). It’s not, because generator expressions are much easier to teach and understand than fold(). It would be very illuminating for you to present the wikipedia page on fold and the wikipedia page on generators to a newbie, and look at the reactions…
@Jessica:
With the understanding that JavaScript is exceedingly “helpful”, and with the assumption that it coerces the second operand to the type of the first operand and not the other way around, I would guess that we have the equivalent of “2” + “3”, and 2+3; that is to say, “23” and 5.
Whether I’m right or not, obviously the real problem is when one of these is hidden behind a variable name…
@JustSaying:
This remains to be proven.
Yes, in a lot of cases declarative beats procedural. But (to echo Jessica’s point about the average programmer) one of the best tools for both teaching and debugging is to “pretend you are the computer.”
When you are pretending you are the computer, looping is not “irrelevant imperative noise.” Quite the contrary — it is a paradigmatic map of how to properly proceed. Now what happens when teaching the student to implement a fold while deliberately, studiously ignoring this irrelevant imperative noise? “Pick one.” “Which one?” “Any.” “This one?” “sure. Now pick another” “This one?” “sure” … “This one?” “No, you already picked that one.”
I do have to say, though, that you are preparing for the day when we are all using quantum computers and “thinking like a computer” will be a completely different exercise.
@Patrick Maupin
Oh so then the extra code for .value is in the generator function somewhere.
Nope.
I don’t think so. A function which halts without completing of its computation, then continues where it was the next time it is called, is more obtuse than a fold function which calls a function for each member of the foldable.
To me, a generator is merely a pure function that inputs and returns an implicit parameter for the environment, so I don’t need any compiler syntax to model them. The way my mind works, I like to simplify concepts by unifying them to their most generative and general essence. This is the way I like to teach also.
I think it was Douglas Crockford who claimed that overloading the plus operator (or essentially weak typing) was the worst design error in JavaScript. Python is strongly typed at runtime, even though it has no compile-time type checking. Jessica, semicolons are never required in JavaScript.
I find I can do that mental reasoning more effectively when operations are declarative, i.e. holistic. If a specific member within a fold is suspected, then set a conditional breakpoint on the callback. There is no valid argument here for thinking in terms of loops.
When explaining the operation of the fold, the members are enumerated in some deterministic order. It doesn’t have to be a loop that is enumerating them, e.g. sequential callback calls in the fold function.
Also AFAIK imperative loops are not as generally parallelizable on multi-core. However without compile-time typing and enforced pure functions, you can’t get there in Python anyway.
Without enforced S-modularity due to lack of compile-time typing, then in theory every function can throw an exception at any time. So I presume this basically means emitting compile-time errors at runtime, e.g. “non-existent method call at line x in file y”. This won’t show up as much when only your well tested code is calling your functions. But throw the function into the wild for third parties to call, then I don’t see how regression and unit tests help, unless you can require every caller in the universe to run them before they ship.
Jessica: Personally, I think the FORTH world has it right: if a word definition (==function) takes up more than one editor screen, it’s too big and needs to be broken up into smaller, understandable chunks.
The standard editor screen in standalone FORTH environments is 16×64.
If you can get the definition of a function into the programmer’s vision all at once, you’ve really made a big win in comprehensibility. Wide editor windows hinder, not help, this process. Personally, I adhere pretty rigidly to an 80-column screen in my own work, though when I work on others’ code, I follow their conventions. My standard terminal window these days is 53×80, taking up the left-hand one-third or so of a 1920×1080 23-inch display. That goes with a font based on the one used by the IBM 3278/3279 CRT displays, which I find highly readable. The combination makes for great readability for me.
While the amount of code one can take in at one time, horizontally, will vary from programmer to programmer, I’d be surprised to find very many at all where that limit is as high as 160.
@JustSaying:
This becomes true once you can show that the average student can usefully make use of pure FP in a real world system. Once he has to start storing intermediate state somewhere, it turns out that generators are a very accessible way of doing that in an almost-FP fashion.
It’s well and good, but sometimes the real world intervenes.
We’ll have to agree to disagree here. A mental model that maps, even tenuously, to what the processor is doing is an aid to enhancing performance, as well as to learning and debugging.
CPython certainly doesn’t go there, but all the C compiler guys would be very surprised at your assertion that it is difficult to parallelize imperative loops of pure functions.
Yes, perfect proof of correctness by construction has been the holy grail for the last five decades, and it will probably be another five decades before people give up on it.
And yet, at the same time, you were explaining that you had to hide away the code that does useful stuff (like your count function below) in the library so that the average programmer wouldn’t have to sully their hands with the deep magic:
And you continue to seem to think that Python requires the same:
But it doesn’t. There is no “extra code”. The .value is in the generator expression just as shown. There is no special library for it — all that is required for the code I wrote to work as given is an object with a value attribute.
Here’s another example of a generator function from code I was working on this morning. It calculates the reset value for a register by adding together the reset values of all the fields in the register:
That’s it. There is nothing special going on, hidden from the programmer. It is perfectly understandable to the programmer that he could achieve the exact same thing by writing:
reg.reset = 0 for field in reg.fields: reg.reset += field.reset << field.lsb(It’s not quite the same in that the generator version has the useful attribute of atomically assigning to reg.reset and another useful attribute of not corrupting the enclosing namespace with a binding named ‘field’, but our junior programmer wouldn’t know or care about that yet.)
OTOH, the same junior programmer wouldn’t have a clue about how to code up your count function, and probably takes having to do the traversal and taking the zeroth element of that to be some sort of magical incantation that you always cut and paste. Do you you extoll the virtues of unordered functions simply to avoid having to explain why this is run by the computer backwards from the order any normal human being would execute it in?
SSA is a good thing for high performance compilers to know how to do, not to ram down the throat of unsuspecting students.
FP is like OOP — there are some problem domains that map to the concepts, and the paradigm kicks butt there. But at some point, the wheels fall off, and the ugliness of going outside the model usually far surpasses the ugliness of doing FP or OOP in a multiparadigm language.
Yes, all your points about correctness and performance are understood, but in my world, much real work is performed quite adequately with slow, buggy programs, which can, btw, be maintained by a more-or-less average human.
(BTW, your point about “some deterministic order” holds for Python, too. If I am iterating over a set or dict, I don’t know the order. But still, the concept is that I am iterating, not that I have a quantum computer doing them all at once.)
@Patrick:
http://www.newegg.com/Product/Product.aspx?Item=N82E16828103041
I think it supports 132-character columns natively. :-)
@Patrick Maupin (et al.)
I appreciate you engaging me in the discussion, because my purpose (in my recent role attempting to be a programming language designer) is to understand your popular perspective.
It was frustration with idiosyncratic corner cases in HaXe’s ad hoc typing design that drove me down the rabbit hole to explore Haskell and Scala circa 2008. Before that, I had never designed a language. The test case was a very elegant, compact, and fast OOP JPEG encoder and decoder, with the r/w bit twiddling and Huffman encoding in two orthogonal classes. I should port it to Python so I gain more insight into which types of coding Python is best fit to.
This is an application of the Rule of Least Power. Some people claim Python makes them 5 – 10 times more productive than Java. I don’t doubt it, because Java requires so much noise in the code. Scala produces 2 – 3 times more compact code, but Scala has too many surprises in its syntax, corner cases in its non-category theory (i.e. ad hoc) collections design, and in the diversity of obtuse conceptual ways its powerful typing can be used (c.f. some root canal examples).
Given the non-C (i.e. not Go or D) alternatives out there, e.g. C++, C#, Haskell, Scala, Lisp variant e.g. Clojure, PHP, Ruby, JavaScript, Lua, Fantom, and HaXe, it is not surprising that many developers choose Python. Programmers want to get some readable work done with the least BS.
I will stop if I am unable to design a marriage of what is best in Python, while attaining more fundamental power through unification of (reducing the number of) concepts and thus not exploding the complexity. This is an application of the Rule of Least Power. I think programmers (including myself) are justifiably skeptical given the track record of the aforementioned languages. I am even analyzing with interest while skeptical of theoretical hand-waving of Why Not Events.
The processor is reading 0s and 1s. Perhaps you shall be more productive coding in machine code. ;)
When a processor is looping or enumerating a fold, it is doing what I told it to do. There’s no advantage in the former mental model for understanding semantics nor for enhancing performance. Note, if my compiler doesn’t do deforestation, I can take of all chains in Jessica’s example and put them in one callback of one fold call. Voila the same performance opportunities as a single loop. Automatic deforestation eliminates this special case (boilerplate) coding, and enables reuse of orthogonal, smaller, more semantically meaningful functions.
I don’t want perfection– just to declare the invariants that can be checked, e.g. that the inputs to a function are of known types, so I won’t get an exception for calling a method (i.e. an operation) doesn’t exist when called by a third party that is unaware of my unit and regression tests. Then subtyping (OOP) becomes the only way to add operations to new types with S-modularity a.k.a. separate compilation.
Dynamic (i.e. uni-) typing works for self-contained projects with relatively few and simple external APIs. But AFAICS it won’t work for a mix and match modules system for the web applications that I am dreaming, c.f. our discussion in the prior Coding Freedom thread about HTML 5 and wishing the browser was an generally programmable applications platform.
No, I think Python doesn’t offer an equivalent declarative means for doing the same in one iteration without case-specific boilerplate, a.k.a. don’t repeat yourself. Your best so far was a separate call to len().
If the programmer doesn’t need that power which reduces verbosity, they will never see it. And if they need the power, but don’t need to know the magic, they still won’t see the implementation of the `count` function.
Somewhere there is code implementing `.value` to access that member. It was not in the C code you provided.
I had a typo. The code should have ended `.st(0)`, thus isn’t accessing the zeroth element– it is executing the state monad providing the initial count.
I agree that copy+pasting that without knowing what it is doing, would be mindless.IMO this can be easily explained to an average programmer.
And IMO it is worth explaining because it is a unifying concept. It is the way to avoid noisy boilerplate when threading some orthogonal (to another computation) state through the composition of functions, some that do and other functions that do not operate on that state. This is important because it enables reuse of (smaller, orthogonal, more semantically meaningful) functions without having to write special cases.
We avoid those root canal examples above.
No need to use it until your code becomes so ugly without that you realize you need some way to reduce the verbosity. The example I provided is rather trivial and can be done in a hand-coded iteration in Python without too much verbosity. But this complexity will explode in any language without the state monad in the general case.
Local State is Poison (google). Persistent local state is uncomposable. If we discard local copies of state, then I have only my aforementioned pure functional version of a generator.
We have two schools of training and thus thought. I think the pure functional one isn’t more verbose nor more difficult to comprehend for the same task. I showed this by example, c.f. sum(accounts.value for account in accounts).
I share your skepticism but not your pessimism, thus am willing to give it a try.
> The code should have ended `.st(0)`
I shall go rename that `.exec`.
> Voila the same performance opportunities as a single loop
Assuming the compiler does tail-recursion.
@JustSaying:
I’ve never taken DRY to mean “only have one loop ever over a particular data structure” but I’ve also never felt compelled to have every single loop in a one-liner.
With classes and generators, it is eminently possible to do similar hidden magic to reduce the verbosity in Python.
Somewhere there is code implementing `.value` to access that member. It was not in the C code you provided.
It’s syntactic sugar, not deep hidden magic. The real magic is the iterator protocol. A generator is a first class object I can do anything with. This:
Is really shorthand for something like this:
The generator expression is effectively shorthand for:
def getval_gen(accounts): for account in accounts: yield account.value getval = getval_gen(accounts)And as I showed earlier, sum() is a built-in function that is the same as:
def sum(iterable, start=0): total = start for item in iterable: total += item return totalThe iterator protocol has proven to be very easily understood; any programmer can write a function, like the predefined sum() that accepts an iterable, and any programmer can write a function, like getval() that returns an iterable, and finally, any programmer can do the shorthand to create a generator expression, which is like a lambda for iterables.
I honestly don’t think it’s as easy to explain as the iterator protocol.
But as Jessica points out, it is extremely easy to be too clever by half and code something that other programmers cannot understand. My understanding is that Yahoo rewrote Paul Graham’s code in something besides lisp, because the average programmer couldn’t maintain it, despite Paul himself being as efficient as dozens of ordinary programmers when working in it. This is the trap that Python avoids very neatly, because while it is certainly _possible_ to code Python as obscurely as that (practically everything is a first class object, and overloading is quite easy, etc.), it doesn’t usually feel _necessary_ to have hidden magic, so most programs don’t. It will be interesting to see if your new language can meet this balance, but judging by your example below, I don’t think it can, at least not yet:
count[T] : T -> State[Int,T] = t -> State( (x+1,t) ) tuple = List(1, 2, 3).traverse( count i.toString ).st(0) (3, List('1', '2', '3'))Even if we stipulate that the count is magic in a library, and that the list has to be created before doing anything with it, this calls 4 functions (traverse, count, toString, st). The equivalent Python calls 2 functions and has one for loop:
This is a noble goal. But one can (and I will) argue that taking the count of something and coverting the elements to strings are, in fact, orthogonal functions, and having two lines to do those functions is not a huge deal; in fact, sometimes it is quite confusing to see a loop do two unrelated things.
So says you. I’d like to see a real-world (generally useful outside some obscure field) function that does this, and how you code it in your language vs. Python, and no fair claiming that “this part of the implementation is not something the programmer will see because it’s reusable library code.”
Yes, local state is like fugu. Those who use local state are stupid, but those who don’t use local state — well, actually, they are usually the complete opposite of stupid. Look at who actually uses FP for what they consider to be real work. It is not the average programmer. In fact, it appears to be mostly academic stuff, and people who hire based on whether you are smart enough to do FP or not. (And, to the extent that XSLT is FP, non-programmers, but XSLT doesn’t seem to meet very many of the goals you have set for your project.)
I disagree, from what I have seen before, including your example. Heck, you even admitted you didn’t quite get the syntax right at first, and by practically any code complexity analysis, the Python wins. But it’s a toy example. I don’t want to spend a lot of time on a shoot-out, but a somewhat bigger example would be illustrative one way or the other.
Perhaps one of the other commenters has a useful real-world problem that should take around a KLOC or so?
@Patrick Maupin
Let’s compare apples-to-apples, Python versus hypothetical Copute (≈ Scala) or similar pure FP language with category theory paradigm.
Note I renamed `st` to `init` since our prior discussion.
The Python example has a structural violation of D.R.Y. (do not repeat yourself), because it requires two references to the input list `[1, 2, 3]`. Also the Python code is iterating the list twice, so potentially slower and less battery life. Also AFAIK the iterations can not in the general use cases be easily (automatically) parallelized by the compiler to optimize multi-core, because Python’s functions are not enforced to pure. Although this is not significantly problematic in this simple example, it is a fundamental deficiency of imperative compared to pure functional programming with category theory.
IMO, D.R.Y. violations will explode the extraneous boilerplate and duplicated CPU load in some use cases. But this claim needs to be proven in real world use cases. It is my theory that if there is wide scale reuse of small modules, this will become a significant problem. My dream is that we also open source at the module, not only at the project granularity. I envision a github for mix-n-match modules.
The Python example is more explicit, because the reader should know that `trav` will iteratively build the new subtype of Applicative that is returned by `cnt`, but it isn’t clear what type `cnt` is building. The novice user will need the help of the compiler (or look at the library code), so that when hovering the mouse on the selection of `[1, 2, 3].trav( cnt i.str )` or `cnt i.str`, a tooltip “State[Int,String]” is display. The `init` method of State returns a tuple of the two types `Int,String`, i.e. `init` provides the initial value for the computation of the first type, `Int`.
New example follows which shows in the general case (of this simple use case) Python needs the initializer too.
Python is structurally noisier, having two instances of square brackets (i.e. list construction calls).
Semantically the Python code reads literally in precedence order, “construct a tuple of the appendage of every member in `[1, 2, 3]` to `thing` paired with a separately constructed list of members which are converted to a string from every member of `[1, 2, 3]`”.
The comparison code reads literally in precedence order, “construct a State by traversal of `[1, 2, 3]` which simultaneously can compute the tuple of the appendage of every member in `[1, 2, 3]`, paired with a constructed list of every member of `[1, 2, 3]` converted to a string, then provide the initial value to append to, execute the computation and return the tuple”.
The Python semantics avoids the construction of the intermediate (discarded) State object at the cost of violating D.R.Y. because it is bottom-up, i.e. it reading from the output to the inputs. The fanout is on the inputs– the computation forks from the single output to duplications of the input.
The FP category theory semantics is top-down, i.e. reading from the input to the output– the computation forks from the single input to duplications in the output. In short, the forking is delayed for as long as possible. This is why it is a structurally more composable programming paradigm for the general use cases.
For the `fold` versus comparative use of a (n imperative, non-pure) generator, IMO the fold is much easier to understand for someone with no legacy bias. I think you unconsciously underestimate how much learning bias you have attained. C.f. as follows.
* For each member fold inputs, the prior or initial value and the member, to a callback that returns the next value. Fold returns the last value of that callback.
* A generator function can be called from its prior halted or initial state and can halt the computation to return a value. A generator which enumerates the members, can be successively called in a loop to a compute the next value, by inputting to each iteration of the loop the prior or initial value and the member. The last value of the loop is returned.
In your mind, the generator is more elegant and orthogonal due to a bottom-up bias that unconsciously ignores the verbosity that you already learned to abstract away in your mind. This same unconscious ability can be applied to FP paradigm. It is the training bias. It is analogous to learning to drive a manual transmission, we have to think carefully about the coordinating movements, then after much practice this becomes unconscious.
At the semantic level, we can trade unifying semantics for bottom-up details, but at some point this explodes in code that has so much boilerplate special cases, that it becomes impenetrable. C.f. this comparison for a simple use case “root canal” example of how impenetrable the bottom-up paradigm can be.
I hate a language that is all symbols. It is looks like gibberish. I also don’t like when the arguments come before the function call, because that is not the way we read naturally.
Also Lisp doesn’t inherently use category theory based OOP.
My intuition is I can write an algorithm that automatically converts the category theory form into your bottom-up preference, so that programmers can see which ever form they prefer. If this assumption is correct (and the vice versa direction can’t be automated), it would prove there is more information in the category theory form. I don’t think this transformation was possible without the unifying category theory semantics, because the compiler would not know also.
This will also help people see the difference in complexity and motivate them to learn the unifying top-down semantics.
Three (3) function calls in the general case in the `append` example I showed, not including the extra explicit call to construct the list twice. But this is nothing compared the exponential blowup of boilerplate, c.f. the link I provided above.
The key point is there is no function to compute the second element of the tuple in Python– reuse of functions has been replaced by boilerplate.
Indeed when the semantics are obscured by the boilerplate details.
C.f. the “root canal” boilerplate link above.
;) Please don’t associate FP with Haskell.
I could not find one comprehensible explanation of the Monad and State monad on the internet before. Much of this is the impenetrable, lazy evaluation, no subtying Haskell formal semantics. The Applicative is actually the more powerful (i.e. composable) construct, and it is simply the way to let functions that operate on types parametrized with those types, i.e. function x + y can operate on List(1, 2, 3) + List(4, 5, 6) without any boilerplate! That is extremely profound.
If we don’t get to that now, I will hopefully be getting there and getting back to you. I am by myself trying to launch a new social network this week based on a cross-platform browser extension and may necessary disappear into Deep Hack Mode extinguishing fires.
> function x + y can operate on List(1, 2, 3) + List(4, 5, 6) without any boilerplate!
Result is `List(5, 7, 9)`, i.e. `x + y` is applied to each respective member. This is not limited to collections. It is powerful for composition of any type that can implement Applicative, e.g. avoiding the boilerplate for destructing and reconstructing tuples or the disjunction with `null`.
Foo did we suffocate? A suggested valediction for the Dunbbell-Krueger .
With OpenMP you can mark a loop in C/C++/… for parallel execution…
Nb. I have recently heard a talk about using functional languages for parallelism (the ‘par’ etc. construct in parallel extension (?) to Haskell), and it looks interesting… if not for “legacy programs” problem.
@JustSaying:
I still think that your examples are more verbose and harder to understand for a newbie, but I will reserve judgement until I see a larger example (I think your presentation of the general case is actually quite unfair to Python, but I cannot poke further at your current example).
> function x + y can operate on List(1, 2, 3) + List(4, 5, 6) without any boilerplate
Umm, yeah, but how is that scoped? If I import a new module that automagically adds this capability, what will it do to code in other parts of the program? That sounds like a potential maintenance nightmare.
@JustSaying
If you remember the genesis of this discussion was my contention that clarity should always be chosen over concision. These examples are EXACTLY my problem with functional programming. It is the obfuscated C contest all over again. You can’t possibly thing that the meaning of these is as plain as a clearly written loop with good variable names.
There might be other benefits that matter in the 1% outlier cases, but the majority of code I see is not time critical,not subject to the need to correctness proofs, or in need of mass parallelization.
Of course there are programs that care about that, but even with them, it is only a tiny fraction of the program that they matter for, perhaps less than 10%.
I find your guys back and forth on Python or Haskell (or whatever FP JustSaying is proposing) kind of like a Chicagoan arguing with a Detroit resident as to who has the better weather.
If I may add a different perspective… (I’m a sysadmin and my coding skills are, um, rudimentary which would put me much closer to the “beginner” programmer you all are talking about than anyone else in this discussion) this is true. It may be plain as day to JS, but to a more pedestrian observer, no please a simple loop.
@Jessica:
(I’ve stopped using the Python reserved word “tuple”, and removed the superfluous parentheses to make this a bit more canonical…)
JustSaying can correct me if I’m wrong, but one huge difference between:
and:
Is that the former is not really functional programming. I mean, sure it looks a bit functional, but as I showed earlier, you can teach this by building on imperative concepts. And if your loop is broken, you can debug it by breaking it down into smaller imperative concepts:
src = [1, 2, 3] result = [] for x in src: result.append(x) result = len(src), resultYou can do this either mentally, or physically, as the situation demands. JustSaying appears to say you can do the same thing for debugging in his new language:
But that’s possibly not quite as true, since there is no looping…
Also, “intermediate changes to the iterable” is sort of confusing — there is no such thing in functional programming, is there?
He also says:
But frankly, imperative spaghetti may be harder to get right on the first pass, but is usually quite easy to debug (except for any sort of IPC or real-time code). That’s one of the reasons why (so far, anyway) I prefer techniques that ratchet the level of abstraction up one easy uZen per level. JustSaying may be right that “if we teach newbies the simplified FP way of thinking, they will absolutely detest imperative spaghetti,” because higher level abstractions are almost always a huge win for design. It’s only when you have to have a conversation about the actual design, with the debugger, with a colleague, or with yourself two years later[*], that you ever figure out that your abstraction level was too high.
JustSaying realizes that the machinery that can make higher levels of abstraction possible (such as his count function) is not viable material for all programmers, but his count function looks so different than the rest of the code that I’m not sure how you get there from here.
The thing is, the best C programmers I know are good assembly language programmers, and the best Python programmers are good C programmers. I happen to think that I am pretty good all the way up from creating flops out of transistors to high-level Python, but I’m sure that’s partly hubris. Most programmers don’t need to know how a flop works. In this day and age, most programmers do need to have a hazy concept of how a stack works, but don’t need to account for it as carefully as programmers of yore.
Anyway, one nice thing about Python is that you can program in a fairly declarative style, but switch if you need to (for debugging, or whatever). I hardly ever need to go down a level to C, unless I need speed. I don’t think I have ever used a Python debugger.
As you point out, for long-running programs where it is hard to pre-test all the edge cases, you can occasionally run into bugs that stronger typing would have found for you, but that’s not where I live, so it’s really no big deal for me. Also, most programs I write have but a single maintainer, so as we discussed earlier in this article, I shouldn’t code too cleverly. Which brings us to my footnote…
[*] I’m having a conversation with myself from 16 months ago right now, and it took me awhile to get back into understanding the design. It’s because the design is quite general, yet there is currently only one working example using this general library, and I am the only programmer, so the documentation, while it exists, has not been battle-hardened. I can assure all concerned that (a) the re-learning curve is quite a bit steeper than it would be if the code were a bit larger and more imperative; but (b) the amount of code I will have to write to re-use the system is miniscule. This is yet another example that leads me to believe that higher levels of abstraction are best for things that are used a lot. DRY as a guiding principle is a good start, but I usually use DRYTMT — Don’t repeat yourself too many times. Unless the required commonality is really, really obvious up-front, I will often use cut-and-paste programming to build an initial system, then write some general functions to cut down on the amount of code.
@Jakub Narebski
> parallel extension (?) to Haskell
“functional programming is of paramount importance for parallelism”, Bob Harper (Carnegie-Melon professor), Parallelism is not concurrency.
He also claims a lazy language such as Haskell is incapable of I/O complexity reasoning, Yet Another Reason Not To Be Lazy Or Imperative. I suspect this is true because the timing and order of execution is indeterminate with lazy evaluation operational semantics. There may be solutions he is not aware of.
@Jessica Boxer
The problem is when that loop is just one of a significant plurality of details in the actual semantics I am trying to express. Then I come back to read my code 6 months later, and I have to build up the model of the semantics in my head from the bottom-up, instead of from the top-down. So instead of spending maybe a few minutes remembering what the code is supposed to do at the high-level semantics, I have to wade through hours of details to reload all those unnecessary, extraneous details in my head.
Also to the extent that details are local state, they explode the complexity of both boilerplate and practicality of reusing code modularly.
I hope the distinction between bottom-up versus top-down reasoning is more useful to you now.
@Greg
If you prefer a simple loop over Patrick’s Python one-liner, perhaps you are behind the Python mainstream of whom many apparently (?) have at least graduated now to pseudo-FP programming employing generators for more declarative expression of their semantics.
IMO it would be easier to go direct to functional programming, than to migrate the long path through generators with local state. But the problem IMO is there is no functional programming language suitable for the mainstream, thus no set of lessons that go with it, to make it so simple as my HTML Introduction.
I am confident I can at least teach you how to fold that simply. As for the higher-level abstractions, I am more skeptical. Problem is that if I can’t turn you on the higher-level abstractions benefit, then you may decide as Jessica apparently has, that there isn’t enough benefit to the fold.
@Patrick Maupin
Python is obviously a very effective language for what it is used for. I am speaking about if it is possible to improve and what might be the use cases to motivate such. I am thinking perhaps wide-scale crowd source modularity. The “single maintainer” is the antithesis of what I am dreaming for. I am thinking you write a module and it is used by hundreds of open source projects, and is maintained orthogonally. I think there is no realistic way this can happen without compile-time typing on the API.
The dream is that if I don’t like some feature in some open source project, I can more easily swap in modular functionality– cool bait.
That is (one reason) why I am I guess declining your 1000 LOC test-bed for now. That might reveal some insight, yet I think more insight will come from trying to achieve the wide-scale modularity and my time is limited.
Ah yes, we are in agreement then. And I want to try to find ways to ease that re-learning of the abstractions that aid the declarative higher-level semantics.
Your Python example is FP to the extent that it calls an orthogonal generator. It retains many imperative concepts. Both yours and my code (and all FP code) can be expanded out to low-level code. If your imperative code would be looping, so will the FP code when expanded. For example, if there is a fold on an array, then the FP code is going to be walking the elements of the list in a loop. The only difference is that instead of placing the code inside the loop, it places it inside a function and calls the function from inside the loop. That is why I said, you can set a breakpoint in the callback function, and it will break on iteration of the loop. It is not different than debugging a loop.
The difference is the expression of semantics is more higher-level and more reusably composable (i.e. top-down) in FP.
In my example here, one can set a breakpoint on the `cnt` (or previously `count`) function and debug through all the steps of the computation of the tuple on each iteration of the loop.
Agreed, and one key weakness we discovered in this discussion is that understanding requires knowing the inferred types.
My new idea generated by this discussion, is to investigate automatically transforming the higher-abstraction to the imperative one you prefer. The programmer could toggle back and forth. This should be an effective learning tool also. Eventually they find themselves driving FP unconsciously. When they have a mental hiccup, they toggle back for that “oh yeah”.
I started in BSEE with math minor (I know flops too), then I wrote WordUP (google explains it was one of the first commercial WYSIWYG word processor with multi-columns, master pages, dictionary and thesaurus, etc) in 1986 in 68000 assembler, then I started migrating it to C just before I abandoned it. The sin of abandoning it is one of the strong emotional reasons I identify with open source. After writing and maintaining that much low-level code and then the fresh air of shifting to a higher-level semantics, I have a giant wart in my brain to remind to continually seek higher-level paradigms sooner.
But I come from big projects world, e.g. briefly one of the 3 or 4 original coders on what is now Corel Painter. And I was lowest on the totem pole, so got assigned to do the messy debugging and also for the low level Mac API to Windows API layer.
I get the impression Python let’s me think in imperative model while mixing some higher-level pseudo-FP declarative style by employing generators. I also expect it will allow me to dig down to the fugly imperative details where I want to. But my mental mode shifted circa 2008, and I now hate to code in non-FP. Actually my coding style was drifting towards FP inherently before that, and I didn’t realize that is what I was doing. I was fighting with the syntax of the compiler. Another significant shift in my mental mode I think began when I first discovered the closure and anonymous function (in JavaScript). The internet drastically accelerated my learning in all areas of study.
Back on the topic of categorizing troll-like behaviors.
@JustSaying
I know Eric prefers “open source” over the acronym.
I had to use this yesterday in another forum. This Dunning-Kruger effect also applies to people who are expert in a related field (e.g. chemistry), but overestimate their omniscience in the related field (e.g. biology) and try to intellectually brow beat the crowd sourcing with assumptions that violate rational skepticism.
On the technological unemployment front, the open source manufacturing with the 3D printed plastic car has arrived and it is stronger than steel.
@Patrick Mauphin
Which I am thinking to wrap in syntactical sugar as follows.
Equivalent to.
Amazing?
It is the antithesis of maintenance. You never have to rewrite the function `f` again for different types that can contain an (a.k.a. parametrized on) `Int`.
Note to be more general, I would instead subtype a class that applies to all types that can be summed (and Haskell can not do this generally).
The youth are ripe to turn to open source programming.
Open source use is higher in Europe (c.f. my post in the disciples blog about IE share by country), yet the youth unemployment is 2 – 3 times higher than the adult population. For example, Italy 39% vs. 11%, Spain 56% vs. 26%, Portugal 39% vs. 18%, and Greece 60% vs. 27%.
The boomers are using their political control to maintain their over-indebted lifestyles at the expense of the youth. I hear from a friend that some factories in Germany are now paying 5 euros per hour in order to maintain new sources of employment. The industrial age is dying.
The youth need to break free somehow. Some leave to former colonies. As another outlet, they could learn programming (internationally marketable skill via the internet) and bypass the economic control of their elders.
There are some gun rights hackers among the brethren, so 3D Printing Downloads of Semi-Automatic Weapons Hit The Internet. A photo of the AR-15. But are machine guns for self-defense? (I don’t know, I only learned to shoot a BB and pellet gun)
@JustSaying –
Eric blogged about this (with quite a bit of discussion) last August; the link you provided is just their latest, best iteration. What’s gotten fabbed is the “lower receiver”, which is the (US-)federally-serial-numbered component that `defines´ the gun – all the rest of the parts can be bought without restriction over the ‘Net. The most critical issued noted at the time (and still outstanding, AFAIK) is the manufacture of the high-pressure components (barrel and breech – shooters, help me out with the right terminology here!), which still can’t be printed in plastic.
If you are near SE Michigan the last weekend of April, come to Penguicon and its Geeks with Guns – we can talk and shoot.
Your CSS is broken:
pre { ... font: "Courier 10 Pitch", Courier, monospace;Should be font-family, not font.
@ESR
>And nothing here has anything to do with what you call the Tao of Unix
I’m the one who brought ToU into the discussion. And it damned well does have something to do with it. Your original post said:
>This matters more in git-land because so many of the tools just show first summary lines
The reason why the ToU matters is that, contrary to what some people keep saying, a comment in a programming language is not necessarily just a human-to-human communication. Everything we write in a document has the potential to be parsed by a machine at some point.
An extreme example of people not getting this: At my previous job, we had a device known as a “Digital Sender”. It was like a fax machine, but instead of transmitting via phone line, it scanned documents into PDF and emailed them. Some of my co-workers would take an MS Word document, print it out on a laser printer, then scan it back into the Digital Sender to transmit it to someone. In doing so, they converted the document from a searchable, parseable format into a bitmap that could not be searched or parsed without first passing it through error-prone OCR, and also takes up far more space on storage media. But to the humans who read the documents, none of that mattered, so the people who did this never saw how ridiculous the practice was.
As long as the mentality is “I’m writing for humans to read, and they can figure out what I mean”, there is no hope of writing tools to automate the task. And it also never occurs to these people that sometimes not even the humans can figure it out, because what they’ve written has more than one possible interpretation.
@The Monster:
While I feel your pain, I am actually gratified to see that normal people are starting to understand that not everybody wants to receive MS word documents. In a two-steps-forward, one-step-back sort of fashion, this is probably actually lurching in the right direction.
@Patrick Maupin
BTW, this is another feature of functional style languages that I dislike — the tuple. Don’t get me wrong, I understand that you can sugar the syntax better, but these anonymous tuples are part of the DNA of FP programmers, and they are just an ugly residue of the math paragdim.
Again in C# (and no doubt other languages) you can get the benefits of anonymous types without the disadvantages. In the above case you would have:
return new { Length = source.Length, List = source }Of course this doesn’t actually make sense, since lists already have a length property, but for illustration purposes… This returns an unnamed type, but it is a strong type and the members are accessible by name. It is all done statically at compile time so none of your ugly JavaScript unfindable bugs.
Giving names to things makes code much clearer. For example C# has a type called a Dictionary that is a red black tree for sorted storage. Obviously it is a generic type, and as a consequence the name value pair is called Key and Value. I really dislike this. So I name the parts, viz:
Dictionary contactList = Database.ReadContacts(); using(var writer = new StreamWriter("Contact.txt")) { foreach(var contactRecord in contactList) { var name = contactRecord.Key; var contact = contactRecord.Value; // Note 1 writer.WriteLine("Name {0}", name); writer.WriteLine("Phone Number {0}", contact.PhoneNumber"); writer.WriteLine("Email {0}", contact.Email); } }Details aside, the point is at note one I give the sub parts of the key value pair actual names, so that I don’t have to constantly refer to contactRecord.Value.PhoneNumber etc. or worse contactRecord.Key and add the extra cognitive load on the code reader to remember that the name is also the key.
This makes the code much more clear at the expense of an extra line. But lines are cheap, programmer attention is not. And of course there is no runtime penalty because the extra declaration is eliminated by alias folding optimizations.
Tuples are the antithesis of this philosophy.
@Jessica Boxer:
I agree that chasing down unnamed things very far can be problematic. As do some of the maintainers of Python. Hence the named tuple:
http://docs.python.org/dev/library/collections#collections.namedtuple
@Jessica Boxer
I agree with you that naming is semantically important. A point of the tuple is reuse of functions without writing boilerplate. A function that inputs a Tuple[Int,Int] will operate on all such instances regardless of what meaning the elements have in your local semantics. An anonymous class can’t do this. The named tuple is what you need.
@John D. Bell
The invite is unexpected and appreciated. Maybe the next time I am state-side.
@JustSaying
> The named tuple is what you need.
Culture counts as much as syntax. It is notable, don’t you think, that both you and Patrick gave your examples without naming the fields. That is the gestalt of these kind of functional programming approaches.
It is notable that C# comes with a very extensive set of coding standards and requirements, and a very powerful set of static analysis tools for validating this, as well ad finding general bad practices that might be semantically valid, but are probably wrong. These tools enforce a culture that strongly encourages quality naming conventions and the use of the various tools to bring out the meaning of the code.
> So instead of spending maybe a few minutes remembering what the code is supposed to do at the high-level semantics, I have to wade through hours of details to reload all those unnecessary, extraneous details in my head.
But you are assuming your conclusion. Trying to get your head around these complex declarative statements is very difficult indeed. The details arein both pieces of code, and they need to be groked, but I don’t think you have demonstrated that your code is somehow easier to understand than the procedural equivalent. On the contrary, as I say, I think it is a contender for an obfuscated code contest.
I think though that in some spheres declarative programming is quite useful. A good example I have been working with a little is AngularJS, a javascript overlay on HTML that allows declarative statements about the visuals of the code. I think it fits well with the HTML model, and I’d recommend all people who do web dev to have a look. It is a google project.
I’d give an example here, but the likelihood of the HTML surviving the WordPress filter isn’t high!
@Patrick Mauphin
> I am actually gratified to see that normal people are starting to understand that not everybody wants to receive MS word documents.
I don’t think that’s what is actually happening there. Those people probably just replaced “print, then fax” with “print, then scan” without really knowing what they’re doing (and, very likely, complaining that fax was way better)
@Jessica:
Umm, I think I was just following JustSaying’s example. If it had been my example, it wouldn’t have had the len() as one of the things in the tuple, ’cause that’s redundant in Python. Also, it probably wouldn’t have had a variable named ‘tuple’, cause that’s a special identifier.
But yeah, sometimes I don’t name things that other people might name. And sometimes people name things that don’t need to be named and it pisses me off. Like on a schematic. If I see a name, I assume the signal is probably going somewhere else besides where I see it…
@Jessica Boxer
Let me try to convince you first with a very clear example, then I will come back to address your valid criticism.
A function.
function Doolittle(a, b, c) { a + (b * c) }If the [ ] is an array that implements Applicative, then the result is.
Now let’s compare the imperative boilerplate to accomplish the same.
And that imperative code has to written every time we have a different case. We could generalize it with loops (left as an exercise) for the specific case of arrays and functions that input 3 parameters, but it would still be fugly verbose hiding the semantics. And we would still need a special case of those loops for each function arity and different type of Applicative other than arrays.
If you think that example is not that useful, then let us consider the pervasive Nullable.
Note Doolittle’s inputs are non-nullable. But what if we want to compose with values that are nullable? Well if Nullable implements Applicative, then we can do declarative magic.
The semantics could not be clearer.
Doolittle has been automagically converted by its inputs from a function that inputs and returns integers, to a function that inputs and returns nullable integers.
Now for the equivalent boilerplate imperative code which buries the semantics in irrelevant housekeeping about nullable.
function(a, b, c) { if(a != null && b != null && c != null) return Doolittle(a, b, c) else return null }Now returning to your valid criticism, which is that the following is not as obvious as the above examples.
Let’s conjure up some clearer syntactical sugar that can be converted by the compiler to/from the above.
Is that clearer?
I missed a closing < tag which is why the “Declarative magic courtesy of the Applicative class.” is rendered as a code block.
I would prefer.
The keywords above are `forall`, `in`, and `begin`. The syntax difference from Python by placing the [ ] only around the member expression `x.str` so there is the generality to create a more complex expression within the `forall` loop. That expression can be initialized with `begin`.
I think the naming of the list should not use = because it is ambiguous with an assignment expression.
The tuple members are named `count` and `list`. The `count:= count+1` is a shorthand for `count: count = count + 1`.
@JustSaying:
You must do completely different things than I do. I hardly ever want to return a null from a function based on a null input, and I hardly ever need those sorts of combinations of lists (although those are quite easy with a minimum of boilerplate in Python).
And it may be unfamiliarity, but I don’t find even the latest syntax all the comforting. One of the things I like about Python is that (with very few exceptions like elif, all the keywords are, you know, words. Not combinations like ‘forall’.
Another thing is that the Python syntax is usually pretty unambiguous. On your example, I have no idea how I would add another initializer besides count if I needed it, and only some sort of vague idea that declaring something to be a list and using ‘:’ instead of ‘:=’ concatenates all the lists together into a single list.
@Patrick Maupin
I assume (based on what you wrote earlier) because you don’t think in terms of typing, as it is not required by your use cases as you don’t write fine-granularity, wide-scale composable APIs to be used by independent compilation and testing by the consumers of your API– in short you don’t do finely grained S-modularity.
If you try to write code without `throw` and `catch`, then you will find that you must convert these exception semantics to values, e.g. thread `null` through your code. As I showed, the ability to declare functions that don’t operate on `null`, yet which can be lifted to do so without any syntax, is essential to avoid the obfuscation of boilerplate.
Thrown exceptions (i.e. `throw` and `catch`) kill concurrency, are not composable and create spaghetti code, because in terms of wide-scale composition (as is the case with dynamic typing) they put the program in a nondeterministic state (w.r.t. to compile-time checked semantics a.k.a. invariants) because the invariants are not checked at compile-time. In short, they violate the compile-time checked API.
Checked exceptions are a widely regarded failure in Java because they just reveal the impedance mismatch between trying make some invariants orthogonal to the holistic typing system (which I assert that analogously Mozilla’s new language Rust employing Typestate will also suffer). Unchecked exceptions hide this mismatch in bugs that are not (yet) encountered. The degree to which this is a non-problem, is relative to the scale of S-modularity required by the use case.
Can you show how easy without boilerplate in Python?
Also perhaps you miss the point that this lifting of functions that operate on normal types applies to any class that implements an Applicative– not just lists. It is a generalized compositional paradigm for eliminating boilerplate.
What ever people prefer `for` or `forall` is what I will choose. I thought the latter gives a more clear semantics, because the noun `x in [ … ]` is plural (there can be multiple `x` are in a list).
The colon-equals := was just a shorthand to avoid writing `count: count = count + 1`. The colon : is optional and is used to give a name to the field of the tuple (the naming should always be optional). I was addressing Jessica’s concern and your reply to use named tuples. Follows the equivalent code without named tuple fields.
Typo, let me put that last code example in pre tags not blockquotes.
@JustSaying:
It’s probably finer-grained than you think, but whatever.
Yeah, you have to do a similar thing if you write C to interact with Python.
Umm, no. I mean, that might be true of CAUGHT exceptions, but if I’m just throwing an exception to say “this is wrong; kill the program” then no. I’m not doing 5-nines uptime here.
But I get the point that you are propagating nulls automagically, essentially for exception handling, but your exception handling is somehow better than the way that Python propagates exceptions automagically. I suppose I can believe this, but obviously, with Python as-is, there is no reason to propagate nulls if they are essentially the same as exceptions. Yes, it’s true that Python is not really a language that does things at compile-time. OTOH, if you _are_ doing as much invariant checking as you claim at compile time, why should an exception ever happen?
Well, there’s a bit more boilerplate than what you showed for your language, but it’s not nearly as much boilerplate as what you show for “the competition”:
Also, if you were doing it a lot, you could write a function (similar to map) to support this. I’m sure this could be done better, but here is a simple example:
No, I got that. Did you get where I said I hardly ever need to do something like this? :-) I also am not sure how the semantics would work for something other than an iterable — would it simply extract random integers from your data structure and deal with them in random order? I have made the point before, and will make it again, that iterables really are easy to reason about and work with — my first example worked first time, typing it into the interpreter, and the second example — well, I left out a ‘*’.
It’s still not completely clear to me how [x.str] knows that each iteration is adding a component to a list, as opposed to creating a new list for the iteration.
@JustSaying
> Is that clearer?
Not at all. You’re points are valid if you are comparing against C, but modern programming languages implement generics, interface, operator overloading semantics and various other things that make all this pretty much the same.
So your first example would be something like this:
Type DoSomething(Type a, Type b, Type c) { return a + (b * c); }Which works just fine in all cases, including arrays, nullable ints and so forth, It is pretty much character for character exactly the same, except that you must specify the types (or generic types) explicitly, and as I have said before, I think that is a superior approach.
Of course it is also a toy example, and these sorts of things rarely extrapolate well to significant tasks.
And FWIW, you can’t possibly thing that:
Is super clear. Honestly, I have read it half a dozen times and am not really sure what it does.
Sorry, WordPress ate my brackets. The function above requires a generic type declaration, it should be:
@Patrick Maupin
Apology, I was struggling with how to characterize the different use case, which would motivate a different set of priorities, not as a personal swipe at your current priorities. We bring more intense gear to climb Mt. Everest on any day, than to hike up Mt. Baker in the summer.
However, I have not shown any evidence (other than reasoned conjecture) of a large scale implementation to prove that these concepts are needed.
Agreed. And the higher in the hierarchy of nested functions (and thus semantics) that occurs, then I posit the greater chance of cleaning up some first and exiting with least damage. So in theory pushing checks out to the callers by typing the values on the API can push the exceptions out to the callers.
There will be cases (I have run into some already in my designs) where you can’t get perfect synchronicity with all the invariants at compile-time, so you cast and throw an exception if the runtime value is wrong. At least you’ve pushed that up the tree as much as possible.
Before the category theory constructs (Functor, Applicative, Monad), these corner cases were more frequent. Hughes’ Arrows might be another way to get more flexibility.
Sweet! I am liking Python syntax more.
But that syntax above doesn’t help you with the Nullable example, because that syntax doesn’t have all of Applicative.apply’s semantics. Iterable isn’t general enough.
Try to do this one in Python.
Nullable is an example. It contains either a `null` or one non-null value. The Nullables are passed into `Dolittle` (using the Applicative.apply method implemented by a Nullable). The input to the `apply` method is an instance of the implementing type (e.g. Nullable) containing the value of a function. See the complete explanation.
Basically any parametrized (generic) type Sub[T] (a.k.a. Sub<A>) that can implement the transformation (T -> A) -> Sub[A] (a.k.a. Sub<A> function( T t, A a )) will be able to implement an Applicative. So Nullable.apply qualified because if the value is null for the member variable that could hold an instance of T, then Sub[null] is returned regardless what the type A is.
* note I shorted count = count + 1 to count+=1
I am not grokking the ambiguity you claim. I see the expression that precedes `forall x` must be computed for each iteration. The [x.str] says collect the computations in a list. The count+=1 says accumulate the computations in the `count`. If the expression was instead (count+=1, x.str) it would instead discard all except the last computation of `x.str`.
@Jessica Boxer
That is the antithesis of modular programming.
Because you conflate the implementation of math operators (which should only apply to ordinal types) with completely unrelated types. This paradigm of programming forces all unrelated types in the universe to implement every possible operation in the universe.
That is concrete boots and combinatorial explosion programming.
For naive readers, please understand that the code above requires every Type (in the universe) that can be input to the DoSomething function, to implement the plus + and multiplication * operators. For example, the Nullable.plus method would return null if either of the operands are null. And since we want our paradigm to work with any possible function in the universe, then the requirement extends that every Type has to implement every possible operator and method in the universe.
That clearly computes `do(x)` for every x in [1, 2, 3].
Now replace `do(x)` with the expression I had written (count+=1, [x.str]) which is a tuple of count+=1 and [x.str].
That tuple expression is a bit more difficult to grok. See my prior reply to Patrick.
Obviously like any variable, the `count` must be initialized unless it was initialized up code (or has a default constructor).
@JustSaying:
“This one” doesn’t make any sense in Python, because types of variables are not declared. It also doesn’t buy you that much because there are exceptions. Nonetheless, if I wanted some sort of similar behavior, I could create a function decorator that I could use for any function that I wanted this sort of behavior on (much simpler than the boilerplate you showed earlier), or I could create a class that had the desired semantics for the operations. The example with the function decorator would be something like this:
I realize this is still more explicit than your solution, but honestly your solution doesn’t buy much in Python, because there are exceptions.
What do I do if I want to only keep the last computation of x.str, but want to return it as a list with a single element?
What’s the point of using the <pre> tag if it doesn’t give a fixed-width font? Sheesh.
@Jessica Boxer
For Functor, Applicative, and Monad semantics, you either need type classes[1] or higher-kinded generics[2]. AFAIK, only ML, Haskell, and Scala have these.
C.f. my explanation.
[1] section Now What?, Monads are not Metaphors. Also c.f. section 7.2 Encoding Haskell’s type classes with implicits of [2]
[2] Generics of a Higher Kind, by Adriaan Moors, Frank Piessens, K.U.Leuven, and Martin Odersky
@JustSaying
> Because you conflate the implementation of math operators (which should only apply to ordinal types) with completely unrelated types.
Why should math operators only apply to ordinal types? What about matrix math? What about complex numbers? What about algebraic equations? What about data structures such as lists or dictionaries?
> This paradigm of programming forces all unrelated types in the universe to implement every possible operation in the universe.
No it doesn’t, it requires you to explicitly define how decorators on a type (such as nullabilty or iterability) impact their underlying operations, but explicit is good, right?
> That is concrete boots and combinatorial explosion programming.
Sorry, that is just plain wrong. A good generic system means that you specify the deltas of functionality explicitly, not a combinatorial explosion, as you claim. I think explicit is good.
> For naive readers, please understand that the code above requires every Type (in the universe) that can be input to the DoSomething function, to implement the plus + and multiplication * operators.
And that panoply of definitions are collapsed to one definition via the use of generics.
> For example, the Nullable.plus method would return null if either of the operands are null.
If that is what you want. Of course to me it makes absolutely no sense to add nullable types anyway. “4 + null” is not ‘null’, it is ‘error’. But of course if you really want to do it that way you can. To me though it is an abuse of operator overloading (which is what you are doing in your FP too, even if you don’t call it that.)
> do(x) forall x in [1, 2, 3]
etc. But don’t you just prove my point, you need to decompose it to explain what it does, which, to me, seems to belie your claim that FP is so darned prima facae obvious. In my experience it is frequently dense and obtuse and turgid. These are not adjectives that I would like applied to my code. And that density and obtuseness is almost a point of pride on the part of coders, which is both a syntactic problem and a memetic problem.
I’m not a fan of Python, but one thing I will say is that the community has generated some very positive memes, DRY, batteries included etc etc., TDD, and that makes Python a much better language for it.
@Patrick Maupin
Unfortunately that has a similar combinatorial explosion problem as Jessica’s proposal, because you must add a @propagate… to every function in the universe for every subtype of Applicative in the universe.
In general case of subtypes of Applicative, there is no solution in Python and there can’t be, because it doesn’t have typing, type classes, nor higher-kinded generics.
I don’t anticipate that exceptions are a suitable substitute for typing in the wide-scale reuse of modules, because exceptions don’t compose,
You know that you know the answer already, because there is only one possibility.
You can’t unless we make a special case syntax for that, e.g. [[x.str]]. Compared to current Python, that is the trade-off to obtain a more general syntax that enables the list to reside in an expression that is computed simultaneously in the same iteration. You can copy it into another tuple on the next line and place into a list.
@JustSaying:
> You know that you know the answer already, because there is only one possibility.
But that’s the ambiguity. Apparently, sometimes [] means “build a list right now” and sometimes it means “build a list across multiple function invocations”.
> You can copy it into another tuple on the next line and place into a list.
It’s not about the capability. It really is about the mental overload of the same thing having two different operations. Python (sort of) has this issue with parentheses because () means empty tuple, so you might think (foo) would be a tuple, but of course it’s not. But this Python wart is not context-sensitive, and the wart you are proposing for list creation very much is.
I’ve said this before, but if complete syntax generality was the main thing, lisp would have won. Having said that, your syntax so far doesn’t look all that general / orthogonal to me yet…
@Jessica Boxer
Correct. I was merely differentiating types that should have a plus + method from types that have nothing to do with plus. You knew that ;)
Incorrect, it is not limited to a few “decorator” types (whatever that is), because Nullable and Iterable are not the only possible subtypes of an Applicative. There are unlimited subtypes of Applicative in theory. My original points stands.
Every Type input into DoSomething will need to implement a .plus method, unless (as you claim below) you don’t want to be able to input those Types to DoSomething (which is a double-speak strawman form of logic).
You can certainly program the Applicative.apply for Nullable to return error instead of null (or None). I probably agree. But still you need the Applicative in order to automagically thread that error result through the program instead of throwing an exception (thrown exceptions being uncomposable).
You are arguing that there are no useful Applicative subtypes, or at least that there are not a panoply of them. Which I am reasonably sure is a wrong assumption (see above).
Arguing that Applicative has no applicability to programming, is a different point than arguing that there can’t be combinatorial explosion if Applicative is useful. My latter point remains valid in any case.
Now you’ve moved the goal posts to the subjective.
If that snippet of code above isn’t obvious, then I subjectively conclude you have no hope of ever writing concise code.
AFAIK C# and Python culture are not close. Apparently Pythonistas associate concise syntax with readability. And I think we will win the most eyeballs once we combine with optional static typing (as Microsoft is dying as mobile disrupts the PC). AFAIK, Python is already winning a significant share of the new scripting and smaller programs.
And for sure DRY (and thus arguably TDD) requires that sort of concise FP-like syntax above.
AFAICS, Pythonistas are migrating conceptually towards where I am conceptually with FP-like constructs, away from the many-ways-to-do-same-thing which is C#. I read that is one of Guido’s core design concepts.
@Patrick Maupin
Correct, so the latter should use a different syntax, e.g. |[x.str]|. Problem solved.
@JustSaying
> You knew that ;)
Not at all, in fact in your original example I was a little confounded when you added two lists and got a list of the sums rather than a concatenated list. This point is further fleshed out below.
> Incorrect, it is not limited to a few “decorator” types (whatever that is), because Nullable and Iterable are not the only possible subtypes of an Applicative.
you have to specify what plus means in the context of an enumerable type, or a nullable type. This is specifying what it means in that context, they are meta types in a sense (though that is not C# terminology.) Somewhere you have to define what plus means in that concept, just one time though. Regardless of what plus means for the elements, what does plus mean for a list of these elements? Regardless of what plus means for the inner type, what does plus mean for nullable? The meaning of the inner types is defined once, elsewhere, as it is in every language.
This is illustrated in my confusion as to what plus meant for lists in your original example, and further what plus means in the context of nullable, which, as I say doesn’t make much sense to me. One does not need to redefine it for Nullable Nullable Nullable<Hashset> just for the meta type Nullable.
> Every Type input into DoSomething will need to implement a .plus method,
Somewhere someone has to define what plus means for every type whether it is built into the language, the library or custom code. That is just as true of FP as of any language. So I don’t know what your point here is.
> to automagically thread that error result
It isn’t magic, it is defined in code. There is no reason why it can’t be defined in code in other languages. The fact that it isn’t is more a reflection of the fact that it is a bad idea in the first place than that it is a missing capability. However, part of the reason for this is that this kind of doublethink type is more necessary in an fp where there is less control over the flow of control.
> You are arguing that there are no useful Applicative subtypes
Not really, I am arguing that the examples you give are bogus. I’d be happy to hear better examples.
> Now you’ve moved the goal posts to the subjective.
I don’t think there is an easily obtainable metric for clarity, it is more a set of heuristics. Density is often an obstacle to clarity (though verbosity is too.)
> If that snippet of code above isn’t obvious, then I subjectively conclude you have no hope of ever writing concise code.
And back to the beginning, concision is not an attractive attribute. Clarity is the goal. Where concision and clarity coincide then all well and good. I assure you though, I can write obfuscated code in any language. Some languages though encourage it more than others.
> AFAIK, Python is already winning a significant share of the new scripting and smaller programs.
Actually I think JavaScript is our destiny. God help us.
> And for sure DRY (and thus arguably TDD) requires that sort of concise FP-like syntax above.
I think exactly the opposite is true. DRY needs a robust mechanism for abstraction. Reusability depends on people being able to understand what they are reusing. Opacity, like the above is the opposite of that goal.
@JustSaying:
Careful. Paul Graham and others claim that Python is attempting to be another lisp. Guido is good at striking a balance. Lisp and FP are both powerful, but both have their shortcomings, and Python is for real work.
If [[]] doesn’t mean a list within a list in all contexts, I want nothing to do with your language. Seriously. One of the best things about Python is also one of the most frustrating during the process. New features are added with great deliberation, and ambiguity is avoided at all cost.
@JustSaying:
Sorry, I missed that you wrote |[]| instead of [[]]. So it’s not an ambiguity problem per se, but it still might be a readability issue for old farts like me…
For creating and subsequently adding to a data structure, this would be more, uh, vibrant:
>[]<
You could do other data structures like a map: >{}<
@Jessica Boxer
This is because you did not factor in the type of the inputs to Doolittle, which are integers. Thus, Doolittle can never add lists, i.e. Doolittle is still always just adding integers even though the Applicative semantics have provided the hidden boilerplate to add lists element-wise.
That is the whole point of modularity– functions only have to do a local consistent semantic and can be composed to achieve higher-level semantics.
Which is the antithesis of modularity. Modularity is when a function does a consistent, local semantics and can be reused in compositions that create higher-level semantics. And the reason it is not modularity, is because of the combinatorial explosion.
Incorrect. Doolittle will forever only operate on integers.
I asserted that thrown exceptions are not composable, which I think is well known. And AFAIK there is no possible way to handle exceptional conditions without thrown exceptions, if you don’t weave* them into values, e.g. Nullable, which will infect every function of your program if you don’t have the Applicative to weave them orthogonally.
* ‘weave’ is a less ambiguous word in our compsci context than my former use of ‘thread’
I suppose the most objective metric will be programming language usage share.
var list = [1, 2, 3] for(var i = 0; i < list.length; i++) { var x = list[i] do(x) }The denotational semantics are much more clear in the first one. The operational semantics are explicit in the latter. At the extremes, the former is obfuscated operational semantics and the latter is machine code.
Our goal is to hit the sweet spot. (LOL, your comment about hitting the g-spot in another blog sprang into mind)
Try to compose several smaller JavaScript modules into a large application.
JavasScript has a very low programmer share.
Agreed with the objective taxonomy. I subjectively don’t agree that the forall one liner above is opaque, but to be objective we need to measure programmer share in the marketplace. And that is why I wanted to listen to all who commented about this here.
Caveat in my prior JavaScript example, `do` is a keyword, so should be renamed.
@Patrick Maupin
I am differentiating the obfuscation of Lisp from Python’s move up in denotational semantics clarity hiding some operational semantics, per my prior reply to Jessica.
@Jessica Boxer
In summary, don’t conflate generics with modularity.
@JustSaying:
This was not clear at all from the given code snippet. What makes it true?
Seems you may be trading one kind of generality for another. My Python version of Doolittle will take anything for b and c that can be multiplied together, and anything for a that can be added to the result of that:
>>> def Doolittle(a, b, c): ... return a + b * c ... >>> Doolittle('Hi', 3, '!') 'Hi!!!' >>> >>> Doolittle(1, 4.0, 7) 29.0 >>> >>> Doolittle([], [0], 10) [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] >>>Sure, but this can be done automagically as a special case. Yes, generality is nice, but if this is your only use case for this nullable, then it isn’t necessarily a good reason for your extra functionality.
I agree completely. And this is easy to learn and teach. I submit that this is partly because it matches well with the given explicit loop example. In other words, it is shorthand that is easily understood by and translatable by a human into C or even assembly language.
Absolutely! But IMO, Python is squarely in the middle of this niche, and what I have seen so far of your proposal isn’t, at least for general purpose programming. Creating a more readable erlang, however, might be useful…
Personally, I don’t view the given for statement as hiding anything. I have taught people various Python concepts, and they get it almost immediately. I don’t think that will be true of your FP language, but I would love to be proven wrong.
Imperative programming is easy to learn and to do. You are right in that, e.g. C programming doesn’t scale very well. But there are two ways to do abstractions. You can have abstractions that make it easier to do things the programmer would have done anyway imperatively, or you can have abstractions that require the programmer to think completely differently. Python has some of both, while your proposed language seems to rely heavily on the latter kind of abstraction.
@Patrick Maupin
We can certainly write Doolittle as a generic function that accepts any type that has the plus and multiply operations. My point remains to Jessica, that Doolittle won’t operate on lists or any other type that doesn’t have plus and multiply operations. In any language, we will need the Applicative to apply these other orthogonal types to Doolittle. Jessica’s proposal of adding plus and multiply operations to every type that wants to use Doolittle, is the antithesis of modularity. Imagine for every generic function in the universe, we have to add all the operations they collectively use to every type in the universe– a combinatorial explosion nightmare.
No it can’t be done as some special case in the language, because Nullable isn’t the only exceptional type, and they won’t all have the same semantics. For example, the type Nonzero, for pushing divide-by-zero exceptions out to the caller as compile-time check.
Python can never do this, as it is isn’t compile-time type checked.
Your assumption remains that programmers all need to think imperatively, as if they can’t be a native speaker in functional denotational semantics. IMO, that is analogous to assuming Chinese must translate to English, before they can understand Chinese. I think the imperative preference is not genetic to the human race, but rather learned.
The usage share of programming languages is not conclusive, although Python is rising fast.
I didn’t know that I made a proposal.
I have been collecting some disjointed ideas, not providing a coherent final specification for a language.
It hides the verbose extraneous operational details of the loop version of the code. Your eyes don’t get clogged up with all that noise. Whether you choose to translate it in your mind to a loop is irrelevant to my point that it is visually more concise. And with 100s or 1000s of lines in a program, visual conciseness is a benefit, if there is not trade-off for doing so.
Were these people who knew nothing about programming or people who had prior experience with imperative languages?
I think people who have been taught imperative languages will try to learn FP by translating it back to what they know, and so yes they will find it a struggle at first.
I think people who learn FP from the start (and are never taught a loop), will hate imperative languages as much as imperative language lovers hate FP.
I think there is nothing fundamental that makes imperative programming easier-to-learn. I think it is an accident of history.
The same abstraction (the exact same one liner you liked above), can exist in a FP language and be implemented functionally.
The additional abstractions I showed added functionality that you can not do in Python nor C#.
There are not two abstractions. Higher-kinded typed functional programming can do the same simple abstractions that imperative programming can do, and it can also do much more complex abstractions that imperative programming can’t do.
I am worried about the complexity of compile-time typing. It adds another Turing complete layer of semantics. That is why I am thinking about syntax that makes it mostly unnecessary to the know the types, such as the example I provided.
Which is equivalent to the loop.
var a = [1, 2, 3] var list = [ ] var cnt = 0 for(var i = 0; i < a.length; i++) { list.append(a[i].str) cnt+=1 } return (cnt, list)And equivalent to the fold.
And equivalent to the traverse.
@JustSaying:
But as soon as you have different kinds of things that support multiply and add, knowing what Applicative is going to do in a certain situation becomes difficult (for the human), no?
It doesn’t seem to be in practice. (Python types (classes) define the operations they support.)
In Python, this is still handled by pushing a null back to the caller. The type of the exception is recorded in a per-thread global. I know you don’t like this, but for many real-world purposes, it works great.
Not “need to.” More “already know how to” as taught by life. The canonical example is “remember — rape, pillage, plunder, then burn.” Life is stateful. I am sure the nascent human brain is capable of more modes of thought, but by the time people start programming, this one is already developed.
Human variation being what it is, I am sure there are people who are hardwired for an FP preference. Most of those probably become managers.
And learned early.
Both.
I think people who are taught any language will translate it back to what they know. This is quite easy for many people for imperative languages.
No doubt. It has already been shown that, for novices, the first language is easier to learn than the second. BTW, there is already some pretty good research on learning to program, and lots of people just can’t.
You can find lots of papers purporting to show that object oriented/functional/imperative programming is the best first year class, but most of those are quite subjective, lacking the data given that shows that some people are going to get it, and some aren’t.
So then, maybe, it’s time to think about the second year programming class and beyond, or even (as the authors of that last paper hinted) we should eschew first-year programming classes altogether, and have skill-building classes for people who can already program.
To the extent that (as that last paper I referenced shows) you need to be able to treat programs as meaningless cruft, you are most likely right. To the extent that people need to be able to reason about why their program didn’t work right, and correlate that with how the computer treats the program, I honestly don’t think you are right. Debugging in most FP languages, and debugging in most OO languages, is difficult. Debugging in Python is usually dead simple, and usually doesn’t even have to be done.
Yeah but my one-liner can have local state in a bigger, real-world program. And I use that.
Not really. You’d be surprised how you can abuse the Python metaclass machinery…
And yet, when current functional programming rubber hits the road, something, somewhere has to store some local state.
Excellent point. I was thinking about this too at same time you were. :)
Our original example follows in long-hand FP, assuming lists implement the Applicative.apply method (a.k.a. operation).
Or.
And follows the short-hand syntax I proposed.
The equivalent Python follows when the result is expected to be [16, 26, 36, 17, 27, 37].
We could instead make the semantics of [ ].apply such that [16, 27] is the result, i.e. the members are matched in corresponding order (discarding extra non-corresponding members) and duplicating each subsequent list as many times as necessary to create exact or extra correspondence. Or if we don’t duplicate, then the result will be [16].
Thus I argue that Applicative.apply should take a second function, which contains this semantic choice. Perhaps I should email Connor McBride (inventor of Applicative) about this.
So then the examples change as follows.
And follows the short-hand syntax I proposed.
I don’t think you can implement this modularity cleanly in Python? If not, IMO this is a slamdunk advantage for FP.
> [1, 2, 3].apply([10].apply([6, 7].appy(Doolittle.lift)))
Correction.
IMO, the long-hand form is too noisy.
I think one of my responses got stuck in the black hole of spam filtering. Not to worry.
@JustSaying on Wednesday, March 6 2013 at 5:06 pm said:
> Jessica’s proposal of adding plus and multiply operations to every type that wants to use Doolittle, is the antithesis of modularity.
Nope, I don’t believe I said that. Every type that plus and multiply has meaning for probably has such an operator already. It might be necessary to add them to meta types such as list or nullable, but that is one place, not some massive combinatorial explosion that you seem to think. And defining operators as part of modules which cohesively describe the totality of that type is the essence of modularity, powered especially by the generics abstraction.
I think that you might not see that because being an FP guy you think it terms of functions primarily, whereas I think in terms of domain relevant representational objects. But that is speculation, I can’t see in your brain.
> Imagine for every generic function in the universe, we have to add all the operations they collectively use to every type in the universe– a combinatorial explosion nightmare.
What you are saying doesn’t make any sense to me, and since it is the heart of your point, I think it would serve us all well if you could clarify, perhaps with some useful, real world example.
> Sure, but this can be done automagically as a special case.
Like I said in my long lost post, it isn’t magic, it is code, and there is nothing particularly special about the code, and certainly nothing that can be done in other languages.
var a = [1, 2, 3] var list = [ ] var cnt = 0 for(var i = 0; i < a.length; i++) { list.append(a[i].str) cnt+=1 } return (cnt, list)See here you are either being disingenuous, or you are unfamiliar with modern static typed languages. What you write above would be a valid comparison to C, but all modern static languages would express this quite differently, and much more clearly.
@Patrick Maupin
@Jessica Boxer
Every type (a.k.a. class) that contains members which are numbers can be potentially be a candidate to have its members fed into the aforementioned function Doolittle, e.g. `class Account` which contains some financial figures. So this means every time we create such a new class with the solution you two advise, we will have to supply a plus and multiply method function (a.k.a. operation).
Oh but what happens when we create thousands of functions analogous to Doolittle, introducing all sorts of new operations on types (not limited to numbers), e.g. minus, modulo, and every possible function in your imagination. Then with your proposed paradigm, we have to add thousands of methods, one each for every operation in those thousands of functions, in every class that can be input to thousands of functions.
But “thousands” does not truly characterize the problem. This is actually the Expression Problem (c.f. Wadler), because those “thousands” is not a static number. New functions will be created by users who want to reuse your code, and instead of separate compilation reuse, they would have to instead add new methods in all those types and recompile other people’s code. (can you also imagine the unresolvable naming collisions) But even that does not characterize the problem fully, because the number of types are not static either.
I will say again, that it is the opposite of modularity.
In short, combinatorial explosion.
What you have done is taken an orthogonal function Doolittle which operates on numbers, and unnecessarily forced (conflated) it to operate on every class that can contain a member that is a number. It is unnecessary, because we can instead use Applicative to lift Doolittle automagically to every class that implements Applicative.apply– which is only one method to add and only once.
I suspect the reason you’ve probably not run into this problem in practice, is because you are not attempting to do the sort of semantic reuse that I have demonstrated for the Applicative. Once you actually try to compete with Applicative using your proposed generics paradigm, then you will see the explosion. And probably the use cases that Applicative can solve more elegantly have been manifested in your code in ways so far that are not intelligible in a way that you could see Applicative as solution. And this is because the imperative model obfuscates generalizing higher-level semantics. This can be proven with category theory because if your imperative models don’t adhere to the properties of the category, then you don’t have a category. So you go about solving such problems in other non-categorical (a.k.a. ad hoc) ways. I don’t have a specific example right now, but I will be on the look out for (or develop one from first principles) a coherent and concise case study to illustrate.
I am reasonably proficient in Scala (and historically C++) and AFAIK Scala has all of the important features from C# plus more that C# does not have. Scala can be used to do FP, OOP, or imperative programming. I come from writing massive volumes of code in assembly and C for algorithmically intensive applications such as word processing, natural painting, and 3D rendering, so I also know imperative programming. I have spent a lot of time studying programming language design and different programming languages over the past 3 years, and this tends to sharpen the deeper understanding of these features. But of course, I am not omniscient thus I am open-minded.
“Any sufficiently advanced technology is indistinguishable from magic.”– Arthur C. Clarke
Not either-or. There is a third case. I just chose to show the canonical historical loop because the most people know it, since I was shoving that example into the very rough taxonomy of “imperative programming”.
One of the creators of Borland C worked with us on (what is now) Corel Painter, and later he contracted for me briefly on my CoolPage. I know of Anders Hejlsberg the current lead is on C# (he created Borland’s Delphi), and I have studied some of his design concepts and where he wants to go with the language. Last time I checked (couple years ago), it confirmed to me that his “everything but the kitchen sink” approach (e.g. delegates, etc) is ad hoc and not unifying.
I am dreaming to unearth the next big unifying concept, after C. Everyone knows C (and JavaScript). We can’t say that about any other (multi-paradigm, general purpose) language since.
@JustSaying
Obviously only forced for those types that do supply the necessary method functions (e.g. plus and minus in the case of calling Doolittle), but that is the point of why it is combinatorial explosion if you attempt to reuse many types and functions this way. In practice, you are not attempting this wide scale reuse, because it does not scale in your chosen paradigm. It does scale with the Applicative.apply paradigm, because it is only one method function to add.
I do agree generics are one of the tools for reuse, and we need higher-kinded generics to implement Applicative, because the genericity we require is higher-kinded (second-order specifically). C# does not have higher-kinded generics. Python doesn’t even having typing. Jessica’s paradigm conflates the first-order generics with the higher-order requirement for modularity. I could unpack this concept if you want, or just trust me because this discussion is getting long-winded.
@Patrick Maupin
I may have to take a break, because I’m finding I can’t keep my eyes open while reading those research papers. I expect my prose will become unintelligible, more verbose, and errors will increase when I am this tired.
I wasn’t surprised while reading that I had the correct intuition that the “consistent group” would be the most successful (before I read that they were), unlike the expectation of most programmers quoted as follows.
@Camel has two humps paper
Recall I wrote up-thread.
@JustSaying
The key to programming is the patience, discipline, fortitude,and IQ to build a mental model of the semantics.
Tying this into the Preprogramming Knowledge paper you linked, the people who use cues and clues instead of a completely understood model, fail miserably at programming. Thus, they can never figure out the cause-and-effect relationships between the changes they make to the code and the changes in the program behavior. This is why all good debuggers are good programmers, but not all programmers are good debuggers. The latter programmers are only good at “template style” work (e.g. copy+paste, add/remove a few fields).
I don’t have any difficulty debugging FP programmers. They are no different than imperative programs in terms of debugging. Those whose mental model of programming are glued to the “loop template” may struggle. (I hope you are not conflating with FP in general, Haskell’s difficult to debug due to non-deterministic timing, lazy evaluation)
I think it takes so much effort for us so that our mental model of a language’s semantics becomes second nature (i.e. analogous to driving a manual transmission semi-unconsciously) i.e. become an expert, that there is a great resistance to being a virgin again (geez can’t get Jessica’s g-spot comment out my mind) with a new mental model that repeats the stumbles and learning curve that we endured in the past when we first learned to program. I really struggled with learning Haskell. It was so foreign in so many ways compared to anything I had been exposed to before it. Haskell, OCaml, and ML still look like gibberish to me until after I re-familiarize myself each time, because I haven’t used them enough for it to become ingrained second nature.
You think that a loop is the most natural way to understand how a computer does iteration. Recursion (with tail-call optimization) is just as natural. These are two different mental models of iteration. You favor one, because you learned it first. I pushed myself to learn both, because I am highly motivated. Now I want to teach novices recursion instead of looping, because I have learned that FP is far superior for modularity than imperative programming can ever be.
I think humans want to be as declarative as possible. See items 6 and 7 on Figure 4 on page 139 (page 6 of the PDF) in the Preprogramming Knowledge research paper, “Add all the wages” and “Divide the wages by the number of workers”.
That is declarative. State and order are not the opposite of FP and declarative programming. FP eschews modal (i.e. locally stored) state, but not state in general. Every pure function is a state transformation from the inputs to the outputs. FP eschews conflating what can be orthogonal. Imperative programming style echews orthogonality in favor of explicit boilerplate and encourages conflation due to a preference for low-level semantics.
Nevertheless, if the semantics employed in FP become too obtuse and obfuscated, then programmers won’t be able to keep a mental model in their head, and then they won’t be good programmers.
So that is the challenge any FP language designer faces. So far, none have succeeded in the mainstream.
I see your mental model of FP so far is that it is some kind of high-level sophistry where the user doesn’t have to know the details, thus your fitness to managers. But IMO this is not the correct mental model of FP. The correct one is analogous to yours with loops (or generators) for iteration, but employing recursion instead as the fundamental building block.
I think Programming 101 could be a brief introduction to a chosen language, readily available trial and error bed, a reference guide, and many source code examples (and IMO preferably broken down into small orthogonal modules).
Local state isn’t a problem if it is does not persistent when the containing function returns. Persistent local state (or writing to global state) is the antithesis of modularity. It can be used quite effectively in small worlds. I do too. But it doesn’t scale to some granularities of modularity. But then again, if we treat the program itself as a pure function (i.e. Unix pipes), then we can view all the state in the program as a non-persistent local state, and then the programs are modular. So the desired granularity of modularity can drive our choice of priorities.
It is not so simple for me to say “don’t ever use persistant local state”. I keep that tool in my toolbox too.
And hopefully in the top-level function hierarchy which does not need to be modular. FP pushes the storage of state up the hierarchy.
@Patrick Maupin
I see from that Tale of two humps research paper you linked to.
50% can be programmers (maybe only 40%)
30% can be good programmers (60% of all programmers)
7% can be excellent programmers (14% of all programmers)
I note the tiobe.com rankings show that functional languages only have 3.3% share, down 0.9% from prior year. These languages are Lisp, Scheme, Haskell, Erlang, Scala, ML, and F#. Some people are doing some forms of FP in JavaScript, Python, and C#, Perhaps the FP programmers come from the upper half of the bell curve at the second hump, i.e. the 3.5% upper half of excellent programmers (7% of all programmers).
If someone was able to make FP somewhat easier, that language might make it into the Top 10 with 3.5% share. And if they combined a way to code in imperative-like semantics that are automagically translated to FP, then Top 5 is perhaps attainable.
Using The Wayback Machine, the 2008 archive says that FP share was 1.4%. So it appears there had been a wave of interest in FP over the past 4, but this reversed in 2012. This could be just a hiccup as the smartest programmers who are early adopters have to still earn a living and these toy languages get put on the back burner until they become less obtuse and have enough share to be accepted for more jobs. I am reminded of Yammer’s letter last about the trouble they were having with Scala’s complications.
Note Lisp was the #2 language in 1988 behind C. This was before the OOP wave. I got into C++ around mid-1990s. FP wasn’t even on my radar.
@JustSaying
> Every type (a.k.a. class) that contains members which are numbers can be potentially be a candidate to have its members fed into the aforementioned function Doolittle, e.g. `class Account` which contains some financial figures. So this means every time we create such a new class with the solution you two advise, we will have to supply a plus and multiply method function (a.k.a. operation).
Sure, but that is the very essence of modularity. What does plus or multiply mean for Account? It isn’t entirely obvious, and so you define at that domain object level what it means. Like I said before, somewhere in the system we have to define what plus and multiply mean for a type insofar as that type supports such an operation. Sometimes the language defines it, sometimes the libraries, and sometimes custom code. But the meaning of plus is tightly bound by the item being added. Plus for int is different than plus for double, is different than plus for Complex.
> Oh but what happens when we create thousands of functions analogous to Doolittle, introducing all sorts of new operations on types (not limited to numbers), e.g. minus, modulo, and every possible function in your imagination.
I don’t understand, there are only a very limited number of mathematical operations, many of which can be implemented by self reference (and most languages support ways to define a few base operations and automatically supply the others, so that add and assign can be combined automatically to make +=, for example.)
I’m not seeing the thousands of operations you are talking about.
> But “thousands” does not truly characterize the problem. [etc…]
Sorry, I really have no idea what you are talking about (and if you are quoting Phil Wadler, then that is no surprise to me.) Can you give concrete examples? I write code for a living; I don’t see the problems you are talking about.
> It is unnecessary, because we can instead use Applicative to lift Doolittle automagically to every class that implements Applicative.apply– which is only one method to add and only once.
You are going to have to explain that one too. I have no idea what you are talking about.
> I suspect the reason you’ve probably not run into this problem in practice, [etc…]
Perhaps you are right, but honestly you are going to have to give some examples of where your paragdim is superior, preferably from real world cases of actual programming work you have done.
I am reasonably proficient in Scala (and historically C++) … I come from writing massive volumes of code in assembly and C for algorithmically intensive applications…
I am unfamiliar with Scala, my comment comes from the fact that everytime you give a comparative example in procedural programming it is always the worst possible way to do it. If you compare the best way to do something in FP with the worst way to do it in procedural, FP will tend to come out on top. But the comparison is neither useful nor fair. And to be clear, the majority of modern programming is not “algorithmically intense.” It is mostly just moving data around, capturing it, reformatting it, displaying it, summarizing it and so forth. Which isn’t to say that we don’t all write some “algorithmic complexity”, but it is a small fraction of what most programmers do.
@JustSaying:
I will respond later to your points, but I take immediate exception to this:
You have not at all accurately captured my mental model. For a start, you should consider the possibility that I have a great deal of respect for good managers. Next, you should look up “sophistry” and figure out if that is really what you meant to write — it certainly has nothing to do with my opinion of FP.
@Patrick Maupin
I warned that I am sleepless. I obviously meant sophistication, as sophistry wouldn’t make any sense in that sentence. Thanks for pointing that out. My point is obviously that one can dig into the low-level recursion of FP just as they can for loops in imperative iteration, contrary to your assertion that only loops or generators allow the programmer to dig down.
I look forward to your explanation of your mental model of FP and why you think it is a good match for managers.
My extreme exhaustion faded for the moment and I am alert, so I think I can make coherent reply.
@Jessica Boxer
But what if there is a way to write all such specialized functions, plus, minus, modulo, and zillions of others for Account<T> by writing only one function that implements Applicative.apply<T>?
What would be more modular? Writing all those functions and constantly having to go back and add news ones, or writing the Applicative.apply only once?
Modularity is the ability to:
Only need to write those specialized functions for the fundamental types.
Reuse those fundamental types in other generic types, e.g. GenericType<Type> (a.k.a. in Scala GenericType[Type]).</li?
Write only one `apply` method function for the GenericType.
Then all those GenericType can be reused any where there is a function that inputs Type.
Then we can create an unlimited diversity of generic types that are parameterized on Type, without having to write all those specialized functions every time and forever into the future.
It is not thousands (that is why I put it in quotes), it is open-ended a.k.a. unlimited.
And it is two-dimensional, unlimited x unlimited. (These are the two dimensions in Wadler’s Expression Problem) I explained above one of the dimensions that causes open-ended writing of specialized functions. Every new generic type similar to Account<Type> would have to include the specialized method functions, instead of only one Applicative.apply method. Which means we would have to go back and change those class source code every thing we add a new operation in the other dimension below. With Applicative.apply, we don’t have to.
The other dimension is that the operations that functions like that Doolittle will do on input types is not limited to math. It is unlimited to any kind of interface in your imagination.
Let’s explain this by first fixing your generic Doolittle so that it will always compile. My C# is a bit rusty, so please accept the following pseudo-code or Java.
interface PlusMultiply extends Plus extends Multiply PlusMultiply Doolittle( PlusMultiply a, PlusMultiply b, PlusMultiply c ) { return a + b * c; }So now we know any GenericType<PlusMultiply> that implements Applicative.apply, can be applied to any function that inputs PlusMultiply. No need to implement the Plus and Multiply interfaces for the GenericType.
And what if we did implement Plus and Multiply for the GenericType, then it might no longer be parametrizable on any Type and we might have to limit the genericity. So that is another reason that your proposal is not modularity. But that is a side-track point. Let me continue on about why we get unlimited operations similar to Plus and Multiply.
So I create a new function.
SomeInterface ScoobyDoo( SomeInterface a, SomeInterface b ) { return a.somemethod(b) }Under your paradigm, I would need to go add `extends SomeInterface` and the method `somemethod` to every case of a GenericType, whereas with Applicative.apply I would not.
Does that clear it up for you?
Note the genericity you are referring to is first-kinded (a.k.a. first-order). The genericity I am referring to is second-kinded (a.k.a. second-order). A kind is a type constructor.
I gave a reference on that up thread, Generics of a Higher-Kind, by Adriaan Moors, Frank Piessens, K.U.Leuven, and Martin Odersky, EPFL, In the proceedings of OOPSLA 2008, Nashville (TN), March 2008..
Still benefits from eliminating combinatorial explosion of housekeeping boilerplate.
@JustSaying
> But what if there is a way to write all such specialized functions, plus, minus, modulo, and zillions of others for Account by writing only one function that implements Applicative.apply?
Perhaps I am being dumb, but that would be great, but the function would have to be psychic, wouldn’t it? After all, in my implementation, adding two accounts requires you to add the amounts net of tax, and then recalculate the tax on the total. Also, Account allows multiplcation but only by a percentage. How does Applicative know these things? That is what adding accounts and multiplying means in my context.
> or writing the Applicative.apply only once?
I stand corrected, apparently it is a magic function.
> Modularity is the ability to:
This is an interesting definition of modularity. Perhaps this is our divergence. To me modulairity is the ability to group tightly related functionality into one box and abstract the inners from the interface. The concept of modularity appeals to the idea of a pluggable module with a magic inner mechanism and a known well defined interface. That doesn’t seem to correspond to your definition which seems to be “modularity == conciseness”.
> Only need to write those specialized functions for the fundamental types.
But one girl’s fundamental, is another man’s non fundamental.
> Does that clear it up for you?
Not in the slightest, I couldn’t follow your explanation at all. It didn’t use terminology I am familiar with. I have no idea why you are talking about GenericType<PlusMultiply>
Perhaps my problem is that you are talking about Scooby Do and Doolittle. Perhaps you can offer some real world examples where what you are saying actually matters. I’m afraid I find it had to get my head around these academic examples, since they don’t translate into real world impact.
> Still benefits from eliminating combinatorial explosion of housekeeping boilerplate.
Like I say, I have written a lot of programs, and none of them exploded… (Actually that isn’t quite true, one time I was writing this device driver ….)
@Jessica Boxer
The Applicative form of modularity only applies to classes that accept the normal behavior of the operations on the types that are input to the function that is input to the Applicative.apply.
The first semantic you stated above could be done within the Applicative form of modularity. Expose only the amounts net of tax to the function input to Applicative.apply and recalculate the tax when the function returns a value.
If your Account class is not generic, i.e. it is not
then to inherit the Applicative<Type> interface, set Type equal to the type of your amounts net of tax, e.g. Float. In Scala this can be done as follows.
class Account extends Applicative[Float] { ...However, for limiting multiplication to percentages, you would need to implement the PlusMultiply interface in your Account class instead of Applicative.apply.
Yours is first-kinded modularity, and mine is second-kinded, which I mentioned in my prior reply. What this means is that yours is generalizing the constructor of a type, mine is generalizing the type parameters of the constructor of a type. This is two different levels of genericity. The former is applicable to fundamental types, and the latter is applicable to types that are parameterized on fundamental types. Your definition of multiplication for Account does not fit the Applicative semantics, so it must be classified as a fundamental type and you must do the genericity of the first-kinded.
So I should correct my prior allegation that what you proposed isn’t modularity. Rather it is first-kinded modularity, which is combinatorial explosion only if applied to all second-kinded cases of modularity (instead of using for example Applicative.apply).
If you have
then you can create an instance of it new GenericType<PlusMultiply>. That is second-kinded modularity.
PlusMultiply is an interface. C# has interfaces. Any type which implements PlusMultiply can be used to construct an instance of that type.
Whereas if you have
that is first-kinded modularity.
Correction:
@Jessica Boxer
Applicative<T>.apply is feeding the instances of T (that are member values in the subtype of Applicative) to the function that was input to `apply` and then storing the result back in those members.
There is no magic. The genericity is on the type parameter T of the class which is the subtype of Applicative (i.e. second-kinded) instead of on the class itself (i.e. first-kinded).
@JustSaying
Correction. If your Account is not parameterized on the type of “amounts net of tax”, then it can not properly implement the Applicative interface. This is because the return type of the function input to Applicative.apply can be any type.
Perhaps it is possible to write a different interface, e.g. ApplicativeStatic<T>, where the return type of the function input to `apply` can only be the same as T.
Your distinction between first and second kind modularity makes no sense to me. List is a module that holds all the tightly coupled list like behavior in one box with a well defined interface. This is true regardless of the fact that list is a meta type. The definition is the same regardless of whether one of the interface parameters is a type or not.
It is true that C# doesn’t implement meta meta types, those are implemented through inheritance and interface inheritance. However, it kind of reminds me of mutiple inheritance. It sounds like a great idea, except that I have never, ever found a case in the wild where it is actually useful, and I have found many, many cases in the wild where it results in a tangled mess.
Can you give me an example of a meta meta type that you have actually used? Not in theory, but in an actual useful program that people use to get work done? Can you demonstrate that it was an abstraction that made your code more robust, more clear, more grokable, more maintainable, or some other attribute that I would value?
Much to my chagrin, I interview programmers a lot. One of the things I find is that many can talk a good talk, but if you ask for concrete examples of the theory, things they have actually done in a productive useful way; well that is how you separate the sheep from the goats.
Me: “Can you explain what inheritance is used for…”
Candidate: “Imagine dog, and cat, they are different but both types of quadrapeds. Quadraped has a walk function, but only dog has a bark function.”
Me: “Really? Interesting. Can you give me an example of where you have actually utilized inheritance in the job you did at XYZ corporation?”
Candidate: “Errrrr, so a square is a quadrilateral and a rectangle is a quadralateral, however, a square does not need a width property whereas a rectangle does…”
Which is to say I am not smart enough to think about these highfalutin math abstracts. I’m Jerry Maguire — show me the money. (Oh btw, for the observant, do you notice I used the word “utilize” correctly above, rather than using it as a pompous way of saying “use”.)
If you are designing a new programming language I suggest that the first order of business that you should do is to write some really large useful programs with it. You’ll learn a lot more about it that way than reading a thousand papers, or going to a hundred conferences.
There are some things that I think would be useful to add to C# generics:
1. The ability to parameterize them with non type parameters (such as numbers or the names properties.)
2. The ability to curry lambda functions.
But I think these are relatively small things that probably wouldn’t have much practical use and can be simulated with a little extra verbosity.
@JustSaying:
I couldn’t disagree more. I create lots of classes that have number members, and lots of functions that it would be an error to feed one of those classes to. I don’t want some random outside Applicative function altering the code to make things that shouldn’t work just silently start to work.
Well, like I said, I don’t want that. But if I did want that, why I’d just use one of the AOP libraries available for Python, and use introspection to automatically wrap the world.
Only if you want to automagically make it where I can’t find my bugs because everything just works now.
This conflates a couple of things, but it Python it is certainly possible to add new methods to any user class you want, without recompiling. And as I mentioned, you can do it pretty darn automatically.
I can’t speak for Jessica, but I certainly do all sorts of semantic reuse. What I don’t normally do is automatic, global code injection.
Writing something that affects something somewhere else is usually pretty obfuscatory. But that has nothing to do with your definitions and everything to do with how normal people think.
In practice, I get lots of wide scale reuse where it makes sense, and it scales great! What I don’t get is the system automagically fixing up types for me so that they work where they shouldn’t.
You can add it if you want. One example is the traits library. Lots of static-typing weenies love it. You should try it!
That severely misquotes the original, which explicitly states that not all good programmers are good debuggers. If you use indirection ;) you can get back to an even more definitive statement:
Why yes, I am, at least the lazy evaulation part. Lazy evaluation is one of the things that makes FP practical, IMO. And I assumed that is where you were going since you are so dead-set against exception handling. (BTW, for some classes of programs, I use lazy evaluation a lot in Python.) Finally, you keep touting that the great thing about FP is order doesn’t matter, so I’m having a hard time squaring that with any kind of complaint about non-determinism in ordering, although I have to say that if you can’t turn off the concurrency and get repeatable ordering for debugging then you have a really stupid design decision in the language implementation, and if you get different final results when you turn off the concurrency for debugging, then all the purported benefits of FP are out the window, because you’ve just shown that your system isn’t truly functional.
I hope you realize how smug this sounds. THe closest thing it reminds me of was a comment in a discussion on Slashdot about Verilog vs. VHDL. The general consensus was that people who learned VHDL first had no problem learning Verilog later, while people who learned Verilog first struggled with VHDL, so one observer opined that meant that people should learn VHDL first.
Frankly that’s a crock, because Verilog is kinda hokey, but VHDL fights you every step of the way. (VHDL is a proper subset of ADA, for those keeping score.)
For the record, I am happy writing top-down parsers, I am proficient in recursion, and none of that has anything to do with my thoughts about FP. It really is all about effectively using local state sometimes.
If you say so. I mean, you can either describe the order up front, or you can declare what you need in raping (live women), pillaging (loot to be found), plundering (found loot to be removed), and what happens with the fire (dead women, loot destroyed), and let the system come up with the right answer. Traditional functional programming usually relies on the latter; traditional imperative programming is declarative to the extent you say this statement is declarative.
I’m with Jessica here — if you think there aren’t great tools available for removing boilerplate (for real-world programs) in several of today’s languages, you’re not paying attention.
No, I think good managers usually communicate with subordinates in a declarative/functional manner. They expect subordinates to be working together on a problem, so the problem is expressed in a way to facilitate this. The big exception is scheduling — known high-level dependencies are usually made explicit in a pert chart so that a reasonable schedule can be made. But even the pert chart is about as declarative as rape/pillage/plunder/burn or as the HTML example you give in your link…
Doesn’t have to be. You can write really modular stuff using Python generators that each have local state. It is really just another way of doing lazy evaluation. And I would argue that a generator that contains local state inside the generator function can be written to be just as analyzable as any pure function.
That’s a funny view of persistence. If I have a chain of programs, each with local state, then the state in each program persists until it exits. The last program in the chain might have done a lot of work before the first program exits. But hey, if you like this paradigm so much you figure out how to use it inside a process, guess what? Congratulations! You have just invented the Python iterator protocol!
Ah. So if you’re not doing pure FP, then you’re doing multiparadigm. Might as well be doing Python. Oh, except you want to subtract some of the capabilities from the other paradigms, right?
I certainly don’t hope for this. At least not in the general case.
Right. Which is sometimes well away from where it is used and is easy to reason about.
Or perhaps not. What the heck kind of speculation is that?
I’m all for language research and trying out new thing, but why does it need to be made easier if it’s no harder and already gives better results? I find your stance very confusing. You are absolutely sure that functional programming paradigms can make programming all better, but also absolutely sure that none of the current FP languages do this correctly. You may be right, but the trial syntax you have shown doesn’t really seem all that much better than Haskell, and the concepts you are touting as the best thing since sliced bread aren’t (so far) convincing. I thought AOP was a great thing a few years ago, but it is extremely easy to abuse. Without further knowledge, I would probably put your Applicative in the same bucket. If I wanted it, I could use something like it easily enough — I already showed you how I could decorate functions in Python. What I might not have shown is that I could use an AOP library and a bit of introspection and decorate _every_ function in a program. I have done this for debugging. Believe it or not, it doesn’t take an appreciable amount of runtime to do this, even for relatively large programs. (But there can be a huge run-time hit on the execution after the decoration.)
I’m guessing that has a lot more to do with economics than anything else. C and Lisp and FORTRAN were cheap. Compilers for most other languages were expensive on most platforms.
Something in the middle. Writing Applicative.apply only once could wreak havoc on someone using your library and not having capabilities restricted where they need to be. Personally, I like duck typing. I am unlikely to have a zillion different classes that need to be usable with a given function.
I had been trying to remember who turned me on to Scala when I expressed frustration with Haxe on the developer mailing list (first in 2008 and then apparently again late 2009), and it popped into my mind when I awoke. Google informed me that John De Goes is now working with three well known Scala gurus Kris Nuttycombe, Daniel Spiewak, and Miles Sabin. I see also Franco Ponticelli from Haxe. Notable they have a female Scala programmer (and another female marketing guru from Google) on such a small team. So this enabled me to search my email archive to determine that I learned Haskell circa Oct 2009, Scala before Feb 2010 where I have emails describing my first thoughts about a Copute language. My memory since 2010 is a blur because I was so ill (debilitating insomnia, etc) with continuously high HPV viral load and resultant peripheral neuropathy.
Ah now I read that John and I have similar vision of the future for web apps.
@JustSaying
That doesn’t convey the meaning I intended. Change to “If your Account is not parameterized on the type of the value that stores the ‘amounts net of tax'”.
@Jessica Boxer
I am really good at building mental models of logic in my head. This is where my IQ which apparently far exceeds my general g factor. This is why I love the abstract, but it is also necessary to test the abstract assumptions in the real world. From Two Humps research paper linked up thread by Patrick Maupin, apparently the ability to think naturally in formal models[1] is a key factor (but probably not the only one) to being in that top 3.5% of all programmers. I suspect good programmers also have to be grounded in real world use cases.
[1] Theorems for free, by Philippe Wadler. Google also “Wadler proofs for free” and “Wadler proofs are programs”.
So I am not sure if I can explain this model of parametricity to you, because I’ve tried already. You want the initial explanation to be in terms of examples, not in terms of the model itself. I like to start with the model and then find the examples later, or to find a model to unify known examples.
However, let us assume that my elucidation has been insufficient.
Multiple Inheritance
First of all, your original proposal has a parallel to multiple inheritance.
Type Dolittle<Type>(Type a, Type b, Type c) { return a + b * c }Your Doolittle can only be called for any Type that implements the plus and multiply operations. It is matching by duck typing on those operations (a.k.a. methods).
That has a similar effect as defining interfaces for Plus and Multiply and using multiple inheritance. I recommend this so that the function’s type is more explicit (remember your point that naming improves comprehension). I encourage you to use explicit types instead of duck typing, so the consumer of your API knows which types are allowed without studying the source code of the implementation of the function. This will greatly speed readability and comprehension.
interface Plus { // define a method for + operator } interface Multiply { // define a method for * operator } interface PlusMultiply inherits Plus and Multiply; // multiple inheritance PlusMultiply Dolittle(PlusMultiply a, PlusMultiply b, PlusMultiply c) { return a + b * c }Thus any type (a.k.a. class) that implements the PlusMultiply interface, will be accepted by the compiler as input to Doolittle.
Kinds of Parametricity (a.k.a. Genericity)
The genericity described above is first-kinded. This means it is generic only on the subtypes that can be constructed from PlusMultiply (or analogously the subtypes of the duck type with plus and multiply operations). So the genericity is based on the construction of a type (a.k.a. class). Please tell me if you did not understand up to this point?
Now I am trying to introduce you to second-kinded genericity. I think you are familiar with defining a class that has type parameters, e.g. class Generic<Type>. That is second-kinded genericity, because the genericity is w.r.t. not the construction of a type, instead the parameters to the construction of a type. (in type theory a type has a constructor function but you don’t need to know that here)
Also C# can do second-kinded generics, but it can’t do third-kinded (a.k.a. Higher-kinded) and thus we can’t implement Applicative properly (because the apply method needs to parameterized on the subtype of Applicative, but this subtype is already parameterized so we need an extra order of kinded genericity). Actually I may be abusing the canonical terminology. Perhaps what I referred to as first-kinded, is actually not kinded. And what I referred to as second-kinded, is actually first-kinded. I should go check when I have time. But my point remains the same regardless.
So if we want to employ second-kinded genericity for the Doolittle function, we must make the PlusMinus interface a type parameter of the class being input to the Doolittle function. But the required change to make this happen does not involve any change to the Doolittle function. Instead we implement an Applicative interface on the class that takes type parameters.
The advantage of this is that the class that takes type parameters, can then automagically operate on any function that inputs any interface. It is much more powerful form of genericity. But as our discussion of the Account example showed, this form of genericity does not fit to all use cases, so then we have to fallback to first-kinded genericity and implement the PlusMinus interfac e for Account instead of the Applicative interface. Thus we know that Account is a first-kinded genericity type a.k.a. “fundamental type” (my own terminology).
Having both of these types of genericity available, allows us to pick the best fit for each use case, and thus avoiding some combinatorial explosion that could occur otherwise.
I already did up thread. It enables weaving the error state (e.g. Nullable) through the functions that don’t operate on those error types (e.g. Nullable). This eliminates a load of cruft and makes code much more readable. Imperative programmers throw their hands in the air and instead throw an exception (or use that global variable modality that Patrick mentioned for Python), which is uncomposable spaghetti.
Until that example is admitted as being important, I don’t see what is the point of me expounding on other examples. I am happy to work with the 3.5% of the best programmers in the world and generate code faster, more provably correct (i.e. reliable), and with better readability amongst us. The important factor is that it not be so obtuse that we can’t read each others’ code. I think I can also get novices to go this direction. This could pull the legacy resistance our way over time. I do understand the power of open source and the Linus rule is “even millions of dumb monkeys pounding on imperative source code can generate higher quality than a few experts coding in isolation” a.k.a. “given enough eyeballs, all bugs are shallow”. However I don’t think that Inverse Commons paradigm can overcome a structural (mathematical in category theory) incongruence with certain forms of modularity.
Exactly my plan. If I complete a new language, it is because I want to use it. But I won’t make a new language if I can’t offer a significant paradigm shift. No sense in reinventing the wheel. My time is limited, I am already age 47. And likely Copute will output Scala. So that it is possible to access all the power in Scala and Java. An older version of Scala also compiles to the C# CLR virtual machine.
Yikes C# has corner cases like that. I expected it to be so, given the ad hoc design. According to that paper on Generics of a Higher-kind, it might be much more difficult to implement higher-kinded typing in C# due to all type parametrization being reified in the CLR. The JVM doesn’t do this. There are trade-offs…but I am not omniscient…
Wow this is getting long-winded, but I am able to slay your demons below.
@Patrick Maupin
I wrote potentially above. I have explained in my past two replies to Jessica, that higher-kinded genericity combined with the Applicative semantics, is not applicable to all type parametrized classes. It is a powerful form of genericity that can be used where it applies. It is a tool I want in my toolset.
And the Applicative semantics are not random (and not global), they are set by the implementation of the apply method for the class that inherits the Applicative interface.
I even stated up thread where we could in theory modify the apply method to input a function so the semantics are customizable (a.k.a. pluggable) on each invocation of the apply method.
And then I gave an example up thread of how we can use that proposal to make operations on lists (collections including arrays, etc) much more declarative and easily comprehended.
So I don’t know why anyone is thinking I haven’t given powerful real world examples already.
You don’t want subtyping? See the PlusMutiply example in my prior reply to Jessica.
Yes of course you don’t want typing, you use Python which is un(i)typed.
Introspection is not composable (I explain why at the end of this post). I am going to have to actually create this modular world of source code, for you to see that in the real world.
Single-point-of-truth (SPOT) is good design fundamental, e.g. a class. Eric has mentioned SPOT. Any abstraction hides details, and that is good, because hiding implementation details is what declarative programming essentially means. But it is not hidden in the sense of no access a.k.a. closed source. The source code for the class is available.
Only if you have implemented Applicative for your class. And then I presume you understand what your code for the apply method is doing. So I don’t agree with this allegation that you can’t debug your own apply method.
The original quote from the two humps paper was, “They found that the majority of good debuggers are good programmers, but not vice-versa.”.
Feeling smug? (I find when people accuse others without good fact checking, they are often practicing what they are accusing)
I correctly conflated that lack of typing is a way to break mathematical correctness of the category, thus leading to all sorts of corner case bugs. In small worlds, you may be able to manage this with the Linus rule (see my prior reply to Jessica), but IMO it will not work in the wide-scale modularity that I seek.
Your implication has a false assumption that Applicative.apply is a global effect. Each class that implements Applicative must implement an apply method. It doesn’t happen automatically globally.
AFAIK, it is not checked at compile-time. It is dynamic typing (i.e. the type information is stored in a runtime manifest).
Incorrect.
Perhaps you get that incorrect assumption about what I am touting, by misreading the Declarative section of my Intro to Computers.
I said that in declarative programming deterministic order (and duplication) is allowed when it is the intended semantics. I said that ordering and duplication dependency is not allowed when it is in extraneous implementation details of the intended semantics. I was not referring to functional programming. Also in that section, I wrote that functional programming is one way to obtain declarative programming.
Many of us functional programmers don’t like non-strict languages (a.k.a. lazy evaluation by default, e.g. Haskell), including Bob Harper one of the authors of ML which is not a lazy language, ditto Scala.
Sometimes != always
FP has a role to play. It is not an all-or-nothing religious war.
I never said order is not declarative programming. I never said FP does not express order.
I never said that. I am trying to improve our toolset.
Agreed. Up thread I wrote as follows:
@Patrick Maupin
Incorrect. Local state can not be generally composed. It may be modular in your mind because it can be syntactically and semantically composed, but the runtime state can not be composed. You may get away with this in your small worlds by being careful how you compose, but in the general case, it will create bugs.
The state of the output of prior programs in the chain does not change after it is output. Local state held constant, does not cause a problem with composition.
The state goes higher up in the hierarchy, so the majority of the program at the lower-level can safely use modular composition.
I don’t propose mixing multi-paradigm in the lower-level modular composition.
Ah because of the below…
Exactly the opposite of what I think is correct. And that is the essence of your mental mode versus mine. I intend to find out whether I am correct or not.
I think you are conflating order with state. Recursion expresses order without storing local state (outside the return of the function).
Because I have noted how intelligent the programmers in Scala and Haskell are. And I note that Eric makes a point to mention that he originated from the Lisp world and that he thinks a well designed FP solution (where applicable) is “absolutely beautiful”. And I think we agree that Eric is damn smart. (He also says that unfortunately he doesn’t think most programmers will be able to grasp FP)
Then again, I am not saying there are not very intelligent programmers doing work in Python. Obviously there are. But in my limited interaction they feel more like very smart journeymen, because I think the person who makes it to the FP level, has a whole other level of awareness about type theory and the ilk.
It is not already no easier, because at least we are battling against the pre-learned imperative mental model and because the existing languages overload the programmer with for example type theory or lazy evaluation non-determinism, etc..
Are you serious? The following is less readily comprehensible than Haskell?
We had this discussion already. The following is not comprehensible?
Based on my prior reply to Jessica and this reply, you did not even understand the applicability of Applicative when you wrote that.
In the general composition case, you can not substitute for higher-kinded compile-time checking with runtime decorations. As you say about AOP, there is no well-defined mathematical category, thus all kinds of unexpected abuses creep in (a.k.a. corner cases).
That is why I proposed up thread that Applicative.apply should input a function so its semantics are pluggable after compilation of the class that implements Applicative. And note, that function will be typed-checked at compile-time to make sure its signature agrees with the Applicative.apply semantics. This is why your runtime introspection (without static typing) paradigm can’t keep all the implementations from abusing the well defined mathematical category that insures correctness.
@JustSaying
>> Can you give me an example […] ?
> I already did up thread. It enables weaving the error state
But this example isn’t useful since it is an example of how you can fix a shortcoming of FP (the need to weave error state back through, rather than push it long distances as is possible with exceptions.) So it is kind of like a Chicagoan boasting to an Phoenix resident that Chicago has better snow plows.
> Until that example is admitted as being important, I don’t see what is the point of me expounding on other examples.
I guess I am looking for an example along the lines of “I was working on this problem and fourth order modularity totally rescued me and made my program much easier to write and maintain.” That, after all is the point, no?
> Yikes C# has corner cases like that.
My mistake, you can curry lambda functions in C#.
@Jessica Boxer
No it fixes a shortcoming that throwing exceptions creates spaghetti code, because they are impossible to reason about under composition, because they are hidden N levels deep in call hierarchy that you can not possibly know (unless you can keep all possibilities of the state machine in your head).
Yeah agreed and I know what you want to see (and frankly I do too to convince myself and learn more about design choices for a new language), but unfortunately I don’t have that language to work with until I build it. I could write code in Scala, but it means much duplication of effort. I have to take some shortcuts.
Removing the cruft of correctly dealing with exceptions as return values is a big enough one for me. If I had more time to spend on this right now, I could probably go think up something more convincing, but I am overloaded at the moment. There are probably already some good examples in the Haskell world, but I struggle reading Haskell. And the Scala world has eschewed category theory APIs for the most part (Tony Morris’s Scalaz is an exception, but is very obtuse because he wants to make Scala act like Haskell with type classes), so thus are not using the Applicative.
I don’t have a good test bed for now. I need to build.
@JustSaying
> No it fixes a shortcoming that throwing exceptions creates spaghetti code,
Your characterization as spaghetti code is not fair. that is an expression normally reserved for a completely different kind of problem, so I don’t think it is fair to implicitly bring all the assumptions about, for example, Basic or Cobol, to exception handling code. You might not like the control flow, but it is certainly not spaghetti in the traditional meaning of the expression.
> because they are impossible to reason about
But I have never, in practice again, seem people “reason about” their code except as a navel examination exercise. To me it is a chicken and egg thing. If I have a piece of code C which is prove correct with proof P, how do I prove that the proof is correct? What evidence is there that determining P correct is easier than determining C correct?
The alternative to proving and reasoning about code is to make assertions about its correct functionality and test those assertions. That is to say, functional testing. The second thing that is known to make code better quality is code inspection. In a sense, this is an informal way of reasoning about code. If you read Steve McConnell’s books, epsecially Code Complete, you will see a considerable amount of data on this, the capture rates of various defect reduction strategies. Functional black box testing, functional white box testing and code inspection tend to find different kinds of bugs and so work well if concert.
The key to effective code inspection is to have very clear code, not concise mind you, clear. It needs to be blindingly obviously correct.
The key to black box testing is to have the right tools in place and good QA people. The key to white box testing is to have a language that provides excellent support for dependency injection and IoC.
If you want to improve programming, address these three key issues.
@JustSaying:
Ah. Well, you kept saying that it didn’t need “any boilerplate” and you didn’t start out by saying it was something that you added to the type, so I, like Jessica, assumed it was some sort of magical pixie dust that was unimplementable with Python, and that assumption just sort of stuck. But the way you’ve been describing it lately, it sounds like something that could be easily implemented in Python, just by overriding operators on types. But you also confused by using the example of “a + b * c” along with lists (or tuples), and (in Python) those operators are already defined on those types…
Yeah, but that’s confusing, because it sounds like boilerplate.
I’m thinking that you haven’t given a real-world useful example of something that I can’t do in Python if I want to. Subclassing to add an Applicable mixin would be dead-easy.
What’s a difference between a subtype and a subclass?
What are you smoking? Python is strictly typed. It’s just that the types themselves control what the interoperate with.
Well, it turns out that for what you’re describing, I don’t need it anyway. But it’s a handy tool in the box.
Again, I think we are in sync here. But at this point, I’m not seeing you describe anything I can’t do in Python (add a class mix-in that allows, e.g. Doolittle to automagically return a large list of results).
Again, you confused by claiming that there was no boilerplate, and that it couldn’t be done in Python. Sounded like magic.
It has more to do with logical thinking than with fact-checking, and you claim to be an expert at this. So, let’s go over this very slowly. The normal meaning of “vice-versa” is the converse. The converse of “the majority of good debuggers are good programmers” is “the majority of good programmers are good debuggers”, and the negation of that is “the majority of good programmers are not good debuggers”. Which is a completely different statement than “not all programmers are good debuggers” especially when you go on to explain that of course this is true because debugging is a necessary skill to be a good programmer (which is the opposite of the opinion from the paper).
This normal reading of this plain English sentence also happens to match the more explicit description in the original paper. So, yes, when I wanted to fact-check, I pulled out the original paper to make sure that my plain-English interpretation of “not vice-versa” was, in fact the interpretation used in the first paper I gave. So are you feeling smug yet?
This is true. But most compile-time static typing systems also insert run-time checks, no?
That’s your opinion, stated as if it were fact (or were you stating as fact that my opinion isn’t my opinion? hmmm — hard to tell exactly what you are disagreeing with) Either way, it really is my opinion, and I’m not the only one who believes this.
Perhaps I do. Or perhaps I get it from your other statements like “Thrown exceptions (i.e. `throw` and `catch`) kill concurrency…” If I explicitly code different threads, this is not true. This can only be true if the system is implicitly doing concurrency for me. I await your keen insights into how compiled code can do an excellent job of doing concurrency for me (using all available resources to the fullest on any system I throw the code at) in a fully deterministic manner.
Doesn’t make sense to me. If something is purely functional, hand it to the machine and let it do only the parts it needs. Other than debugging I don’t see a downside. Again, if there is a downside, then it wasn’t really functional, was it?
OK, but you seemed to think that it was absolutely needed for a whole laundry list of features, all of which you seem to think require it…
Now you’re apparently saying it’s all-or-nothing again — that if I don’t do FP I will have bugs.
You realize that in Unix piping, the entire contents are not usually dumped in one go, right? That the local state of the first program actually changes after the second program starts executing, and that subsequent outputs from the first program (after its state changes) are used in the second program? And that this paradigm, which you think works great for Unix, can be faithfully replicated inside a single Python process using the iterator protocol? (You should realize this — I just said it all twice now.)
Yes, yes — different kinds of tools for different levels of hierarchy in the same program. Sorry I get enough of that kind of crap when I design hardware, and software is my refuge from this.
That will be interesting.
No, it’s just that you’ve repeatedly gone out of your way to insist that in many cases order is meaningless drivel (which I agree with), thus (apparently, not sure about your reasoning) iteration is bad compared to recursion. Or something like that.
I know of several Python programmers who could stand toe-to-toe with any Scala/Haskell programmer. But I will give you that Python allows, more than any other language I know, ease of learning and use by beginners. It is the successor to ABC, which Guido designed as a teaching language. Partly because of this background, the language is amazingly consistent, which esr noted in his “Why Python” paper of yore.
I sincerely don’t understand why lazy evaluation is problematic. I have successfully used it as the tool of choice for some really difficult problems in the past.
What is it with your reading comprehension today? In what natural language does ” doesn’t really seem all that much better than” equate to “less readily comprehensible than”?
I didn’t understand how it was created. You originally said “no boilerplate”, not the equivalent of “subclass and inherit” or however it is used this go around.
Compile-time is very important to you. Not so much to me (most of the time). Practicality beats purity.
Exceptions can certainly be misused to create spaghetti code, e.g. to do routine out-of-band signalling, rather than to signal a truly exceptional condition. I don’t use them that way. OTOH, within a layer of pure functional code (no side-effects), exceptions are actually not such a big deal as if you have files open, etc.
Personally, for small programs with few users and unknown inputs, I find asserts to be a great tool. I will start off by asserting that all inputs fit my world-view with bare assert statements, and then as we gain experience with the use of the program, some of the asserts will be re-worked into warnings, others will turn into exceptions with more context for the hapless user, and still others will remain as asserts, often forever.
@Jessica Boxer:
I’m not normally a test guy, but when I am — I always do white-box testing (but I will fall back to black box for code that is too difficult to read). I find I can torture things a lot better than the average geek, and can do so much more quickly when I can examine the structure of the source looking for weaknesses…
@Jessica Boxer
The wikipedia definition of spaghetti code.
When we reason (i.e. think) about how our program is going to behave under erroneous conditions, this is a normal activity of programming that has nothing to do with our navel. I did not say prove correctness, I said reason about correct behavior. It is impossible to reason about something which can’t be known. The control structure of thrown exceptions can’t be known under any reasonable degree of composition, which is what Joel Spolsky (creator of stackoverflow and the ilk) wrote about in the link I provided. Notice the one caveat he stated was the cruft of as turning thrown exceptions into return values. Applicative helps remove that cruft.
I should know because I been eating spaghetti nearly every day for the past weeks, because it is the easiest for me to cook here by myself in the mountain. I really hate spaghetti at the moment.
Convincing use case
Remember we decided up thread that the semantics of adding and multiplying your class Account would require inheriting from the PlusMultiply interface.
But class List<T> can’t inherit from PlusMultiply, because we don’t know a priori which types T will be– some T might not support PlusMultiply.
Fortunately we can make class List<T> inherit from and implement the Applicative interface. Thus when we have a List<Account> (or more generally a List<PlusMultiply>), we can mix the adding of instances of Account with instances of List<Account> (or more generally instances of PlusMultiply with instances of List<PlusMultiply>) without any crufty boilerplate that hides semantics. Thus making our code more readable, comprehensible, and thus less likely to contain bugs.
This syntactical sugar would be automatically translated by the smart compiler I am proposing to the following FP code employing Applicative.apply.
If you can’t visualize how incredibly composable this paradigm of genericity is, then I guess you will just have to wait for me to write some programs in this style.
Follows is an example of the boilerplate cruft required to implement the above with imperative control structure.
function anonymous() { var list = [ ]; for account in [new Account(…), new Account(…), …] { list.append(Doolittle(new Account(…), account, new Account(…))); } return list; }()Maybe some people will have the opinion that doesn’t look too crufty compared to the one liner above, but consider that every time you write a differently structured one liner, you have to think deeply about how to implement it in imperative cruft.
The above is much more complicated to implement in imperative control structure. It requires one loop using an index into the two lists, and then logic for duplicating the last member in the shorter list or dropping the members in the longer list.
Typed vs. dynamic
Btw, for all those who say static typing is bad, consider that the above genericity would not be clearly delineated and the reasoning that drove the creation of the above genericity would not have occurred, if not for the reasoning about the static types. Also the types document the semantics above.
Types are an inseparable aspect of programming, ignore them if you want your programs to less optimally designed. In some “smaller worlds” (for lack of a more apt phrase to describe the applicable use cases), the tsuris from typing out outweighs the benefits.
IMO, the key is using typing intelligently where it is worthwhile, so it aids your coding e.g. Applicative and PlusMultiply, rather than hinders it e.g. Type Doolittle<>Type>(…).
@me:
I forgot to add that I also deliberately code in such a way so as to provoke the Python runtime itself to issue exceptions when something I don’t expect happens. For example, you could write:
This will barf if bar is empty, but won’t throw an exception if bar has 2 or more elements. If I want to declare it an invariant that bar only has one element without wasting extra cycles, I will code it like this:
This is (yet another) use of the iterator protocol. Iterate over bar, and the result should have exactly one item. It also has the benefit of working if bar is a generator rather than a list or tuple.
I am all for finding violations of this sort of invariant earlier rather than later, but so far all of the languages which can do this have other onerous properties that make them unsuitable for my everyday work. (Google’s go looks promising for a few kinds of tasks, but I haven’t gotten at all serious about using it.) Perhaps you can do better; time will tell.
@Patrick Maupin
You continue to ignore my repeated point that the possible operations that have to be implemented for each type are unbounded. Imagine every time I add a new operation in a program, I have to go add it to Python’s list class and tuple class (and every other class that I want to apply).
Also, the genericity paradigm of overloading the operations does not enable you to be able pass a list into a function where a number is expected. We are not always guaranteed that everyone will use open source, they may give us a compiled library instead. You and Jessica both made the assumption that functions like Doolittle will only do operations that have been implemented in for example a list, but maybe Doolittle will do operations that are only known to be implemented for numbers. And if we may not have access to those operations.
Python could implement an Applicative interface. But it can’t type check it at compile-time. This is a problem, because what if the members in a list don’t support the operation(s). Okay this is generally true of run-time typing any way, which is why you need more unit tests. So you could I guess you can implement the Applicative interface in Python with that caveat.
No it is declarative because it is specifying which algorithm to use at the each use-site occurrence. There could be a default set at the definition-site for the class that implements Applicative.apply, for when a choice is not specifed at the use-site.
Doing the implementation of Applicative for all types that need it, e.g. list, doesn’t automatically transform the short-hand, Doolittle(1, [1, 2, 3], 0) to the correct combination of apply, map, and lift calls. Could you do this with some form of meta programming in Python?
I assume you probably can (?), but you still don’t have compile-time checking, but you never do in Python any way.
So yeah, go ahead and “steal” my idea :)
I think you are going to run into a problem at some point with analyzing how to do make translations between syntax when you don’t have the compile-time type information. Or it is going to be slow if you are doing these translations at run-time.
I am confident there is some cost to not having the compile-time typing.
Python does not check types at compile-time. Run-time checking of types pushes type errors to the users (or to extensive testing before release but this can never catch all the same errors that static type analysis can).
It wasn’t my reading comprehension. It was an inability to write what I was thinking. My point was intended to be that my proposed syntax is much more comprehensible than Haskell (and not so different from the existing Python syntax), and that Haskell is much less comprehensible than Python. My unconscious brain tried to merge that into one and it failed in the circuitry that connects my thought process to my writing elucidation. These comments are getting so lengthy that I am loath to proof read them before clicking submit. Writing moves much too slowly for the thought processes in my brain. This was compounded by the fact that I have been doing 16 – 18 hours of programmer per day for past weeks and I have reached that burnout point (head on keyboard) where I need to take a break. So I am making shortcuts.
This is also compounded by the tsuris of composing these in this <textarea> in the browser and having to wade through all the details of marking up correctly with <blockquote> and <pre> and not forgetting to use < instead of < to deal with this blog’s HTML engine. Also needing to copy+paste to text editor to save to hard disk frequently since I don’t have my UPS set up here yet and brownouts are frequent (I do have backup battery but not yet set up for UPS).
So let me rephrase my question. Are you serious? Is my syntax really at the same level of obfuscation as Haskell? If yes, I certainly do not agree.
I will continue in my next comment. Want to break this reply into more manageable chunks.
@Patrick Maupin
There is no inconsistency in logic between those two quoted assertions.
If most of subset of A aren’t B, then not all of A are B.
Btw, you did not mention that the one unconscious typo I made was replacing ‘majority’ with ‘all’ in “all good debuggers are good programmers”. Had I proof read that carefully, I would have changed it.
Logic fail. No one can be a good programmer if they are not also a good debugger. Who writes code that often works the first time? You are applying the converse to an ambiguous syntax known as English, and failing to account for the contextual information that makes English non-ambiguous. We had this discussion about ambiguity of human language when we were explaining the Dunning-Kruger effect to Saurav Sengupta up thread.
I am not perfect nor omniscient, but I maintain my claim that I am above average on the IQ curve for logic and extracting the generative essence of abstract models. I struggle somewhat with the O in I/O. My reading comprehension is high.
This is true. But most compile-time static typing systems also insert run-time checks, no?
Types can not model all the semantics. So yes we have run-time code that deals with semantics that are not typed checked. Types can check some invariants at compile-time. And when you get into the complexity of higher-kinded genericity, I don’t think human or unit tests are going to be as lucid and thus rigorous as the compiler. C.f. my citations from Wadler up thread that programs are inherently proofs (so they need to be checked by a compiler when the proofs get too complex for the human mind to do rigorously). As you know, computers are better at pedantic, rigorous checking than humans are.
I’ve read in the past the paper you linked, and it is written by one of the creators of Haskell. For an opposing view read Bob Harper’s blog post The Real Point of Laziness, one of the creators of ML.
Fact, because I’ve had this discussion already and because I understand the generative essence of the model of a non-ambiguous language is a fundamental duality– inductive versus coinductive typing (corresponding to strict vs. lazy).The conclusion is that both lazy and strict have trade-offs. And most recently I discovered that lazy is the dual of strict in the sense that the Top type in lazy has same attributes of the Bottom type in strict, and vice versa. Studying the Filinksi papers at the suggestion of Bob Harper started me down that rabbit hole.
FP as a form of declarative programming ostensibly doesn’t obfuscated your accidental orderings (as much), and thus you can see more clearly where you have order that creates modality that could conflict with the concurrency. As you know, stored state (modality) is incompatible with concurrency. Why Not Events discusses the problem of modality.
This is an active area of research, so I am not stating this as fact. Concurrency is tough nut to crack.
FP is a tool that can in theory help reduce bugs, and I believe it will be significantly more effective as the combinations of reuse proliferate. However, for smaller worlds where reuse by other third party programmers is not high, FP is ostensibly a subjective choice.
The benefit from FP is not only about avoiding local stored state, also about expressing some semantics more clearly (declaratively) so that bugs are not obfuscated in implementation tsuris. See my prior reply to Jessica.
This remains declarative to the extent there is no feedback loop, i.e. where the latter can’t influence the output of the former.
But that isn’t general reuse because once you have feedback loops, then this model breaks (creates local modalities) in general composition.
Haskell can model imperative effects using the state monad. There is no need to use different language or tools. Once you return the state monad from a function, it will no longer be stateless for callers. So the programmer strives to make the inner hierarchy stateless. Eric mentioned this in his blog about learning Haskell.
Iteration is less generic than recursion, thus less composable. The examples in my reply to Jessica show that Iterable isn’t as general a model as Applicative.apply. I can reuse the same apply in all examples, yet the code employing Iterable has to drastically change for each case.
This derives from modeling computation as functions with defined input and outputs instead of as blocks of instructions with undefined inputs and outputs.
I will continue in my next comment. Want to break this reply into more manageable chunks.
Reposting with fixed closing tags that butchered the prior submission. Hoping Eric will delete the prior post.
@Patrick Maupin
There is no inconsistency in logic between those two quoted assertions.
If most of subset of A aren’t B, then not all of A are B.
Btw, you did not mention that the one unconscious typo I made was replacing ‘majority’ with ‘all’ in “all good debuggers are good programmers”. Had I proof read that carefully, I would have changed it.
Logic fail. No one can be a good programmer if they are not also a good debugger. Who writes code that often works the first time? You are applying the converse to an ambiguous syntax known as English, and failing to account for the contextual information that makes English non-ambiguous. We had this discussion about ambiguity of human language when we were explaining the Dunning-Kruger effect to Saurav Sengupta up thread.
I am not perfect nor omniscient, but I maintain my claim that I am above average on the IQ curve for logic and extracting the generative essence of abstract models. I struggle somewhat with the O in I/O. My reading comprehension is high.
Types can not model all the semantics. So yes we have run-time code that deals with semantics that are not typed checked. Types can check some invariants at compile-time. And when you get into the complexity of higher-kinded genericity, I don’t think human or unit tests are going to be as lucid and thus rigorous as the compiler. C.f. my citations from Wadler up thread that programs are inherently proofs (so they need to be checked by a compiler when the proofs get too complex for the human mind to do rigorously). As you know, computers are better at pedantic, rigorous checking than humans are.
I’ve read in the past the paper you linked, and it is written by one of the creators of Haskell. For an opposing view read Bob Harper’s blog post The Real Point of Laziness, one of the creators of ML.
Fact, because I’ve had this discussion already and because I understand the generative essence of the model of a non-ambiguous language is a fundamental duality– inductive versus coinductive typing (corresponding to strict vs. lazy).The conclusion is that both lazy and strict have trade-offs. And most recently I discovered that lazy is the dual of strict in the sense that the Top type in lazy has same attributes of the Bottom type in strict, and vice versa. Studying the Filinksi papers at the suggestion of Bob Harper started me down that rabbit hole.
FP as a form of declarative programming ostensibly doesn’t obfuscated your accidental orderings (as much), and thus you can see more clearly where you have order that creates modality that could conflict with the concurrency. As you know, stored state (modality) is incompatible with concurrency. Why Not Events discusses the problem of modality.
This is an active area of research, so I am not stating this as fact. Concurrency is tough nut to crack.
FP is a tool that can in theory help reduce bugs, and I believe it will be significantly more effective as the combinations of reuse proliferate. However, for smaller worlds where reuse by other third party programmers is not high, FP is ostensibly a subjective choice.
The benefit from FP is not only about avoiding local stored state, also about expressing some semantics more clearly (declaratively) so that bugs are not obfuscated in implementation tsuris. See my prior reply to Jessica.
This remains declarative to the extent there is no feedback loop, i.e. where the latter can’t influence the output of the former.
But that isn’t general reuse because once you have feedback loops, then this model breaks (creates local modalities) in general composition.
Haskell can model imperative effects using the state monad. There is no need to use different language or tools. Once you return the state monad from a function, it will no longer be stateless for callers. So the programmer strives to make the inner hierarchy stateless. Eric mentioned this in his blog about learning Haskell.
Iteration is less generic than recursion, thus less composable. The examples in my reply to Jessica show that Iterable isn’t as general a model as Applicative.apply. I can reuse the same apply in all examples, yet the code employing Iterable has to drastically change for each case.
This derives from modeling computation as functions with defined input and outputs instead of as blocks of instructions with undefined inputs and outputs.
I will continue in my next comment. Want to break this reply into more manageable chunks.
@Patrick Maupin
Time and memory-space indeterminism.
I don’t think intelligent people are arguing that it is not applicable in some use cases, rather the argument seems to be that it is not the appropriate default for a language. And I distilled it down to the generative essence that humans don’t think coinductively (c.f. my link in prior reply to you).
Higher-kinded genericity is too difficult for the human mind to track pedantically. Once you violate the category, you will have bugs that are difficult to reason about, because our mind doesn’t think naturally at this higher-orders of typing relationships. Ditto it will difficult to design comprehensive unit tests, because the higher-order combinatorial semantics are not easily flattened.
I am nearly sure we will need static typing, but I am open to opposing evidence.
In Scala, you can opt out of the static typing jail where needed– cast to Any. However, no doubt that Python is better as a default dynamically typed language.
Or you could declare these assumptions in the types in your input. So the caller has to enforce them. Then your caller can force these assumptions on its caller, and so on up the hierarchy.
I think we should declare invariants in the types. When we need to break out of that jail because of some impossible incongruence, then generate an exception. But please be nice, return it, and don’t throw it.
Oh my the obfuscation. I would have no clue what this is doing if you had not told me.
Perhaps I would employ something more declarative, that is more general than Iterable.
Or the following translated to the above.
But I don’t like it still. So read on…
That is dependent typing. Scala has a limited form of it. Dependent typing at its extreme requires giving up Turing completeness.
Instead of what I suggested above, I think the best solution is set your input type to a one element Tuple, i.e. in Scala that is Tuple1[T]. So you push the invariant up the hierarchy to your callers. This enables you to see at compile-time, what sort of jail your semantics actually require. And it enables some caller above in the hierarchy to optimize the case at a higher level.
Differentiate between good and prolific. My definition of a good programmer is one that produces code that works correctly. Someone might be prolific in turning out code that others can debug, and some might characterize this as a good programmer. However, that doesn’t mean that programmer is not also a good debugger. Generally speaking if they are good programmer it means they build a complete mental model and thus they can also debug well if they choose too. Not doing debugging could have been an opportunity cost decision. The qualitative word good implies an ability, not a choice made between held abilities.
Also I did not quote the paper. I made my own statement.
@JustSaying
The prior link for Why Not Events hides my own copy of my latest replies in moderator queue for days, including one I just made that delves into the essence of the logic for the definition of declarative programming.
I’m afraid this discussion is becoming a full time job, so I’m going to have to withdraw. However, one final thing about duck typing and all that loosy goosey declarative stuff..
The past few days I have been working with a declarative framework for html called angularjs. There is a lot to like about it, but I have had this unfixable bug that has been causing trouble for a couple of days. It hasn’t had the full glare of my attention, but it is a pain in the ass.
Basically I have a declarative line that goes like this:
If you are unfamiliar with angularjs the ng-model attribute creates a two way binding between a model variable (item.text) and the text box.
Problem is, that it wasn’t working. The value didn’t initialize, and didn’t save. Because it is declarative there is no place to put a before and after breakpoint to see what is happening. It is just an opaque black box that “doesn’t work.”
Well I finally found the problem. The property is called “title” not “text”. So the line should be:
This is classic javascript trauma. Any decent programming language would have immediately told you that there was no such property as text, and I would have saved two days of beating my head against a brick wall. Further, any programming language that allowed me to get a before and after snapshot would have enabled me to find the bug that way.
So the two things — undemanding, sloppy, duck typing, and declarative un-breakpoint-able code just cost me a couple of days of pain. And of course finding out it was so trivial is the most painful part of all. A little bit of redundancy and a little bit of procedural control would have made this problem a cinch to find and fix.
Would I have been happy to spend a few minutes entering a few type names to prevent a problem like this? You bet you ass I would.
@Jessica Boxer
Ditto myself. Thank you and Patrick for the discussion.
This discussion opened my eyes to some of the realities of designing a language for the mainstream. It motivated me to read the reasons Pythonistas rejected compile-time typing.
Perhaps the key insight I learned is Applicative.apply will need (via syntactical sugar) to look the same as a normal function call in order to motivate its use among normal programmers.
Jessica I hope you took away from the discussion that Applicative is the way to generalize reuse of modular functions that input interfaces (a.k.a. supertypes), for interfaces that each generic type does not require its type parameters to implement (i.e. be a subtype of). That is the elevator summary.
It is not loose concept, rather very precisely specified domain in the preceding paragraph.
Btw, I don’t know if C# supports a similar upper bound syntax as Scala where the generic type can require the type parameters to be a subtype of specified type as follows[1].
In Scala, generic types parameters are contained in square rather than angled brackets.
Note based on feedback in this discussion, I think the design of a language should only employ symbols where they are frequently used in cases where a lot of verbosity is reduced, thus I would prefer the above to instead be as follows.
[1] Note Scala has compound types (i.e. conjunction of types), so the upper bound can be restricted to a conjunction of types. But Scala does not have the disjunction of types, at least not as a first-class type. Ceylon has the first-class disjunction of types, but does not have higher-kinded generics.
I lean towards agreement with the benefits from compile-time typing, as ostensibly do a supermajority of programmers[2].
Assuming that angularjs library is implemented in JavaScript, and thus is parsing the HTML ng-model attribute in JavaScript code, you could set a breakpoint in the library where it parses.
But this is not the fault of a declarative style of programming. The bug is not yours, rather it is in that library’s parser which is failing to check all of its invariants.
And it is not the fault of duck typing, which can be typed checked at compile-time or run-time. Rather it is the fault that JavaScript is not strongly typed– but can be added with for example StronglyTyped: A library for strongly typed properties & constants in JavaScript. Python would have generated a run-time error because Python is strongly typed.
[2] See the Comparison of programming languages section in the tiobe.com index.
@Patrick Maupin
I suspect the meaning is subtyping applied only to classes, i.e. excluding interfaces and built-in a.k.a. nominal types?
@Jessica Boxer
Let me relate my recent experience with developing a cross-brower extension using the kango framework. The JavaScript code runs in the context of the brower, not the web page (except for the scripts I injected into the web page via document.createElement(‘script’)). The Google Chrome extension was working fine and I had been able to debug the JavaScript in the browser, but the Firefox extension using my same JavaScript code was silently failing and the darn development tools for Firefox don’t debug the scripts in the context of the XUL chrome (at least not that I could figure out quickly).
So I resorted to alert() calls for debugging. I isolated that the problem was that in Firefox, JavaScript does not allow you to call a function that is defined further down the source code. I think I had known this before, but I haven’t been doing JavaScript for a couple of years. Little details like this, that should be supported by every language, or at least consistent across all JavaScript implementations. And then I come to find out that some of these rules for JavaScript change in Firefox depending on whether the containing page is in “standards mode” or not.
Loads of extraneous cruft, caveats, and sloppy programming out there in the real world…
@me
Improved.
Or if we want it to read more verbosely as a sentence.
I think it is more lucid with equals as a symbol, and perhaps : instead of then.
@Patrick Maupin
And note `forall` would not work in place of `for`, so I agree with you that `for` is better.
Also the `begin` expression should allow a tuple of initializations, and the expression between `for` and `in`, should allow destructing the element’s value into variables with Constructor patterns, Sequence patterns, and Tuple patterns.
Here is a feeble attempt to tie in the original conciseness point that drifted onto declarative language design into the former topic about the Dunning-Kruger effect. And hopefully provide some distillation of our discussion comparing Python, imperative vs. FP, and inputs from C# world.
@me
Perhaps this also applies to Eric’s recent point (not sure if it was in this blog or prior recent one) that most technological innovation comes from those who find it by doing, not from the ivory tower of academia.
I have been arguing in the previously linked Why Not Events blog with someone who I believe to an academic about the correct definition of the “declarative property” for programming.
He ostensibly wants orthogonality so much, that IMO he is ignoring the reality of the universe as I see it in my updated definition from my perspective in accomplishing real world programs with significant users.
The definition is important because it impacts where we go looking for ways to improve declarativity. Otherwise one can waste time barking up the wrong tree– there is no cat there.
For example category theory (e.g. the Applicative) trumps communitivity and idempotence in terms of gaining consistency and thus the absence of accidental semantics while expanding generality and modularity, as I explained in the prior comments above.
With the Applicative, I can now remove all the cruft and make functions calls on lifted (parametrized) types syntactically the same as function calls on non-lifted.
I want advancements I can apply in the real world, not some academic theory which is vacuous in the real world.
Oh and to tie the knot, modularity is a form of getting the same behavior from the same programming construct. Yeah the specific behavior varies according to the variable plugin code (seemingly in conflict with my definition for the casual thinker), but the deeper thinker realizes that the modular portion of statements remains consistent with its semantics (thus not a contradiction of my definition). This is why the Applicative is declarative programming.
@JustSaying
>foo = (val = val is none ? elem : break) for elem in bar begin val = none
Honestly, we live on different planets if you think this is clear. Just the use of the ternary is bad enough, and this is even worse because you mix an expression and a control flow in the ternary. Even the worst C never did that.
I’m not even entirely sure what this does, but I think this is the equivelent C# code, to save wrapping in a function I’ll do a concrete type, plainly you can make it generic.
This is plain and straightforward. Even a college grad would understand this. Yours is just an obfuscated tangle.
>modularity is a form of getting the same behavior from the same programming construct
I don’t know of anyone else who would use that definition. My judgement of whether a module is sufficiently cohesive and externally decoupled is a simple one — can you give it a simple, all encompassing name? I know what a List is. I know what an Account is. I know what a GPSSensor is. I know what a Dialog is. To this day I am still not sure what “Applicative” is. In fact, it isn’t even a noun, except perhaps figuratively. Modules, in my experience, are always nouns.
Apologies to go on and on. I have some appreciation to toss back at Patrick and Jessica.
@Jessica Boxer
@Patrick Maupin
Thanks to your real world feedback, I have contemplated a solution which I think is different than we’ve seen in any mainstream languages?
1. `expr ? expr :: expr` will be if-else but only allowed if all on same line., c.f. my comment on March 11 2013 at 10:50 am.
2. Otherwise, `expr ? expr` (`? \n expr` is allowed) must be followed by either `end` or `else` on next line.
3. Blocks can thus be identified w/o indenting and aforementioned end-to-end principle is maintained, but compiler will enforce indenting too.
4. `expr ?= expr` sets the enclosing assignment or return value.
Note #4 is useful for avoiding unnecessary else blocks and note that control structure is an expression in modern languages.
@Jessica Boxer
Sorry if I drew you back into unproductive discussion, so let me try my best to make this productive for you! I really hope (and think) I can!
So I proposed.
And you attempted an understanding.
That is close. Follows is my intended meaning.
So it seems you basically got it, except inverting the conditional test and thinking `val` is a list. I don’t know what made you think `val` is a list. No where in my proposed line of code was I adding `val` to a list. The parenthetical is only to make the precedence clear that the whole chunk is the operand of the `for` operator.
I don’t know why you say my proposed syntax is not clear, when in fact you are able to grasp it already?
I think you mentioned up thread that you don’t like dense one-liners?
You could invert the order of my proposal and it would look nearly the same as your code.
I am not sure which order is more clear to more programmers? Python uses the former.
Expand with whitespace.
Okay you don’t like `break` in an expression which is a value. It simply means break before assigning the value. In modern languages where everything is an expression, then `break` is always going to have that meaning.
The ternary conditional operator is only bad when it is nested, crams too much on one line, or when your debugger is stupid and can’t break twice on the same line, once for the conditional test and again for the return of the test.
I required the ternary to be used only when all can fit on one line (and I forgot to mention not allowed to be nested).
Agreed, you are providing examples that agree with my statement.
It simply allows you to make for example a generic List interact with functions that operate on the type parameter of the List. So if you instantiate a List containing PlusMultiply elements, then you can call input the List into functions that input PlusMultiply, instead of having to write case specific boilerplate each time to extract the elements from the List and call the function on each element.
You proposed instead that you could just inherit the List from PlusMultiply, and then the function would know how to operate on Lists. But the problem is that Lists can be instantiated with any type of element, not just PlusMultiply. So your proposal is not modular. You fuse the type of the type parameters to the implementation of the list. You have lost the genericity.
Correction, I missed one case. We don’t want to have to put a verbose `end`, when we only have a simple if-no-else case.
1. `expr ? expr :: expr` will be if-else but only allowed if all on same line and never nested. The else `:: expr` is optional.
2. Otherwise, `expr ? \n expr` must be followed by either `end` or `else` on next line.
Note in modern languages like Scala everything is an expression, even a block of line statements that evaluates to the value of the last line of the block.
Btw, the reason for tossing `if` is it is superfluous and I don’t want to waste `:` character as Python did nor require a parenthetical as in C derivatives. When I was reading Guido’s proposed syntax for static typing, he was unable to use the colon on return types, because of their prior use of the colon to start blocks. Any way, this is getting too detailed for this blog, so I won’t expound further. Just wanted to send some appreciation back to the discussion.
@JustSaying
>You proposed instead that you could just inherit the List from PlusMultiply, and then the function would know how to operate on Lists.
Well I was totally baffled by that. But for what it is worth you can do precisely that in C# with the rather obscurely named function Select (which is a function in the IEnumerable interface, so automatically works with all enumerable items):
Many programmers implement an extension method on IEnumerable, so that you can do this:
Which is clearer but not part of the standard library for various pedantic reasons.
Or you can use a sql like syntax, viz:
var result = from item in getSomeSampleData() select functionToApply(item)Which some people prefer, though I think it is a little obscure and annoyingly different from actual SQL as to be confusing. (I had to look up the syntax because I never use it.)
And really, you can’t seriously think that the tenary operator is anything except horrible. It is the very poster child for what not to do in C.
@Jessica Boxer
But that isn’t good enough, because Applicative knows how to handle a function of any arity. So the Select would never to be overloaded on every possible arity.
Even then, it would not be good enough to compete with Applicative. What does your Select do if the type of the items in the list are nullable (i.e. can be a value or null/none) and the function inputs only values? So then you have to put a bunch of boilerplate inside.
var list = getSomeSampleData(); var result = list.Select(item, item2, ... , itemN => if (item != null && item2 != null ... && itemN != null) functionToApply(item, item2, ... , itemN));But still that would not be good enough to compete with Applicative. Because Applicative knows how to work with new semantics something like nullable but different in ways we can’t imagine and we don’t know how to write the boilerplate for (due to open extension we may not even know the subtype that we are operating on).
So sorry, your idea does not address the genericity case I mentioned.
In what I proposed, the abuses that make it bad are disallowed. Do you just hate the question mark?
I love the terseness when it is not abused, because I want to focus on the conditional expression, not a superfluous `if`.
@JustSaying:
One of the things I, personally, like about Python vs. (for example Perl or ruby) is readable words…
@Patrick Maupin
Yeah I remember you mentioning that prior. But I don’t think you like Python then because it is uses a nasty colon! (we can barely see those darn things:).
So I changed the colon to something more visibly catching which implies a test of a question– the question mark ?. Okay I realized Python wants to emphasize the `if` and hide the `:`– a reasonable stance until…
The other motivation is that if we make expressions that can comprise a condexpr use words such as `for`, `in`, `begin`, then the start of the conditional test gets lost visually in a sea of gibberish verbage. I saw that in prior comment where I replaced the ternary that was embedded in an assignment with a `if` and `then`. I could no longer read it quickly, at least for me.
@JustSaying
> because Applicative knows how to handle a function of any arity.
I have no idea what “arity” means.
var list = getSomeSampleData(); var result = list.Select(item, item2, ... , itemN => if (item != null && item2 != null ... && itemN != null) functionToApply(item, item2, ... , itemN));But if your code solves this without a guard then you have the very essence of non-cohesion. You have two operations here: what do I do if the arguments are null, and what to do if they aren’t. Apparently your code assumes a particular meaning in the first case which you can’t possibly know ahead of time. The solution in C# is simple:
Here the two operations are separate and separately specified. Of course none of this has much meaning because it is a vacuous theoretical example. If “item1” or “item2” or “functionToApply” were real things rather than abstractions then we could have a serious discussion about what all this really means.
All you are claiming is to have some fancy syntax to automatically deal with some highly specialized case: filtering out null arguments in a strange sort of a way. It isn’t apparently to me that it has any real world applications. It is very much along the vogue of “inheritance is where you have a base class animal that has a walk() function, and derived classes dog, which has a bark() function and cat which does not.” Which is to say theory not grounded in practice.
Why don’t you write some real, substantial programs, then tell us how 3rd order abstraction saved your butt?
To me, I’ll take 1000 lines of plain, simple code, over 10 lines of hyper-tangled, super abstract complexity any day. The implementation of the idea should be so plainly simple that I don’t have to think about the language encoding, I can simply think about the functional problem. I’m fairly smart, but I can barely decode your ideas. I assure you, most of the coders I work with wouldn’t have the first clue what you are talking about.
I had to look it up….just…*had* to…
*
* http://en.wikipedia.org/wiki/Arity
*
;P
@JustSaying:
Non-syntax-highlighting editors are so early 90s.
@Jessica Boxer:
I happen to like the ternary operator, but then I do lots of low-level bit twiddling.
The Python ternary took a bit of getting used to, but is very comfortable once you have used it a few times. The equivalent of:
is:
My understanding is that Guido was dead-set against the ternary for the longest time, but Python logical operators return one of the operands passed to them:
So naturally, it was canonical python to abuse this to fake a ternary:
Which is all well and good until one of the valid values of a evaluates to false. Naturally, this is just asking for trouble in a lot of contexts, so the right answer was to develop a ternary and promote that.
@Dan:
So was Medusa an arity-head?
She had a serpent-ary noggin, for sure.
@Patrick Maupin
N-ary was it his unary-ness Jordan?
I feel the successor to Brainfuck approaching out of the proverbial hazy conceptual flog.
I was expecting that throwback to the 60s response.
There are only 3 or maybe 4 words below that are not keywords. Shall we will color the rainbow psychedelic?
I couldn’t resist incorporating your Python ternary in the former example.
Now I vaguely remember reading that about Python. Python seems to prefer grammar that reverses the logic of the steps the computer does, c.f. the `expr for expr in expre`. Doesn’t that go against your argument that the imperative programmer wants to be able to walk through the steps in their mind?
It reminds of adjusting to filipino language (Tagalog and Visayan dialects) where they reverse the order of the predicate and subject, e.g. “Malaki ang bahay” instead of “the house is big” or in “supsupan mo ang tutoy ko” literally translated “will suck your tits I”.
Ah an overload of boolean operand semantics. What is the legitimate use case?
It has the downside of not coercing operands to boolean or generating a compiler error at compile-time, which Python can’t do and so I venture a guess this was to prevent throwing a run-time exception?
@Jessica Boxer
Yeah I forgot to write the else case that returns null to Select. (but all of it is wrong any way, see below)
The person who made the decision to apply nullables to functionToApply, can also chose to make the decision to test for null before calling, if they want an alternative semantics to the default in Nullable.apply. Or I as I had suggested up thread, apply could be overloaded to take an optional additional operand to specify alternative semantics.
I doubt I can predict all the ways programmers can use increased degrees-of-freedom. The Applicative is an additional degree-of-freedom (dimension) in the genericity.
If we mark elements of an array or hashap with null to indicate their index or key is valid yet valueless, the Applicative enables skipping the non-set elements while applying operations to the set elements, without any boilerplate. And more importantly it can automatically handle any Wrapped semantics that are unknown a priori, i.e. operations on T over Hashmap[Wrapped[T]].
I can already picture how it will make my code very, very easy to understand. I can focus on what the semantics is doing, not on all the cruft to get me there and my semantics will be open under extension of Wrapped whereas all your example code isn’t.
Significantly, I forgot to point out before that your prior examples do not emulate the return value of the Applicative, which will be a List, not a Nullable. So even my attempts to repair your examples were not emulating Applicative.apply.
Remember the functionToApply is the input to the Applicative.apply, so the container maintains control.
Equivalently.
And my proposed syntactical sugar which the compiler would translate to the above.
Exactly.
I enjoy the humor about “exactly” in this linked article about edumacation (Ctrl+F is our friend).
Probably because I haven’t written down the Applicative in your native language C# so you could realize how simple the concept is. I predict you will “Ah ah!”. You are in a fog as to the machinery of the Applicative.apply (as I was too!) because you haven’t seen the code for Applicative. And the research paper is in Haskell. I had to spend considerable time translating that Haskell to code that is native for me. Unfortunately, C# can’t express the code so elegantly because AFAIK it does not have higher-kinded typing. But there is probably some clever way to express it in C#. Sorry I am only superficially conversant in the C# syntax.
“supsupan mo ang tutoy ko” literally translated “will suck your tits I”
I know where I’m going on holiday next time :D
@Jessica Boxer
I have tried to write some pseudo-code which hopefully similar enough to C# for you to read it. The Haskell and Scala code for Applicative uses type classes, which are confusing IMO. I have translated to the approximate syntax I want for Copute, which expresses the construct in more familiar inheritance.
class List<T> implements Applicative<List,T> { List<A> apply<A>( List<T => A> liftedFuncToApply ) { List dotprod = new List(); this.foreach( elem => liftedFuncToApply.foreach( elem2 => dotproduct.append( elem2( elem )))) return dotprod } List<T> lift( T unlifted ) { return new List( unlifted ); } }There are several confusing points to clarify about the above.
1. Note how the type of List is a type parameter of the type it inherits from. This is higher-kinded typing, and it is necessary so that Applicative.apply can return a type of itself, i.e. the interface (or abstract class) for Applicative is as follows, so that our code can operate on any kind of Applicative not just known concrete subtypes of it.
interface Applicative<Sub<T>, T> { Sub<A> apply<A>( Sub<T => A> liftedFuncToApply ); static Sub<T> lift( T unlifted ); }2. The syntax I am using for the type of a function above is T => A, where T is the input and A is the return type. But it is optionally a n-ary function, note that A can be sometimes a function type, i.e. B => C => … to any arity (of the function applied where for the last apply will be the type will be the type of the last argument to the function applied)
3. When elem has (i.e. T is of) type Applicative, the call elem3(elem) in my proposed syntactical sugar would be converted to elem.apply(elem.lift(elem3)). This is how when T is Nullable, they get handled as I said my prior comment.
As you can see, there is no magic. You just need to study the above code and wrap your mind around the implications.
@Dan
I don’t know your persuasion, but the supsup sounds enticing if it is not a ladyboy with bolitas.
Dan I had a short reply for you, but it is in moderator queue.
@me
Applicatives generally compose, i.e. Applicative<T>, Applicative<Applicative<T>>, Applicative<Applicative<Applicative<T>>>, etc can be applied to any function that inputs T. Monads do not. IMO, the linked Scala code is confusing, because it use type classes.
@Jessica Boxer
Here is an example abstracting validation in Scala, F#, and C#.
Applicative.apply lifts the return type to the subtype of our Applicative, i.e. lifting functions to the type parameters. Thus the container type gets master control over how its elements are used as inputs and outputs to the iterative partial application of functions (that input the type of its elements). Monad.bind parameterizes the control structure. Different types of functors. Practical uses of applicative functor.
@JustSaying:
You can if you want. I’m happy with no highlighting, or with just about any highlighting I’ve seen. Usually, it’s keywords one color, built-ins another, comments another, perhaps numbers and strings another, and then everything else.
If you say so. I assume that equals is the same as ‘=’, but who knows? And I know you were discussing this with Jessica, but the conflation of out-of-statement control flow with assignment is probably not desired by most programmers.
Possibly on the for, but as long as it all fits on a line it seems to be pretty grokkable as a single loop.
Not really on the ternary. Depending on the assembler, “a = b if c else d” could look something like:
move b -> a
if c goto done
move d -> a
done:
As I mentioned, _before_ the ternary operator was added, it was legitimate — canonical, even — to use this sort of thing to emulate a ternary:
a = b and c or d
This would return c or d. The potential problem was whenever it was possible for c to evaluate to False. So then there would be ugly workarounds, e.g.
a = (b and [c] or [d])[0] # Always works right because [c] never evaluates False
Of course, with the ternary, you don’t have to worry about this.
Shor-circuiting certainly allows you to avoid run-time exceptions and to avoid doing extra work. It is obviously available in lots’o languages.
But I don’t know why the boolean ops don’t return boolean values. Perhaps it was to allow this pseudo-ternary; perhaps it was merely an optimization or easy way out in the interpreter to keep from having to create a new object and trash the old one.
@Patrick Maupin:
Applicative.apply is for generalizing the partial application of functions to parameterized types (a.k.a. generics) at any level of nesting (composition) of the type parameter. This is all about making more generalized composition possible. The generality can’t be accomplished by pulling it outside the completed evaluation (i.e. return value) of the function, analogous to the onion can’t be peeled from the inside-out.
Thus it isn’t conflation, it is a new degree-of-freedom that is not currently available to you. Per our discussion up thread, this is why you must throw exceptions or stored them in a global variable, because your language doesn’t have this degree-of-freedom. And that is not the only application of these category theory functors (expounded in my comment in moderator queue).
I provided a link to an example abstracting validation in Scala, F#, and C#, which is currently stuck in moderator queue. Compare the obnoxious C# version of the code. And the reason is because the C# is not generalized. I intuitively expect that C# case-specific boilerplate will explode geometrically as the program grows.
We shouldn’t make every reserved syntactical entity an english word. Balanced use of symbols aid rapid comprehension via delineation. We need a balance between the code above and Brainfuck (where everything including literal strings are expressed with one of 8 symbols). There are three axes in this domain– the number of keywords, whether they are symbols or words, and whether they are context-free (not overloaded to have different meaning in different context).
For one liner ternary (I would enforce one liner for ternary), I see a terseness and visual delineation advantage for `expr ? expr :: expr` versus `expr if expr else expr`, because the `expr` are not likely to begin or end with a symbol– a probability function of the frequency of use of (pre- and postfix) unary operators. I think it is a good PL design to eliminate all symbolic unary operators other than the negative sign `-`, e.g. `expr is not literal` or `expr != literal` binary operators instead of bang `!expr` prefix unary operator.
For the multiline if-else, I am conflicted. The advantage of `?` instead of `if` is consistency with the `? ::` ternary and replacing the colon `:` of Python (or the parenthetical of C-like) with a symbol that is more meaningful w.r.t. to the boolean question “is true?” while eliminating the superfluous `if`. OTOH, the `if` is more consistent with `else` both in english and also in the column alignment in the source of the multiline if-else. So then we can make an argument for the Python choice, because it maintains consistency with the `if else` ternary and the multiline if-else, but then as a trade-off it needs the additional colon and loses the `? ::` ternary’s terseness and probablistic delineation.
@me:
The latter idea won’t work so simply, because the Applicative is composable to an unbounded depth, as shown in the former. The caller of Applicative.apply would need to specify pluggable semantics for every subtype of Applicative in the nested composition in the type parameter. If the language is statically typed well enough (so all the structure of the type parameter is known at compile-time), then this latter idea might be feasible.
@Patrick Maupin:
Perhaps I have devised a better control structure syntax than Python. I think this addresses Jessica’s criticism of ternary?
I wrote:
There is another axis for this aspect of the syntactical domain– all caps only for keywords.
The problem with relying on code-highlighting is the same as relying on block indenting without redundant delimiters– as I claimed before, the reliance on interpretations not in the file violates the end-to-end principle (the text file contains only text).
I decided that every control structure must begin with a keyword, otherwise the LL(k) grammar needs backtracing and/or semantic analysis to distinguish from being a function argument of/with the preceding expr, given function arguments separated with spaces when not parenthesized. And line endings with semicolon wouldn’t fix all cases, plus I hate the superfluous noise of semicolons at end of lines. Note although contemplating to support both parenthesized and non-parenthesized argument lists, I am not proposing to support Scala’s obfuscating infix style for function names used as prefix or binary operators.
The decision solves the lack of columnar alignment of keywords for the multi-line if-else:
It also eliminates Python’s style of ternary, which is good, because I don’t like the inconsistency between the `condexpr if expr else expr` ternary and the `if condexpr: expr else expr` if-else. Again the problem being as stated above, that the ternary won’t always be following an `=`, it might be following another `expr` and thus the ambiguity over which part of the `condexp` is actually an argument(s) of/with the preceding `expr`.
Thus the proposed single line ternary if-else:
Is not inconsistent with the four allowed variants of the multi-line if-else:
The END is necessary when `block` is not `expr` on same line as `ELSE`, because of the redundancy needed to respect the end-to-end principal, even if we are enforcing syntactical block indenting.
There are also `IF` without `ELSE`, and unlike the preceding, these can not be the LHS of assignment nor a function argument:
I prefer DO over `?`, `:`, or `THEN`, because it maintains the consistency of tersest english keywords and works for the other control structures.
Correction:
I appreciate Patrick turning me on to the use of short english words over symbols. This choice of `DO` (and using `END` instead of braces only where the block begins on a new line) seems to play well with Scala’s pattern matching (more powerful analog of switch-case), which I think it is an important feature for the next generation typed programming language. Instead of `match { case … case … }`, I propose:
The above in Scala:
mammal match { case x : Cat => x.driveByLegs case x : Dog => x.makeSadEyes case x : Ape => x.pickLice case _ => mammal.sound } fold((s,0), f) match { case (t,_) => t case _ => none }Also then the `expr IS expr` is an binary, infix operator that returns a boolean, so it can used in the condexpr of `IF condexpr`. This is a more consistent syntax with only one way to do each logic, compared to for example Scala’s `InstanceOf`.