LISP — The Language That Will Not Die

I’ve spent large parts of the last week editing maps for a game
system I’m working on. I’ve been using the GIMP graphics editor, and I’m pretty
impressed with it. I haven’t found anything I can’t easily make it do
— except, oddly enough, draw straight lines between defined
endpoints. (I suspect there’s actually a way to do this using the
path facility.)

I have a requirement to prepare about six different variants of a
base map, using the same topographic map but with different
arrangements of national borders. I’ve handled this by creating a
multi-layered XCF file with the topo map as the background and the
different borders as optional overlays.

OK, so I save the variants to flat PNGs by hand whenever I change the
image, but that’s a pain. What I wanted was a way to put in my makefile
instructions that say, for each variant map, that it depends on the XCF
and the way to make it is to composite a particular selected subset of layers
by running GIMP in batch mode.

Fortunately, GIMP has an embedded Scheme interpreter that’s good
for exactly this kind of thing. Looking at some Python-Fu code by
Carol Spears taught me enough about the API to get started; the fact
that I’m an old LISP head got me the rest of the way.

Here it is.

;; Batch-mode select and save of a layer set as a PNG.
;; Has to be copied into  ~/.gimp-2.2/scripts to work
;;
;; Note: This assumes that gimp-drawable-get-name returns a list with
;; the actual string name as its car.  This is what gimp-2.2 does, but
;; not what the documentation says it should do!

(define (layer-set-saver infile select outfile)
   (let* ((image (car (gimp-file-load RUN-NONINTERACTIVE infile infile)))
	  (layers (cadr (gimp-image-get-layers image)))
	  (ind 0))
     (while (< ind (length layers))
	    (let* ((layer (aref layers ind))
		   (layer-name (car (gimp-drawable-get-name layer))))
	      (gimp-drawable-set-visible layer (if (member layer-name select) 1 0)))
	    (set! ind (+ ind 1)))
     (file-png-save-defaults RUN-NONINTERACTIVE 
		     image 
		     (car (gimp-image-flatten image)) 
		     outfile outfile)
     )
   (gimp-quit 0)
   )

Here's one of my makefile productions. The second arg is a list of layer names.

basic.png: basic.xcf
	gimp -i -b '(layer-set-saver "basic.xcf" (quote ("topographic" "skinny-borders" "grey-switzerland")) "basic.png")'

Apologies for the long line.

LISP truly is The Language That Will Not Die. And that’s a good thing.

56 comments

  1. Lisp won’t die because it (particularly in its Scheme variant; whoops, my bias is showing) is something approaching a platonic ideal of algorithmic expression and meta-expression.

    What kind of game system? It sounds like perhaps a tabletop or web-based war game or political game.

  2. I am trying to get into lisp. I understand its concepts well and its ways are not strange to me. But the state that it and its community are in make in exceedingly difficult to jump in and start hacking. This saddens me much, and it’s also why I still use Python when I need to get something done, even though I can’t stand Python. It’s just *convenient*.

    Eric, please tell me why this is and what I can do about it.

  3. I am not sure if this is the answer you were looking for in making a straight line in gimp, but you can do it freehand as follows:

    Click the left mouse button on the first point of the line, hold the shift key, and move to the second point on the line–click again.

    Thanks for all the open source contributions you have made over the years. I am but a humble user as I do not know how to program, but I promote open source where ever I can.

    Happy Thanksgiving.

  4. The code is for mediating games in the style of Diplomacy. There’s a Python-Fu binding to GIMP, but it’s (a) poorly documented, (b) would introduce an additional dependency, and (c) the author’s FTP site was refusing connections when I tried to get it.

    I don’t know how to fix the LISP community’s problems, alas.

  5. *twitch*

    I could probably read the Python equivalent even had I not known Python. That code chunk is utterly opaque to me.

    Just what is it about LISP that makes it so magical?

  6. If I understand Paul Graham correctly, all other programming languages are just implementations of the Universal Turing Machine, while LISP in itself is a computation model different UTM, and each program written in LISP is supposed to be an implementation of that LISP computational model – each program actually a different language. Is that true?

    If yes, it may mean LISP will not die but I think it’s representation should change. If I understand LISP correctly, then it seems structure in LISP is more important than in other languages: you start reading an S-expression from the inside, you have an assumption on what it does, and then when you reach the outmost S-expression, it turns you assumptions upside down. And enrolling this all in another S-expression can lead to a wholly different feature too.

    And if the structure is so important, then it should be noted: for example, a LISP IDE where all S-expressions in each other would have different background color… I think LISP could easily be visual. Where structure is very important, visual tools could rock.

    Actually, sometimes I think LISP was designed to be represented visually, we just somehow seem to have forgoten it. Mostly because the real world is visual and LISP is I think a good representation of the real world. If Java is theist, LISP is definitely buddhist.

  7. esr: playing at being a dove instead of a hawk for a change? :)

    Will the game make it online or is it a proprietary trade secret?

    Shenpen: Scheme is based on the lambda calculus. Church and Turing showed that lambda calculus and UTMs are equivalent, of course; and any language which has a SET! operator is not entirely lambda calculus. What makes the Lisp family different is the fact that the model can be extended into the evaluator or compiler itself which C++ strains to do with templates.

    I cannot see the justification for your claim that we’ve “forgotten” how to work visually with Lisp. Structural editors (of which Taylor Campbell’s paredit.el provides something of an approximation) have been around in the Lisp community for decades, and it isn’t terribly hard to find or write an emacs hack that does syntax highlighting based on depth or type of s-expression. Overall I find paredit combined with the default Emacs indentation to make my code’s structure visually manifest.

  8. John, I think Russ is closer to being right than you are. I agree with Peter Norvig’s evaluation, “Python can be seen as […] a practical (better libraries) version of Scheme” Read it and the supporting argument here.

  9. Except that Python hasn’t any support for continuations, nor a macro facility. And Guido hasn’t found any use for lambda expressions; therefore they are useless and will be removed from Python.

    Python is only like Scheme if you write Scheme the way you write Python. Hard to avoid in GIMP Script-Fu (which is not based on a modern R5RS Scheme); but other Schemes offer flexibility way beyond anything else I’ve ever seen, and many have extensive library support. SISC, in particular, is a complete R5RS Scheme based on Java, with a generics-based object system that integrates seamlessly with Java objects.

  10. If anything, then this excellent online book can revive LISP: http://www.gigamonkeys.com/book/

    About the visual stuff: you know I always felt that Gibson (Neuromancer) was essentially right when he felt like like representing network and program objects as 3D objects in space – program design appears to me as visual workflow in space in my mind, and then I translate that to code the way one describes a painting in words. Is it a general experience or am I just a right-brain-hemispherer? I accept that visual programming like connect-the-dots in Delphi is very lame, simply because the mouse/trackball is a clumsy input device and the keyboard has a lot more bandwith, however, there is something in me that says programming should be more closer to the visual representation of the workflow. And if there is any language that could be starting point for this, it could be LISP as in LISP the structure of the workflow is naturally-visual. What I am dreaming about is an input device like in the (excellent) film Minority Report – moving electronic gloves in 3D space and drawing up 3D objects that each would be an S-expression… is it too bold? :-)

  11. Shenpen:

    …or like Rez, the best video game ever.

    I’m feelin you, but I don’t think we’re anywhere near a total-immersion eye candy IDE that can come anywhere close in terms of efficiency, ease of integration, or ergonomics (when designing your Minority Report interface, remember the gorilla arm!) to good old Emacs, optionally enhanced to taste with syntax highlighters, tools for sexpr manipulation, etc.

    The place to look would be toward the Squeak and Croquet projects (although Smalltalk and not Lisp is their host language). There has been a lot of interesting work in that regard; by the same token they’ve had to reinvent quite a few wheels to tackle problems solved decades ago in the world of plain text.

    As for me, program design embraces and transcends the visual, approaching the tactile and kinaesthetic; on the other side of the spectrum we owe the development of programming *languages* to textually-minded women programmers who tired of wiring plugboards and wanted simply to *tell* the computer what to do. Mileage definitely varies here.

  12. I find it hard to believe that Guido would remove lambdas; whether or not he finds them useful, removing them would break a LOT of existing code.

    (FWIW, I’m not sure *I* find them useful, either, but that’s probably because I’m not a LISP-head. To me, if something needs to be a function call, it needs to be a true function, for readability and maintainability.)

  13. Heh, when I saw that “grey-switzerland,” I said to myself, “that must be diplomacy.”

    If you need players (you probably don’t but I figure I’d ask), I’d love to join in!

  14. Guido is not going to remove lambdas. Among other reasons, I told him that he’s get my lambdas back only when he pried them fom my cold, dead fingers…

    One great use for lambdas is for defining callback hooks. There are others.

  15. Maybe he’s not going to remove them, but he’s not going to uncripple them, either. There are basically arbitrary restrictions on what you can and cannot put in a lambda.

    Python is probably the most lisp-like mainstream language. But it leaves *much* to wish for if you’ve used an actual lisp. Plus, Python makes a lot of easy things hard. For instance, what does 2*(1,2,3) evaluate to? Not (2,4,6), but (1,2,3,1,2,3).

  16. Jay Maynard:

    Guido toyed with the idea of abandoning lambda constructs for a hypothetical future Python version a few months back, that probably bears the same compatibility relationship to contemporary Python as Perl 6 does to Perl 5. But as Eric relates it seems he was suitably pressured by the Smug Lisp Weenies not to put it into practice any time soon.

    Speaking of smugness, AFAIK Python lambdas are not true closures — yet another point in Scheme’s favor.

  17. Actually, in the MS-DOS age, there was a programming language called Clipper, that not only supported lambdas (called “code blocks”) but heavily relied on them as many standard functions were either expecting them as arguments or returning them as a return value. A good example was the grid (“browse”) UI component, that expected a lambda as an argument that calculated the data to display in the grid column. Was an amazingly great idea… (Actually, a modernized version of Clipper, called CCC was recently open-sourced, but I will disclose the details only when we finish translating at least the basic documentation from Hungarian to English as people would quickly lose interest without it…) So anyway, I think lambdas could and should be used much more extensively than they are ususally used today…

  18. Not a clone, but an extremely productive application language tightly integrated with the DBASE databse, but yes, exactly – lambdas were called “code blocks”. For example, to break up a slurped CSV file by \n to lines, and break it up into columns by the “,” CSV separator character, you would write:

    aeval(wordlist(memoread(csvfilename),crlf()),{|x|aadd(csv,wordlist(x,”,”))})

    memoread meant to slurp a file. wordlist to break up a string into an array by separator characters – in this case , the \n character. If I remember well, in the original Clipper CRLF was a constant (remember, it was DOS), but in CCC the crlf() function returns the line separator on every supported OS (DOS,Windows,Linux,FreeBSD,Solaris).

    so, wordlist(memoread(csvfilename),crlf()) gave you an array where every item is a line. And then the aeval function evaluated (mapped) a lambda (code block) to an array. The code block syntax was {|parameter|return value}, in this case, to break up the parameter into an array by “,” -s. aeval calls it for every line stored, and then what you get is a 2-dimensonal array representing the CSV file.

    Lambdas were one of the reason Clipper was so a productive tool: it not only allowed it, but relyed heavily on them. For example, all the functions that added a column to a browse, that added a menu item to a browse, that validated user input on a mask, whatever, all expected a lambda as their parameter.

    This was what made Clipper extremely productive, and this is one of the reasons why we want to revive it as CCC. (Clipper to C++ Compiler)

  19. Actually, this is what was always heavily missing from the Open Source/UNIX world: a tightly integrated database, forms and programming language enabling you deliver good enough applications fast by simply making all the not important decisions instead of you, enabling you to concentrate on the business logic. MS-DOS had it with Clipper. Windows has it with MS Access. IBM had it with AS/400 and RPG/400. So it seems everbody has it, except Open Source / UNIX. Now, Ruby on Rails is finally something similar in (also) the Unix world, but it is too web-oriented. I do not share the enthusiasm of Paul Graham and others of web applications – Amazon is nice to order ten books a month, but if your job is to order 800 different books a day, then you will surely hate it, and long for a responsive and keyboard-friendly desktop (preferably, console) application. The web is simply not suited for really professional applications – CCC wants to fill the gap, to provide something like “Clipper on Rails”.

  20. Except for the fact that CCC has not much to do with Clipper nowadays. I’d say that CCC and Clipper have common roots, but that’s all. For instance, CCC is object-oriented, multi-threaded, etc.etc., which is pretty far from Clipper. But the initial point IS right, CCC relies heavily on codeblocks, mostly for callback functions.

  21. Shenpen, Unix hackers like loose coupling. Go read ESR’s The Art of Unix Programming, and then ponder why such a “tightly integrated” thing is not The Unix Way, whereas Ruby on Rails hits closer to the mark.

    There have been efforts to write integrated database/programming tools before CCC; not many have found purchase in the open source world. Unix has always appealed to a particular mindset; Linux and other open source operating systems appeal even more to that mindset.

    Your last statement reminds me of some statements that I’ve heard COBOLheads make; not surprisingly they say only COBOL can be used to build truly “professional” applications and then mumble something about floating-point roundoff errors (which aren’t a problem if you use fixed-point big integer math). The fact that serious, high-volume applications use the Web as a transport all the time, both on the internet and via corporate intranets, puts the lie to that statement which is clueless at best and somewhat insulting at worst.

  22. Yes, I know the “original” Unxi philosophy value freedom ove productivity, simply because Unix always been primarily a research environment. But now Linux became mainstream and considered by most users just as a better, more stabler and free version of Windows. Just look at KDE if you don’t believe me, or compare OpenOffice to, say, Tex. Or KDevelop to Emacs. And if “world domination” is the goal, it means Open Source has to break out of the traditional Unix cliches. Actually, it had already happened (KDE again).

    The Clipper (or RPG/400 or whatever) phisolophy is that freedom is a resource: you don’t need the freedom of meaningless decisions, you only need the freedom of choice where there is an important benefit in the making.

    Actually, Open Source can mean the taking away of unimportant freedoms and replace them by productivity. An example: in the closed source world, database independence is important, because you don’t want to force all those people who bought MS SQL to buy Oracle. But in the Open Source, you can just say that this application works with PostGres, and period. You don’t need the freedom to choose between MySQL and PostGres, and this in fact boosts productivity, as you are not forced to use a least common denominator (say, JDBC) but use the powerful features of PostGres. A live example is TinyERP. Also, it would not sound horrible to me to write Firefox-dependent web applications, as Firefox is for free and is for all important OS so there is no reason not to tell users to install it, if it has some features that makes application development faster. Why would you need to be foo-independent if foo is for free and for most OS-es? Another good example is Ruby on Rails: you don’t name your objects, directories etc. the way you want, but RoR names them for you. Simply because these freedoms also not important, and by taking them away and thus being able to make many, many guesses it can save you a lot of work.

    I know this kind of thinking will never be popular for research-minded people. But if you make a living of writing custom invoicing software, and you have a customer that is willing to pay only $1000 for it, then you will want to write it in a week, not in a month or more, and to the hell with freedom if hardcoded decisions in the language or enable you to do it in a week or less. This was the amazingly good idea with Clipper, that it finally could step over the idea that software developement is research, and they finally understood that most development is for providing users those simple tools they want to use, and 95% of this kind of development is the repetitive shit of making boring stuff again and again, because you are forced to make meaningless decisions again and again because the language and the tools you use are based on freedom, not productivity. However, I agree that this might hard to understand in the USA and Western Europe, as writing $1000 invoicing software was always looked down on. I don’t really get why – from a technincal viewpoint, they are boring, but from a human viewpoint, this is what people really need and willing to pay for. No wonder the other Clipper compiler for Linux, CLIP is from Russia. It seems Eastern Europe has more need for practicality and less need for freedom.

    As for the web, my point is that web applications are the second wrong step on the road where the first wrong step was a GUI interface. GUI’s provided a shorter learning curve but a slower way of using – web applications usually provide an even shorter learning curve, zero installation time – and an even slower way of using it. No webmail client ever will be as productive as a desktop client as 1) it cannot download mails beforehand to browse them fast 2) you cannot move with the arrow keys. Which means if you have 100 mail a day, it can mean a difference of one hour between a webmail and a desktop mail. As for non-GUI, console applications, comparing Thunderbird and mutt, it can again mean ten minutes in favour of mutt. Or just imagine a Shop Floor Control software. We are in a factory, with cheap 14” monitors, that are dirty that you can hardly see anything, and users are running around and stop for a minute to randomly beat the keyboard until the manage to enter data. Which is better in this environment, 1) a traditional green screen console app 2) a desktop app 3) a web app? You see, the web is simply not a for a really professional use. The error you are making is exactly the same as Microsoft is making with Navision/MS CRM/whatever: they think computers are used be well-groomed 30 years old salesmen, who comfortably sit in a nice office before a 17” LCD screen and have the patience to click around with the mouse in a nice environment. While in reality all important software in the world used by dumb people in noisy and dirty environments running around and stopping for a second before an outdated, non-brand computer to beat in some data.

  23. That discussion is moderately fascinating for me, as my father once wrote a payroll application for a company with very weird requirements in Clipper.

    Even so, the world is moving toward web applications as the norm in much the same way that it moved away from mainframes and toward Windows: because upper management became enamored of the idea while reading airline magazines and the Wall Street Journal, not because of any particular requirement that only web apps can fill.

  24. Shenpen, if you cannot see how separation of database access, back-end processing, and interface presentation is a Good Thing, from a *software engineering* standpoint, then you have completely missed the boat. If you cannot see how there are different ways of doing this that are 100% ncurses compatible and yet may *still work over the web* then you are clueless. I dare say you are making the same mistake all those COBOLheads made — the faulty assumption that the most important thing in the world to do with computers is CRUD-screen data entry and simplistic business calculations, and that this must always be done in certain stereotyped ways with certain stereotyped tools.

    As to Eastern Europe needing practicality more and freedom less, you can take a good long look at the unmitigated *practical* disasters that the Workers’ Paradises in that region have been and see where that attitude gets you.

    Microsoft Access is a total loss. I have never in my life seen a more broken, crufty, artificially limited piece of software. Clipper may be better. As for IBM… there’s a reason for the culturally ingrained Fear and Loathing of the IBM of the bad old days in the hacker culture. None of them will get us where we want to go, which is to *totally automated* factories where robots do all the work those users you’re so concerned about currently do. Hackerdom and open source culture will. Distributed applications, of the sort you’re slagging off as “unprofessional” will. That’s because hackerdom has the cultural expertise necessary to manage the complexity that such applications will require, whereas your average database-driven DP guy experiences brain core meltdown at the prospect of a problem which cannot be solved with the stereotyped tools and techniques that have worn deep grooves in him after x decades in a company somewhere. Hacking is about finding better ways to do things and it’s currently the only methodology in software development with the demonstrated capability to do this reliably; the only thing separating us from a more automated, efficient, and better society is people like you slagging off this methodology as “unprofessional”.

    I’ve noticed something about words like “professional” and that is that they really have no intrinsic meaning; they mean whatever the speaker wishes them to mean. If you can’t argue against something based on its merits, say it’s “unprofessional” and you’ve won the argument. Andersen Consulting (now Accenture) used the word to reinforce herd behavior within its ranks too. I still remember a decade ago when Linux and open source were just taking off. The writing was on the wall for Microsoft and the old guard, and the best that the suits who made a living off MS with could come up with was that Linux lacked an easy-easy GUI, and so therefore was… wait for it… not suitable for professionals. (Meanwhile, pro athletes engage in unprofessional behavior all the time — partying to excess, womanising, shooting up performance-enhancing but life-shortening substances — yet they are never called “unprofessional”. Go figure.)

  25. Jay Maynard:

    At least those managers reading the airline magazines are thinking “distributed” instead of “monolithic” for their large applications! Distributed applications can fill niches that monolithic applications can’t; however “Web apps” are not so special. The Web is just something people are familiar with and all the protocols are in place, and plus, many people think “internet=web”. It’s a suit-compatible wrapper layer for some of the easier bits of cutting-edge software, much like “Agile Programming” is a suit-compatible interface so that hackers can do what they always have done without being called “unprofessional”.

  26. The problem is that there are niches that monolithic applications fill better than distributed ones, and yet we keep hearing “imminent death of mainframe predicted”. That’s been a drumbeat for at least 20 years now, and yet it’s not dying – and folks that try, more often than not, to move away from mainframes find themselves moving back.

    There is no one right approach to all problems, neither distributed, nor monolithic, nor web. Every problem has its best approach. *THAT* is what I find aggravating about “management-by-airline-magazine”.

  27. Jay, I’m in agreement with you; however, distributed and networked applications are taking on an importance that they’ve never really had before outside of universities and research; part of the reason why mainframes thrive today is that the networked application sphere is *including* mainframes, and playing to their strengths, rather than attempting to supplant them. The importance of networking becomes really manifest — and not a curiosity or an e-commerce toy — when you consider how many individual CPUs must be linked together to make the dream of a fully automated factory a reality.

    Of course the airline magazine guys are going to botch things up, just as they did in the mainframe-to-desktop transition. I saw a Microsoft document that suggested using OLE automation to fire up a copy of Excel so you can get access to its sum() function from within Access. Can you imagine what it will be like when office programmers hit a server with SOAP requests to add up a column of numbers, because their local language doesn’t have a sum() function?

  28. ESR, it looks like dotimes would be better than while in that code. declaring a loop variable and incrementing it at each iteration of the loop is SO c/c++. :)

    your placement of parentheses is also non-standard – excessive whitespace.

  29. Jay Maynard:

    I write C in LISP all the time. Check out Pre-Scheme, a Scheme subset compiler that comes with Scheme48 ( http://www.s48.org ) and is used to write its VM. It lacks garbage collection and other assorted features; but it compiles cleanly to C and requires virtually *no* runtime. I used it to write some high-speed blitters for a game engine and got comparable performance to C blitters (good) with very little time and hassle spent on them (better).

  30. This code looks like it could have been directly translated from, say, Python, especially with it’s un-Schemish use of a while loop that increments a counter instead of a tail-recursive loop. Not that there’s anything wrong with it, but it just doesn’t highlight anything that makes Lisp different, other than the syntax. I have to wonder why ESR particularly considers it a good thing that Lisp doesn’t die.

  31. Dear Jeff,

    it seems to me we are somehow misunderstanding each other. Two topics:

    1. On Access

    Do not misunderstand me, I don’t like Access and other typical MS tools, as they are artificially limited, yes, indeed. But I have a 40 years old friend, who started out his life as a miner. Not, not data miner, but coal miner, blue-collar, yes. Later on he went to business school and learned marketing. Graduated. Started to work in marketing. Started to use some MS tools, as these were the standard in that (and in fact, sadly, most) office. He quickly understood that really effective marketing is about analysing data, and not about drooling about the color of the product logo. He created Access databases and learned more and more in programming, just by trial and error. Later he quit his job and started custom-writing CRM (Customer Relationship Management, basically, one-to-one marketing and sales force automation) applications in Access for small companies. I would not think it would be possible in a multi-tier flexible and free environment.

    And what it actually means is the democratization of programming.The future is that programmers, who are just programmers, will disappear (actually, their job will disappear to India). Nowadays, speaking in role-playing game terms, a programmer “character class” is always a multiclass, like fighter/mage multiclass in AD&D. This friend of mine is a marketing professional/programmer multiclass. I am an warehouse and manufacturing process analyst/programmer. (This is why I am always speaking about small, dirty screens :-) ) I think you must be some kind of researcher/programmer, because you seem have a kind of researching mindset. No one is just programmer now.

    And no, I don’t say Access is a good tool for democratic programming. We need something more flexible and powerful. Maybe, PostGres strongly integrated with libglade and OpenReports, with an ncurses interface that can also read Glade XML. Whatever. But Access was not a bad starting point for smaller companies with simpler needs.

    2. On CRUD, hacking and the panacea way of programming

    Ask yourself the question that why – by Gartner Group figures – a really big proportion of ERP implementations just fail and the majority is hated by users? If you happen to have the answer, I would be really thankful, because I could not find it in these 3 years I am working with ERP. We did not stick to the book. We did some things that would make the average business analyst freak out, for example, wrote code generators that generated code generators. And it just still sucks, users just hate the software and we work overnights.

    I don’t have the answer, but have a strong guess. Recently I just received the data of 15 products in spreasheet to import it to the ERP for a demo. I decided that 15 is not enough to write an import program, I will just enter them manually. By the eighth one, I was half-seriously considering suicide and by the tenth one, I was looking around in the office for an assistant girl who I could bribe with some chocolate to enter the last five for me. Keying in data was just so boring, so disgusting, so not worthy of human time, so feeling like a worthless, mindless robot… and if it is so, then the least we MUST do is to make it as fast, as efficient, and as error-prone, as possible.

    So, if you are curious why some people like have this simple-minded “application development is CRUD” mindset, my answer is 1) because CRUD is important as all business relies on it 2) but it is such an amazingly disgusting job to use a CRUD application and people hate it so much, that is very, very important to make it right, and, actually, that no one ever managed to figure out how to do it right. It is also a kind of research, yes, on how to make something horrible into something, at least, bearable.

    As for software engineering. The problem with it is that it is engineering. Engineering is for creating good things. Do we agree? CRUD development is to minimize the suffering arising from the fact that someone, eventually, will have to enter many boring data. If you want to make people happy, you can and you have to be endlessly creative and for this, you need freedom.
    But if you want to reduce suffering, like a doctor, you need both a disciplined, not free approach on one way, and many nasty hasty ad-hockery to solve really serious problems in a quick-and-dirty way. This is a contradiction, yes, but this is the core of both medics an CRUD.

    You will not understand CRUD-optimized tools unless you understand it’s for reducing suffering, not for increasing happyness. Everything else you can figure out from this, I think. Am I right that you have never thinked on software development this way, of being a panacea before? I think this is the reason we don’t understand each other.

    I agree that hacking is the only proven development method, but I think there are many ways to do that. Trational Unix-style research hackers and dumb people who spend x decades working with DP are just two extremes. See, Unix research hackers were spending their time researching for ultra-cool things that could make people happy, in an environment, in a way that made themselves happy. Software as a panacea was totally out of the picture.

    I meet with real, real hackers in the CRUD area – one of them is an excellent flute player, another is mathemathics genius, and I think this is generally good hacker material…but they have a little bit of different mindset that research hackers. They tend to keep everything in their heads, they want to solve every problem alone, they don’t really like the many interconnected parts idea of the traditonal Unix way, they don’t like to rely on others, but like to reinvent every wheel on their own into one ultra-productive supertool. CRUD hackers are tend to be “the only code I trust is my own” – type of hackers, which is generally un-Unixish. This is simply because they suffered too much, as users are very proficient in transfering their suffering to programmers. But they are still hackers, as are clever, like interesting problems, and like to come up with unusal solutions. Let’s call it “panacea-hacking”, or something like that.

    As for what professional means, yes, it can mean different things. For me it means “heavy-duty”. An application, that does not allow you to enter bad data, no matter how hard you with. An application, that is visible on a small and dirty screen. An application, that you can work fast with. So, an application that is a useful panacea.

  32. As someone who has been doing a lot of Common Lisp recently, I found Script-fu isn’t even that particuarly good a lisp. I found it rather under-documented (I’m hoping to fix this a bit, if I ever get time) and the naming conventions all a bit random; but it still packs hell of a lot of functionality into a very small box.

    Lisp is magical becase it’s malleable to the nth degree: code is data, you can do anything at compile time that you can do at run time, so you use the langauge to compile itself and create new domain specific languages and syntatical forms.

  33. Shenpen:

    You’re still flailing about, hitting all the wrong targets. This isn’t really about the Web, CRUD screens, or anything of the sort; it is about loose coupling. Read my original post. You brought Web apps into this, saying they were unprofessional, as if by extension to imply that loosely coupled apps are unprofessional. This is Software Engineering 101; it’s not something cutting edge, it’s been around a long time and people do it even if everything they write is their own code. That’s because it’s much easier and saves on development and testing time to write simple things that work together as opposed to “ultra-productive supertools”. (There are exceptions, Emacs being one of the canonical ones, but it is basically a Lisp environment with editor primitives; and it, too, plays nice with other Unix tools). That your Real Programmer friends don’t seem to have their heads wrapped around this just yet probably goes a long way to explaining some of their perennial suffering and death-march production cycles.

    I’ve worked with dBASE, Paradox, and even Microsoft Access back in the day. Each of these programs has misfeatures and broken-as-designed limitations that ought not be there (and wouldn’t be in a conventional programming language). The number one problem is they’re convenient for certain things but stray away from these things and the program just gets in your way… and there is no escape. The open source community tends to frown on such “walled garden” approaches; they have been tried in the past, as I’ve mentioned before, and have found very little purchase in the OSS bazaar.

    What I have seen, however, is plenty of people write Perl or Python scripts that connect to a MySQL back end, and an X11 or curses front end, for a particular application such as managing record collections and the like. I see no reason why this couldn’t scale to the needs of a small or even a medium-sized business, and I haven’t heard any good reason from you yet either.

  34. The only reason I still do not agree with loose coupling that something like MySQL + Perl + GTK for example are three different things you have to learn all of them and keep them in your head, and the worst thing is that you have to switch three “syntax interpreter” in your head on every tenth line.

    And this is just okay if what you do is a) just programming a very well defined task like as a COBOL bank-slave or b) you do it like a research hacker: just try something and if it does not work today, well, who cares.

    But in CRUD programming business processes, contradicting wishes, business whatifs etc. take up 95% of brain capacity – what you basically do is the same thing that stuff like SSADM design was in the old times, but now you just have to do it on the fly right in program code. So programming itself must be as simple as possible, because it is the least important part. And ususally, it has to work *today*. No time for research.

    As for walled “garden”: but an Open Source Access with PostGres and GTK or whatever, which is straightforward as Access, but any time you hit a wall just ask link it with any C or C++ library that would do the trick, why would that not be a good thing?

  35. Loose coupling is ultimately about segregating code to deal with specific tasks. You could see a MySQL + Perl + GTK stack as a software representation of MVC divisions. Yes you switch “syntax interpreters” when you are dealing with the data as opposed to display, but you are hopefully already switching function calls to do that anyways. I am not trying to dismiss having to literally switch programming languages for various project parts, that is an important problem, but I think the extra flexability in the specific problem domain are worth the switch. By using SQL instead of Perl to interact with your data you are using a language designed for that interaction. Expressing how you want to store/retrieve data is easier in SQL than it is in Perl, provided you know the language.

    That, ultimately, is what this is all about. Loose coupling has a higher requirement for knowledge investment. You have to know the languages/toolkits/whatever involved to get things done, instead of just having one way to do it. After you have this knowledge it makes creating something that “works today” easier. Instead of spending half an hour coming up with a new function to sort the data by X, Y, and Z then compare it to foo you spend about five minutes writing SQL statements and get on with it.

  36. John Connors, Script-Fu is based on SIOD, a tiny, relatively weak Lisp interpreter that, if you squint, kind of resembles Scheme. Real Scheme is as powerful as Common Lisp, though it lacks such a robust standard library (support for SRFIs helps mitigate this).

  37. I disagree with the stating that Python is the most Lisp like mainstream language.

    That spot was already taken by Ruby. You only miss macros there. And you have wxRuby.

  38. As a schemer I am a concerned that the old, non-standard Scheme variant in GIMP
    is doing more harm than good publicity wise. I’d like to see GIMP include one
    of the newer, full Scheme implementations (e.g. PLT Scheme). They are just
    as practical as Python.

  39. Meditating on the GIMP’s deprecation of Scheme, Yahoo’s rewriting of its store code after the acquisiton and other serious blows to the Lisp community led me to come up to my own answer of Greenspun’s 10th Law:

    Any sufficiently sophisticated Common Lisp or Scheme program will eventually get rewritten in C++, Java, or Python.

  40. this is the code of an “old LISP head” ey? If that’s how they used the language it’s no wonder it declined in popularity.

  41. Jeff,

    by now I am starting to feel the limitations of even Ruby as well, so I was again taking a look at LISP. SISC looks pretty promising, becase a Scheme interpreter with full interoperability with Java means you have everything from Struts to Castor there right in your Scheme app, which should be the most productive programming environment ever invented, right? But what’s the catch here? I mean had there been no catch, everybody would be using it, right?

    My mind again wanders on the visual editor idea. Instead of those ugly parens, one could represent Scheme code in a tree widget. And it would not necessarily mean slower coding, if we had useful keyboard shortcuts, then pressing, say, F3 (for a new child node, basically wherever we would write a paren), writing “define”,F3, “square-root” n, pressing F3, writing n * n, should not be longer than typing (define (squre-root n) (n *n)), right? We even save the keystroke of closing parens.

    Besides achieving the most readable code in the world, it would finally fulfill the old dream: to wash the borders between development and configuration. Once we defined an easy sublanguage with syntactic macros, every clever business analyst with zero technical background could add functionality, while what they actually write this way would still be a legal Scheme program, so we would not lose an inch of flexibility while providing unparalleled ease of use.

    And imagie going to a child S-exp, pressing a shortcut for adding a testing assumption, which would make that node return the value of what you write there and not being evaulated, and go to a higher leven S-exp, and press another shortcut for REPL testing … what an amazing testing tool would that be.

    I am quite excited by this. I think I will build it. What do you think?

  42. I haven’t much to say about SISC except that I have built production systems with SISC and Tomcat, which proves its viability as a platform suitable for use in industry.

    If you are attempting to wire it into complex, horrid abominations like Struts I think you may find yourself in for more trouble than it’s worth. Look up SISCweb, and relax a bit.

    Good luck with that editor of yours. Should you produce something substantial, I’d like to see what you come up with.

  43. SISCweb looks interesting, although it needs quite a lot of work to become something like Ruby on Rails, although, given Scheme’s flexibility, it might one day become something lot better than it. Until then, it’s not a real competition. Three things are missing the most, model classes, AJAX and HTML templates. The later are the most important missing feature, because you can’t give a Scheme program to a non-programmer webdesigner to beautify. What makes Rails so popular is that when you defined, say, an SQL table called comments, a Comment class is autogenerated and linked to the table. Therefore, to put a list of comments added this week, you say @comments = Comment.find(:all, conditions => [“date > ?”, beginning_of_week]) and then in the HTML template,a foreach through the @comments list, repeating a HTML table row for each, and it’s done. I think such features could easily be added to SISCweb, maybe model classes could be simulated with generating Hibernate classes and wrapping the into model closures, and adding HTML templating could also be a breeze, and all the AJAX stuff is but wrapping around the Prototype JavaScript library. So it’s not a big work. But it needs to be done, in order to make it a real competition…

  44. Jeff,

    how would you _elegantly_ implement the following Ruby pattern in LISP? We have a puts function that can print any object on the stdout: it looks for a to_s method of the object, which should return a string, calls it and prints it. So if a Person object returns first_name + ” ” + last_name in the to_s, then a puts person “magically” prints that. With such idea do we write kind of a DSL’s in Ruby. Now, what I mean elegantly means:

    – without the CLOS, because there is no point in doing the same thing in LISP that you could even do in Java with interfaces

    . without syntactic macros, because that I would consider cheating for such small purposes

  45. Another thought. I learned some LISP and now I think the code ESR wrote here has nothing you could not do in Python/Ruby. One of the problems of LISP is that it is often misrepresented. Such problems simply don’t need LISP and such examples don’t show why should one learn it. Look at this: http://www.gigamonkeys.com/book/practical-a-simple-database.html for a good example. It starts out as a simple CD database and ends up rewriting a significant subset of SQL in LISP: (select (where :title “Give Us a Break” :ripped t)) Such examples show the real power of LISP.

  46. Shenpen,

    The code Eric wrote here has nothing you could not do in C++ (except for maybe use of automatic garbage collection; and even then, link against Boehm). It isn’t particularly exemplary of good Lisp/Scheme coding practice.

    You want me to implement a generic dispatch system in Common Lisp without using the CLOS? Have you a fetish for needlessly reinventing wheels?

    SISCweb is aimed at an entirely different audience than RoR. If you fail there’s significant work that “needs to be done”, by all means, go ahead and get started. I’d be interested to see Shenpen’s Scheme on Skids when it is finished to the point of usability (and that’s not trolling, it’s the truth).

Leave a Reply to Jeff Read Cancel reply

Your email address will not be published. Required fields are marked *