Brian Mastenbrook

paste.lisp.org CAPTCHA in acrylic

Mon, 19 May 2008 17:15:52 GMT

Becky Stern of the MAKE magazine blog recreates CAPTCHAs in acrylic. Her latest creation is the CAPTCHA I created for paste.lisp.org:

The CAPTCHA is done purely using DIVs and CSS2. The font is Chris Cannam's superb Codec monospace bitmap font, which I've used as an Emacs font for years. The code which creates the CAPTCHA can be viewed on paste.lisp.org.

The painting is for sale, if you've got the perfect place to hang a 5x7 acrylic painting of "lisp".

Public Service Announcement to Debian and Ubuntu Users

Tue, 13 May 2008 16:09:20 GMT

A weakness has been discovered in the random number generator used by OpenSSL on Debian and Ubuntu systems. As a result of this weakness, certain encryption keys are much more common than they should be, such that an attacker could guess the key through a brute-force attack given minimal knowledge of the system. This particularly affects the use of encryption keys in OpenSSH, OpenVPN and SSL certificates.

The fix involves updating your OpenSSL packages and regenerating any private keys made on recent versions of Debian (etch and newer) or Ubuntu (Feisty or later). For more information, see:

This issue was caused by a Debian-specific patch to OpenSSL and does not affect distributions which aren't derived from Debian.

MCL 5.2 has been released as open source

Fri, 21 Mar 2008 19:53:12 GMT

Digitool has released MCL 5.2 as open source. From the announcement:

MCL 5.2 is now available at:
ftp://ftp.clozure.com/pub/MCL/MCL-5.2-Final.dmg
It is open source. I am sorry that it does not run on Intel. I hope that some users will be able to find the resources for a port of MCL to Intel processors.
Please let me know of any questions or problems.

MCL is the distribution from which Clozure CL (nee OpenMCL) was derived. Clozure CL's compiler has been much enhanced over MCL's, which is currently PPC-only and not Rosetta compatible. MCL features a Carbon GUI with an Emacs clone named Fred, an interactive inspector and debugger, and a number of other useful tools.

I'd encourage anyone who is interested in making MCL Rosetta-capable or porting to Intel to sign up for the info-mcl mailing list. MCL is an important part of the Common Lisp legacy and it's good to see that it will not be lost to history like some other important implementations (Genera).

New Publication

Fri, 06 Jul 2007 01:17:06 GMT

Frank Weil, Brian Mastenbrook, David Nelson, Paul Dietz, and Aswin van den Berg. Automated Semantic Analysis of Design Models. To appear in ACM/IEEE 10th International Conference on Model Driven Engineering Languages and Systems (MoDELS), October 2007. PDF, 138 kB.

Many of the techniques given in this paper are well known in the compiler community. The application to verification of models of industrial scale telecommunication applications has been able to spot significant but difficult-to-test problems.

There will also be an experience paper given at the 13th System Design Language Forum in September. When I have a copy of this paper, I will post it.

Packrat parsers in Lisp

Tue, 12 Jun 2007 12:18:39 GMT

CL-PEG is a packrat parser generator in Common Lisp. (Found on CLiki.)

New look for Planet Smalltalk

Fri, 09 Mar 2007 13:09:06 GMT

Planet Smalltalk has a new look. While I miss the ability to filter out blogs in languages I don't know, the page is much more readable in its new design.

You don't bury survivors

Fri, 02 Mar 2007 20:14:25 GMT

Pure genius.

I am not a number; I'm a free man!

Fri, 22 Dec 2006 03:40:12 GMT

Westfield Sportscars has teamed up with the University of Warwick to produce a hybrid sports car based on the Lotus Seven. The Seven is the iconic car that The Prisoner drives in the opening credits of the eponymous show. Needless to say, I'm in geek heaven. The Seven has been on my "buy someday" list for a while now (along with a garage to put it in). When that happens I might even be able to buy a hybrid version.

The Tesla Roadster is based on a much more recent Lotus design, the Elise.

Lisp image uptimes

Sun, 10 Dec 2006 03:13:21 GMT

Prior to SBCL 1.0, GET-INTERNAL-RUN-TIME returned the time since the first call to GET-INTERNAL-RUN-TIME in that image, not the time since image start. Thus, when I tried Xach's uptime reporting code, my longest-running image reported an uptime of 0 seconds.

Fortunately Linux keeps this around in the proc data structures, and ps can pull it out in human readable form:

$ ps -eo pid,lstart,cmd | grep sbcl
17848 Tue Jun 27 06:25:03 2006 sbcl

As of this writing, my longest-running image is 165 days, 15 hours, 41 minutes, and 58 seconds old, which bests Xach's by 19 days. This image is also a commercial application: it provides a web application to Formally Modern Tuxedo, a family formalwear store in Chicago.

The image which provides this blog is much younger (90 days, 7 hours, 10 minutes, and 4 seconds). I'll likely upgrade this image to SBCL 1.0 soon.

Emacs notes

Sun, 29 Oct 2006 19:35:09 GMT

If you haven't been following the goings-on of the Emacs Unicode branch, you may be unaware that there is a working Xft GNU Emacs available.

That Emacs looks like this when compiled with the GTK backend:

Click for larger image.

I've received many requests in the past for my Emacs colors. Thanks to some prompting by email from a fellow named Krishna, I've taken the time to learn how to use color-theme and produced this theme. The font I use is Bitstream Vera Sans Mono, which is easy to read and has a distinguishable zero ('0').

A good way to waste a lot of time

Tue, 10 Oct 2006 12:07:16 GMT

Attempting to run your own mail server is a good way to waste a lot of time. I switched from mbox to Maildir a few months ago, and I never quite got my IMAP server (Dovecot) working correctly.

It turns out it was a bug in JFS. Now that I've got my Maildir hosted on an ext3 loop filesystem, everything is running fine.

iTunes, iPod, and Gapless Playback

Wed, 13 Sep 2006 12:20:59 GMT

Apple released a new version of iTunes yesterday that includes a new Gapless Playback feature for both iTunes and the iPod. I've been ripping my albums as one continuous track since I started using iTunes due to the lack of this feature. I can't imagine listening to this album with gaps in between tracks.

Gapless playback seems to work for any album which would play back without gaps if you burned it to a CD from iTunes. This includes content ripped from CDs and purchased (iTunes Music Store) content from a few particular labels, including Global Underground. However, iTMS albums from Distinctive Records still do not play back correctly; now it sounds as if they skip a beat in between tracks. I'd recommend not purchasing content from Distinctive via iTunes until they fix this issue.

Spam-B-Gone

Mon, 03 Jul 2006 14:33:05 GMT

I first heard of greylisting when I asked why mail delivery on common-lisp.net was taking so long. A few months and hundreds of spam emails later, I decided to try it on my own server, but with a more reasonable delay of 60 seconds. I've received no spam since I installed postgrey.

Of course, at some point the spammers will catch on, but if this brings the spam down to a reasonable level, I'm all for it. Hooray!

Allegro CL 8.0 Express Edition Available

Wed, 14 Jun 2006 18:01:23 GMT

The long-awaited free trial of Allegro CL 8.0 is now available. Those looking for a free way to try their code on Allegro now have the ability to use the latest version of Franz's flagship product. In addition the trial supports Franz's new database, AllegroCache.

Can someone else work on your code?

Fri, 03 Feb 2006 00:03:18 GMT

Stefan Scholl asks whether you can get someone else to work on your Common Lisp program. I think that if you ask most people whether someone else could maintain their code, they would answer no, regardless of the language. It's not because they are so proud of their own abilities that they doubt anybody else's ability to maintain their work. Rather many programmers I've talked to think their code is so bad that nobody else can understand it.

I work with a large Common Lisp codebase at work, and I'm often surprised at how easy it is to maintain and make changes to even the "bad" parts of the code. I think this is a combination of having good tools and having test cases for the code. Regression testing makes it easier to understand what impact a change has on multiple test cases. The ability to run just one of those test cases, break inside the function you're interested in, and then inspect the local variables helps an awful lot too.

I think this also prevents a lot of people from asking for help with their codebase by opening it up. They feel the code is not ready for public distribution because it's not good enough. No code is, once it's subjected to real users and their requirements. The adaptability of the codebase is what matters.

Dear Diary

Fri, 27 Jan 2006 01:21:51 GMT

The subject of how to teach programming always elicits from me a feeling of vague unease. I find it hard to engage an elaborate discussion of whether HtDP or SICP is better for a new programmer. My gut reaction is that neither is right, really. It's not that I don't think Scheme is a good language to teach. It's the idea of learning how to program in the modern environment which makes me uneasy. It's too tempting.

Like a lot of people my age, I learned to program in BASIC. It was a mostly self-directed activity. There were people who showed me the basics of the language, but after a certain point I learned a lot from a reference manual and other people's programs. (Back in those days it was possible to go to the local library and check out books filled with BASIC source for various programs, mostly games.) I learned because it was fun. Part of the reason it was fun was the sense of accomplishment in making the machine do something useful. The other part was that it felt like I could accomplish most of the things that I had seen the machine do. The Apple II was not an especially complicated machine. Simple graphics, sound, and disk I/O are all firmly within the grasp of the beginning programmer.

I think the urge to do what you've seen the machine already do is found in most beginning programmers. Very few programmers learn how to program without already knowing what their computer can do. So it's only natural to expect to be able to create GUIs, use the network, and create flashy 3D games when you first learn how to program today, despite the fact that most working programmers don't use most of that, most of the time. (No, I'm not going to quantify that.)

If you're going to teach programming on modern computers, there are two options: you can give in, and integrate this into the curriculum, or you can hold out, and attempt to teach good thinking discipline before teaching shader models and light maps. Neither method is all that great.

If I had to teach programming today, I'd be sorely tempted to start with one of TI's or HP's high-end calculators. These machines resemble the machines that I learned programming on quite a bit more than modern computers do, and they share the same essential characteristics. It's easy to create useful programs, and the full range of the machine's capabilities are easy to harness. They also teach the fundamentals of programming in a modern environment because they are based on dynamically typed languages with garbage collection.

Of course, neither machine is based on Scheme or Lisp. I have no problem with teaching these languages, because it's not the language which I am concerned about. It's the context in which that language is learned. Even Squeak which to some degree establishes its own context, at least visually, tries far too hard to show off its abilities to produce GUIs, web programs, and 3D graphics.

I think many of the issues involved in teaching programming are conflated with the issues of teaching introductory computer science. I learned to program long before I learned anything of computer science. Of course what I learned in my undergraduate studies helped to mature my abilities, but it was not essential to the task of learning how to make the machine do something useful and debug it when it doesn't do what I want. Learning to program was about exploring systems of knowledge. I still remember when the thought of installing Linux brought a sense of glee, just because it was large and complicated and was not something which I already knew. There is very little in the academic context which could have evoked that same feeling.

Learning how to program is tied to an individual's mode of learning and problem solving far more than the academic context allows for. I think that one day universities will realize that solid programming skills in at least one language should be a prerequisite for a Computer Science curriculum, just as music majors are expected to already play one instrument. In fact there's much that we could learn from the field of music education - perhaps the best way to teach programming is one-on-one with someone who can adapt his teaching style to the individual's learning style. I wonder if practicing programmers will start to give programming lessons on the side to pass along their love of the discipline. It might do a lot of good for education in general.

Briefly: when I wrote about XML mixed mode I was under the delusion that I had somehow worked around CALL-ARGUMENTS-LIMIT by making the XML tag in (<tag> body1 body2 body3) a macro which was bound by a lexically-surrounding WITH-XML form. I hadn't; those element body arguments still ended up as arguments to LIST. The WITH-XML macro is really unnecessary, and now <tag/> reads as a form which returns an object, while <tag> reads as a LAMBDA form which can be used at the head of a list or evaluated to return a function. That function, when called, returns an object which represents the tag wrapped around the arguments as a body list.

I also have an implementation of XML to XML transformations based on pattern matching, using XML mixed mode for the lexical syntax of patterns. This is intended to replace the hacky templating system I wrote for cl-blog, which was so poorly designed that it was incredibly difficult to express conditional or iterated expansion. The new system is much nicer, and allows templates to be created by people who do not know the template system.

I recently started working on a generic system for intentional types in Common Lisp. Since I am working on a library for web programming, this should have been a no-brainer, as the task which I have spent the most time solving is perfect for such a system. I already have a good abstraction set up for producing and processing forms, but the processing of values from strings and conversion of strings to values was always a little bit hacky.

Of course, CLIM also offers an intentional type system for presenting and accepting values, but the mechanics are tied into CLIM. I think it would be useful to have a generic system for defining and representing intentional types, as well as defining operations using those intentional types. I may even provide an equality operator.

paste.lisp.org et al

Mon, 16 Jan 2006 15:33:09 GMT

Until the common-lisp.net migration is complete and all DNS records have been updated, paste.lisp.org and related services will be down. I will update this post when they are available again.

Edit: paste.lisp.org is available again.

Thanks,
Brian

XML Mixed Mode: A New Lexical Syntax for XML Generation in Lisp

Sat, 24 Dec 2005 20:58:36 GMT

Current representations of XML in Lisp use lists of symbols and strings as the basic representation of tags and tag contents. While these representations are adequate for processing XML, they are not the most convenient representation for producing XML documents from code. The list-based representation relies on Lisp's basic list template functionality, quasiquotation, to express templated XML output. In this representation, the only template inclusion operators are a direct inclusion of one list element and a splicing of multiple list operations. As a result, basic idioms such as a conditional inclusion of a tag must be represented as a conditional splice of a list containing one tag.

Futhermore, quasiquotation is often used in macros, which implies that macros which generate code which generates XML involve either two levels of quasiquotation or abandoning quasiquotation for list templating in one or the other of the macro or the XML-generation code. Either approach leads to contorted code: two levels of quasiquotation leads to difficult-to-read syntax such as ,',@ and abandoning quasiquotation leads to an explosion of list construction forms.

Abandoning the use of templates seems preferable. To do this, an unambiguous means of differentating code and XML productions is required. In Common Lisp, only symbols or lists starting with LAMBDA are allowed as the first element of a list which is to be evaluated as a form. Thus, symbols must be used to represent tags. Since the output should be arbitrary XML, and not one particular XML schema, any tag name should be allowed. In order to keep the mechanism unambiguous, these tag names should never collide with existing user code, and thus should be part of a separate package.

XML Mixed Mode solves this problem by introducing a reader macro which translates a superset of XML tags into newly created symbols, which can then be found by a macro and bound to macros or symbol macros that expand to XML-producing code. Thus XML tags are used directly in the Lisp code, which leads to a greater correspondence between the code and the data it produces.

Here is an example of XML Mixed Mode in Common Lisp:

(defun main-menu () (with-xml (flet ((link (handler text) (let ((url (urlstring (handler-url handler)))) (<li> (<a href=?url> text))))) (<ul> (link 'main-handler "Main") (if *is-authorized* (link 'logout-handler "Logout") (link 'login-handler "Login")) (link 'new-project-handler "New Project") (link 'new-task-handler "New Task")))))

The XML mixed mode syntax includes extensions for using the value of a Lisp symbol as a tag name, attribute name, attribute value, and as a list of attribute name / value pairs. When an XML start tag is the first position of a form to be evaluated, and that form has been surrounded lexically by a use of WITH-XML, it creates an XML form whose contents are the result of concatenating the other elements of the evaluated form. The normal XML syntax for a tag with an empty body is also supplied.

I have converted my Trail task-tracking application to use XML mixed mode for output. This version is not running on the live Trail instance on my server because cl-blog has not yet been converted, and the library that both applications depends on has also been converted to require XML mixed mode. However, you can see the source of the converted version as an example of both XML mixed mode and some new improvements to my as yet unreleased CL web programming library (which will change names from "webutils" before it's released).

Can I make a request?

Wed, 21 Dec 2005 13:16:18 GMT

To: The Lisp Community
Cc: Santa Claus
Subject: The R-word Site in Lisp

I'd like to make a request. While I appreciate the urge to show off your particular toolbox of web programming in Lisp abstractions by implementing a site which allows users to submit links and vote them up and down, can you please refrain from saying that you've implemented the R-word site in Lisp in an absurdly small number of lines of code until all of the following hold?

Links must survive a server reboot. If you haven't gotten this implemented, you haven't really implemented anything but a toy.
Does it, *ahem*, escape HTML properly?
Is it foolproof? Is it smarter-than-foolproof? When I see an R-word clone which starts off with :unsecure t, I begin to suspect that the answer is "no".

-- The Management

Edit: Sven responds by email: "The option (:unsecure t) means that no login is enforced. The default for KPAX (:unsecure nil) would enforce login against any of a number of sources (but that would then require registration and account management, not hard but too much for a demo)." So this is right for a demo - but clearly some form of access control (whether by login or captcha) is required for a real site to prevent flooding by bots.

He also adds: "The Reddit story had a serious impact on the Lisp community; showing that it is possible and even easy to write web applications in Lisp seems like a logical reaction. In my opinion we should indeed not try to better them or prove them wrong. The right reaction is to try to fix some of the problem they mentioned."

My concern is that by underselling the amount of work necessary to produce a complete web application, and showing off a lot of little prototypes, the Lisp community may come off as un-serious. "See! I can do this in 4 hours! 2 hours! 20 minutes!", when the end result is clearly only a prototype (and this is true of all of the Reddit clones I've seen - and all clearly indicated as such) does nothing to disprove the typical reader's takeaway from the Reddit conversion: that Lisp is good for prototypes and unsuitable when the rubber hits the road.

Climbing the abstraction tree

Wed, 21 Dec 2005 02:42:18 GMT

Somewhere noodling around in my brain is a theory of how programmers must practice at a thing (language, domain of programming , etc.) for a while before they can contribute things which are of value to other people. It's been over a two and a half years since my first forays into the world of web programming in Common Lisp, and only recently do I think I'm getting to that point. Nothing documented yet - I'm still climbing the abstraction tree on a number of different projects, but the change has been fairly rapid and what I have now allows me to bang out small programs with ease.

One example of this would be a little task tracking program which I recently moved to run on my public web server instead of locally. It hosts tasks for a few of the personal projects I'm working on, as well as some private projects, which you shouldn't be able to see without logging in. It also (like any good Web 3.0 application) serves its own source. It's obvious that I haven't gotten rid of all the superfluous bits yet, but it's getting really close. I expect solutions to the remaining bits expressed in ways which don't offend my sensibilities can't be far off.

We've gone from the dreaded R-word topic on Planet Lisp to impending political hysteria. In an attempt to flush as much of it as possible, I'll reproduce the following ditty from James Lileks. It cracked me up, not least because I'm still recovering from my own data loss.

Ladies and Gentlemen: Miss Shirley Bassey.
Dumm-dum (wa wah wah wah wah!) DUMM-DUM (wa wah wah wah wah)
Dissssk failure!
It's the thing, the thing that you oft denied
And now you cry
(da da-da, da da)
Such a bliss nailer
(wa wah wah wah wah!)
B-tree's hosed, directory's naught but hash
Magnetic rash
How you shrugged at the long access time
How you thought "disk's just full, all is fine"
Now you know you've many bad sectors -
Killed your data like Mister Lecter, that's the
DISK FAILURE
(wa wah wah wah wah!)
Sys admin, you'd better begin to pray
Last backup: May!

If hard drive crashes were a common occupational hazard circa 1943, this is what they'd sound like.