Brian Mastenbrook: XML Mixed Mode: A New Lexical Syntax for XML Generation in Lisp

Current representations of XML in Lisp use lists of symbols and strings as the basic representation of tags and tag contents. While these representations are adequate for processing XML, they are not the most convenient representation for producing XML documents from code. The list-based representation relies on Lisp's basic list template functionality, quasiquotation, to express templated XML output. In this representation, the only template inclusion operators are a direct inclusion of one list element and a splicing of multiple list operations. As a result, basic idioms such as a conditional inclusion of a tag must be represented as a conditional splice of a list containing one tag.

Futhermore, quasiquotation is often used in macros, which implies that macros which generate code which generates XML involve either two levels of quasiquotation or abandoning quasiquotation for list templating in one or the other of the macro or the XML-generation code. Either approach leads to contorted code: two levels of quasiquotation leads to difficult-to-read syntax such as ,',@ and abandoning quasiquotation leads to an explosion of list construction forms.

Abandoning the use of templates seems preferable. To do this, an unambiguous means of differentating code and XML productions is required. In Common Lisp, only symbols or lists starting with LAMBDA are allowed as the first element of a list which is to be evaluated as a form. Thus, symbols must be used to represent tags. Since the output should be arbitrary XML, and not one particular XML schema, any tag name should be allowed. In order to keep the mechanism unambiguous, these tag names should never collide with existing user code, and thus should be part of a separate package.

XML Mixed Mode solves this problem by introducing a reader macro which translates a superset of XML tags into newly created symbols, which can then be found by a macro and bound to macros or symbol macros that expand to XML-producing code. Thus XML tags are used directly in the Lisp code, which leads to a greater correspondence between the code and the data it produces.

Here is an example of XML Mixed Mode in Common Lisp:

(defun main-menu ()
  (with-xml
    (flet ((link (handler text)
             (let ((url (urlstring (handler-url handler))))
               (<li> (<a href=?url> text)))))
      (<ul>
       (link 'main-handler "Main")
       (if *is-authorized*
           (link 'logout-handler "Logout")
           (link 'login-handler "Login"))
       (link 'new-project-handler "New Project")
       (link 'new-task-handler "New Task")))))

The XML mixed mode syntax includes extensions for using the value of a Lisp symbol as a tag name, attribute name, attribute value, and as a list of attribute name / value pairs. When an XML start tag is the first position of a form to be evaluated, and that form has been surrounded lexically by a use of WITH-XML, it creates an XML form whose contents are the result of concatenating the other elements of the evaluated form. The normal XML syntax for a tag with an empty body is also supplied.

I have converted my Trail task-tracking application to use XML mixed mode for output. This version is not running on the live Trail instance on my server because cl-blog has not yet been converted, and the library that both applications depends on has also been converted to require XML mixed mode. However, you can see the source of the converted version as an example of both XML mixed mode and some new improvements to my as yet unreleased CL web programming library (which will change names from "webutils" before it's released).