musings of a Lispnik

Keyword arguments

There’s been an ongoing debate about how to pass optional named arguments to Clojure functions. One way to do this is the defnk macro from clojure.contrib.def; I hesitate to call it canonical, since apparently not everyone uses it, but I’ve found it useful a number of times. Here’s a sample:

user> (use 'clojure.contrib.def)
nil
user> (defnk f [:b 43] (inc b))
#'user/f
user> (f)
44
user> (f :b 100)
101

This is an example of keyword arguments in action. Keyword arguments are a core feature of some languages, notably Common Lisp and Objective Caml. Clojure doesn’t have them, but it’s pretty easy to emulate their basic usage with macros, as defnk does.

But there’s more to Common Lisp’s keyword arguments than defnk provides. In CL, the default value of a keyword argument can be an expression referring to other arguments of the same function. For example:

CL-USER> (defun f (&key (a 1) (b a)) 
           (+ a b))
F 
CL-USER> (f)
2
CL-USER> (f :a 45)
90 
CL-USER> (f :b 101)
102

I wish defnk had this feature. Or is there some better way that I don’t know of?

Sunflower

The program I’ve been writing about recently has come to a point where I think it can be shown to the wide public. It’s called Sunflower and has its home on GitHub. It’s nowhere near being completed, and of alpha quality right now, but even at this stage it might be useful.

Just as sunflower seed kernels come wrapped in hulls, most HTML documents seen in the wild come wrapped in noise that is not really part of the document itself. Take any news site: a document from such a site contains things such as advertisements, header, footer, and many links. Now suppose you have many documents grabbed from the same site. Is it possible to somehow automate the extraction of the document “essences”?

Sunflower to the rescue. It relies on the assumption that documents coming from the same source have the same structure. It presents a list of strings to the user, and asks to pick those that are contained in the text essence. Then it finds the coordinates of the smallest HTML subtree that contains all those strings, and uses those coordinates to extract information from all documents. And it comes with a nice, easily understandable GUI for that.

This technique works remarkably well for many collections, although not all. An earlier, proof-of-concept implementation (in Common Lisp) has been used to extract many press texts for the National Corpus of Polish.

I’ve given up on the symbol-capturing approach to wizards I’ve presented in my previous posts. Inspired by the DOM tree in Web apps, with a bag of elements with identifiers, I now have a central bag of Swing widgets (implemented as an atom) identified by keywords. This bag contains tidbits of the mutable state of Sunflower. This means that I can write callback functions like this:

#(with-components [strings-model selected-dir]
   (.removeAllElements strings-model)
   (let [p (-> selected-dir htmls first parse)]
     (add-component :parsed p)
     (doseq [x (strings p)]
       (.addElement strings-model x))))

Name and conquer: having parts of state explicitly named mean that I can reliably access them from just about anywhere. This reduces confusion and allows for less tangled, more self-contained and understandable code.

A case for symbol capture

Clojure by default protects macro authors from incidentally capturing a local symbol. Stuart Halloway describes this in more detail, explaining why this is a Good Thing. However, sometimes this kind of symbol capture is called for. I’ve encountered one such case today while hacking a Swing application.

As I develop the app, I find new ways to express Swing concepts and interact with Swing objects in a more Clojuresque way, so a library of GUI macros and functions gets written. One of them is a wizard macro for easy creation of installer-like wizards, where there is a sequence of screens that can be navigated with Back and Next buttons at the bottom of the window.

The API (certainly not finished yet) currently looks like this:

(wizard & components)

where each Swing component corresponding to one wizard screen can be augmented by a supplementary map, which can contain, inter alia, a function to execute upon showing the screen in question.

Now, I want those functions to be able to access the Back and Next buttons in case they want to disable or enable them at need. I thus want the API user to be able to use two symbols, back-button and next-button, in the macro body, and have them bound to the corresponding buttons.

It is crucial that these bindings be lexical and not dynamic. If they were dynamic, they would be only effective during the definition of the wizard, but not when my closures are invoked later on. Thus, my implementation looks like this:

(defmacro wizard [& panels]
  `(let [~'back-button (button "< Back")
         ~'next-button (button "Next >")]
   (do-wizard ~'back-button ~'next-button ~(vec panels))))

where do-wizard is a private function implementing the actual wizard creation, and the ~'foo syntax forces symbol capture.

By the way, if all goes well, this blog post should be the first one syndicated to Planet Clojure. Hello, Planet Clojure readers!

Hiking in the Apennines

I’ve recently done a week-long hike in the Umbria-Marche region of the Italian Apennines (the vicinity of Monte Catria, near Cantiano, to be more precise), and here are some tips I’d like to share.

The Umbria-Marche Apennine doesn’t seem to be frequented by a lot of tourists, especially in mid-March. The information offices, although helpful, are often closed (this is not only the case with the mountain region: contrary to information available on the Web, the tourist information at Forli airport was closed on Sunday morning), and most of the Italians we’ve met didn’t speak English.
The tourist trails in the region are not well marked. Direction marks are nowhere to be found, nor are the signs visible on junctions. We had to ask the locals when leaving Cantiano for Monte Tenetra (and ended up on M. Alto instead anyway).
There are a lot of rifugi (mountain huts), but most of them are closed at this time of year. We passed by six or seven, out of which only one was available for sleep: Rifugio Fonte del Faggio (depicted), merely a small bothy with one worm-eaten bunk bed. Another one, Cupa delle Cotaline, with restaurant facilities and situated by a station of a local skilift, opened in the morning, but was closed for the night.

Rifugio Fonte del Faggio

The pitfalls of `lein swank`

A couple of weeks ago I finally got around to acquainting myself with Leiningen, one of the most popular build tools for Clojure. The thing that stopped me the most was that Leiningen uses Maven under the hood, which seemed a scary beast at first sight – but once I’ve overcome the initial fear, it turned out to be a quite simple and useful tool.

One feature in particular is very useful for Emacs users like me: lein swank. You define all dependencies in project.clj as usual, add a magical line to :dev-dependencies, then say

$ lein swank

and lo and behold, you can M-x slime-connect from your Emacs and have all the code at your disposal.

There is, however, an issue that you must be aware of when using lein swank: Leiningen uses a custom class loader – AntClassLoader to be more precise – to load the Java classes referenced by the code. Despite being a seemingly irrelevant thing – an implementation detail – this can bite you in a number of most surprising and obscure ways. Try evaluating the following code in a Leiningen REPL:

(str (.decode
       (java.nio.charset.Charset/forName "ISO-8859-2")
       (java.nio.ByteBuffer/wrap
         (into-array Byte/TYPE (map byte [-79 -26 -22])))))
==> "???"

The same code evaluated in a plain Clojure REPL will give you "ąćę", which is a string represented in ISO-8859-2 by the three bytes from the above snippet.

Whence the difference? Internally, each charset is represented as a unique instance of its specific class. These are loaded lazily as needed by the Charset/forName method. Presumably, the system class loader is used for that, and somewhere along the way a SecurityException gets thrown and caught.

Note also that there are parts of Java API which use the charset lookup under the hood and are thus vulnerable to the same problem, for example Reader constructors taking charset names. If you use clojure.contrib.duck-streams, then rebinding *default-encoding* will not work from a Leiningen REPL. Jars and überjars produced by Leiningen should be fine, though.

Downcasing strings

I just needed to convert a big (around 200 MB) text file, encoded in UTF-8 and containing Polish characters, all into lowercase. tr to the rescue, right? Well, not quite.

$ echo ŻŹŚÓŃŁĘĆĄ | tr A-ZĄĆĘŁŃÓŚŹŻ a-ząćęłńóśźż
żźśóńłęćą

Looks reasonable (apart from the fact that I need to specify an explicit character mapping — it would be handy to just have a lcase utility or suchlike); but here’s what happens on another random string:

$ echo abisyński | tr A-ZĄĆĘŁŃÓŚŹŻ a-ząćęłńóśźż
abisyŅski

I was just about to report this as a bug, when I spotted the following in the manual:

Currently tr fully supports only single-byte characters. Eventually it will support multibyte characters; when it does, the -C option will cause it to complement the set of characters, whereas -c will cause it to complement the set of values.

Turns out some of the basic tools don’t support multibyte encodings. dd conv=lcase, for instance, doesn’t even pretend to touch non-ASCII letters, and perl’s tr operator likewise fails miserably even when one specifies use utf8.

This is a sad, sad state of affairs. It’s 2010, UTF-8 has been around for seventeen years, and it’s still not supported by one of the core operating system components as other encodings are becoming more and more obsolete. I’m dreaming of the day my system uses it internally for everything.

Fortunately, not everything is broken. Gawk, for example, works:

$ echo koŃ i żÓłw | gawk '{ print tolower($0); }'
koń i żółw

and so does sed.

Update 2010-04-04: I should have been more specific. The above rant applies to the GNU tools (tr and dd) as found in most Linux distributions; other versions can be more featureful. As Alex Ott points out in an email comment, tr on OS X works as expected for characters outside of ASCII, and also supports character classes as in tr '[:upper:]' '[:lower:]'. This is yet another testimony to general high quality of Apple software; in this particular case, though, it may well be a direct effect of OS X’s BSD heritage. Does it work on *BSD?

Clojure SET

I’ve just taken a short breath off work to put some code on GitHub that I had written over one night some two months ago. It is an implementation of the Set game in Clojure, using Swing for GUI.

I do not have time to clean up or comment the code, so I’m leaving it as is for now; however, I hope that even in its current state it can be of interest, especially for Clojure learners.

Some random notes on the code:

Clojure is concise! The whole thing is just under 250 lines of code, complete with game logic and the GUI. Of these, the logic is about 50 LOC. Despite this it reads clearly and has been a pleasure to write, thanks to Clojure’s supports for sets as a data structure (in vein of the game’s title and theme).
There are no graphics included. All the drawing is done in the GUI part of code (I’ve replaced the canonical squiggle shape by a triangle and stripes by gradients, for the sake of easier drawing).
I’ve toyed around with different Swing layout managers for this game. Back in the days when I wrote in plain Java, I used to use TableLayout, but it has a non-free license; JGoodies Forms is also nice, but has a slightly more complicated API (and it’s an additional dependency, after all). In the end I’ve settled with the standard GridBagLayout, which is similar in spirit to those two, but requires more boilerplate to set up. As it turned out, simple macrology makes it quite pleasurable to use; see add-gridbag in the code for details.
Other things of interest might be my function to randomly shuffle seqs, which strikes a nice balance between simplicity/conciseness of implementation and randomness; and a useful debugging macro.

Comments?

Reactivation (and some ramblings on my blogging infrastructure)

This blog has not seen content updates in more than a year. Plenty of things can happen in such a long period, and in fact many aspect of my life have seen major changes over this time. I’m not, however, going to write a lengthy post about all that right now. Instead, I just would like to announce the reactivation of the blog.

You might have noticed that many things have changed. First, the blog has a new address: http://blog.danieljanus.pl; the address of the RSS feed has also changed and is now http://blog.danieljanus.pl/index.rss — please update your readers!

Probably the most important change is that you now may post comments under the entries, even though this blog continues to be just a bunch of static HTML pages. This is possible thanks to the Disqus service. I wonder whether it will encourage people to give feedback: I have received very few email comments since I started blogging. Also, the static calendar at the top of each page is gone, replaced by a bunch of links to archive posts.

I have long been considering changing Blosxom to something else. The main reason for such a step is that it’s written in Perl, which makes it particularly hard to debug upon encountering an unexpected behaviour. The single most irritating thing was that Blosxom would unexpectedly change the date of a post that was edited (which did not let me fix typos and other glitches); I found a patch for this somewhere, but lost it.

On the other hand, I really liked — and still like — Blosxom’s minimalistic approach and the ease of adding posts. (The very idea of installing a monstrosity such as Wordpress, with its gazillion of features I don’t need, posts kept in a database and what not, makes me feel dizzy.) I fiddled for a while with the thought of reimplementing Blosxom in Common Lisp, but that turned out to be a more time-consuming project than it initially seemed. So when I found The Unofficial Blosxom User Group and learned that, contrary to my belief, Blosxom is still actively maintained and has a thriving community, I ended up staying with the original Perl version, refining my installation so that it no longer gets in the way (this FAQ entry did the trick). I also rewrote all my source text files to Markdown, which made them vastly more readable and easy to edit, updating links and adding short followup notes where appropriate, but otherwise leaving old entries as they were.

I’d like to thank Maciek Pasternacki for inspiring me to finally get around to this. While my plans are not as ambitious as his — I am not courageous enough to publicly prove my perseverance, so my blogging will likely continue to be irregular — I plan to write more (having accumulated many ideas for blog posts) and I hope the periods of silence will be much shorter than hitherto.

I would like to take this opportunity to wish my readers all the best in the New Year!

Google Books

Yesterday, upon a midnight dreary, while I pondered, weak and weary, over a renowned volume of the olden lore (and specifically, upon one of the problems contained in the Polish translation of the first edition), I suddenly felt a need to consult the original version, to check whether there are no mistranslations or unincluded corrections for my copy. So I headed for Google Book Search, and apart from finding what I needed, I followed a link that sounded interesting. Quoth the link, “Groundbreaking Agreement”.

Basically, what it all boils to is two pieces of news — you guessed it, a good one and a bad one. The good news is that Google have come to agreement with several major U.S. publishers that will allow them to provide online access to digitized copies of out-of-print but still copyrighted books. Lots of books, and even though the service is not going to be free, that means all this richness will be at the fingertips — no more need to travel half the world to the Library of Congress to get one of the rare copies we’re after. Sounds cool, huh? Well, here comes the bad news: it will only be available to U.S. citizens.

Or will it?

I wonder how are they going to check for this precondition. IP-based geolocalization springs to mind. And unless they blacklist some IPs or restrict the credit cards used for payment, all I will need is some proxy on some server physically in the U.S. Say, a shell account on someone’s Linux box. I remember reading a Polish blog post about gaining access to American-exclusive content of some website (last.fm I believe it was) in a similar way. Hmm, hmm. We will see.

So, anyone got a shell account to spare?

anti-procrastination.el

Fighting procratination has been my major concern these days. I’ve devised a number of experimental tools to help me with that. One of them is called snafu and can generate reports of your activity throughout the whole day of work. It’s in a preliminary state, but works (at least since I’ve found and fixed a long-standing bug in it which would cause it to barf every now and then), and I already have a number of ideas for its further expansion.

Reports alone, however, do not quite muster enough motivation for work. I’m doing most of my editing/programming work in Emacs, so yesterday I grabbed the Emacs Lisp manual and came up with a couple of extra lines at the end of my .emacs.

;;; Written by Daniel Janus, 2008/12/18.
;;; This snippet is placed into the public domain.  Feel free
;;; to use it in any way you wish.  I am not responsible for
;;; any damage resulting from its usage.

(defvar store-last-modification-time t)
(defvar last-modification-time nil)
(defun mark-last-modification-time (beg end len)
  (let ((b1 (substring (buffer-name (current-buffer)) 0 1)))
    (when (and store-last-modification-time 
               (not (string= b1 " "))
               (not (string= b1 "*")))
      (setq last-modification-time (current-time)))))
(add-hook 'after-change-functions 'mark-last-modification-time)
(defun write-lmt ()
  (setq store-last-modification-time nil)
  (when last-modification-time
    (with-temp-file "/tmp/emacs-lmt"
      (multiple-value-bind (a b c) last-modification-time
        (princ a (current-buffer))
        (terpri (current-buffer))
        (princ b (current-buffer)))))
  (setq store-last-modification-time t))
(run-at-time nil 1 'write-lmt)

Every second (to change that to every 10 seconds, change the 1 to 10 in the last line) it creates a file named /tmp/emacs-lmt which contains the time of last modification of any non-system buffer.

That’s all there is to it, at least on the Emacs side. The other part is a simple shell script, which uses MPlayer to display a nag-screen for five seconds, and then give me some time to start doing anything useful before nagging me again:

#!/bin/bash
TIMEOUT=300
while true; do
   cat /tmp/emacs-lmt | (
      read a; read b; 
      c="`date +%s`"; 
      let x=c-65536*a-b; 
      if test $x -gt $TIMEOUT; 
          then mplayer -fs $HOME/p.avi; 
               sleep 15; 
      fi)
   sleep 1
done

The nag-screen in my case is an animation which I’ve created using MEncoder from a single frame which looks like this. Beware the expletives! (This is one of the few cases I find their usage justified, as the strong message bites the conscience more strongly.)

I’ve only been testing this setup for one day, but so far it’s working flawlessly: I got more done yesterday than for the two previous days combined, and that’s excluding the hour or so that took me to write these snippets.

If anyone else happens to give it a try, I’d love to hear any comments.

The immensely powerful tool

A pen and a sheet of paper are simple utilities; but there lies vast and sheer power in them that I was not aware of. Up until now. So what can they be used for that one might possibly not realize?

Short answer: serializing the stream of consciousness.

Yes, it’s simple, and you may laugh at me now. I myself am a little amazed why I haven’t noticed this before. But this answer lends itself to another question: what good is this serialization, and what exactly do I mean by it, anyway? And the answer to that is a little longer. So here goes.

I’m one of the people who tend to have problems with concentrating when thinking, especially when thinking hard. This is not to say that I am not capable of thinking hard: I am, but doing so requires a level of concentration that is tricky for me to exert for a prolonged period. (Unless, of course, I am in the state of absolute fascination, where this is taken care of subconsciously. But that’s another story.) More often than not, a tough problem requiring a significant amount of work just has to be dealt with. And then things start to distract attention. There is an itch to scratch, thoughts are shreds, each one pertaining to a tiny bit of the problem, but intertwined with hundreds of other bits of other problems, forming a dense, tangled web, hard to navigate over, and jumping fast from one to another, it becomes more and more unclear what’s next.

So what can one do? One way is to grab a writing device and just start writing. Running text is linear in nature, so you end up traversing the thought graph depth-first and writing down each thought as you traverse its node. And what’s more, translating ideas to written language slows you down, which is a Good Thing because it makes you see your way through the graph more consciously. It might take you longer to walk from point A to point B than to drive there by car, but definitely you will see more of the landscape as you go. Arriving at the final destination, or simply putting down the pen because enough thoughts have been collected and serialized (there’s never really any end of the stream), makes you end up with a half-product: an unsmithed lump of ore out of which you can forge ingots.

But why a pen and paper, as opposed to, say, a text editor? I think any writing utensil would work to some extent, but for me this seems to be the best option, for several reasons. First of all, I can type on the keyboard much faster than I can write legibly by hand, so this further slows down the pace (which is a Good Thing as we have observed already).

Second, there is something magical in handwriting which a text editor will never be able to achieve: it’s hard to describe. But the net effect is a very evident focus on Here and Now, the pen moving across the paper, the sheet filling up with more and more lines of script. This environment is naturally single-tasked: no Alt-Tab to press to switch to another terminal, no blinking icon of an instant-messaging program (unless a phone happens to ring). This causes synergy with the concentration caused by serializing thoughts.

If you have never tried this approach, feel free to do so. Although I cannot guarantee it will work for you, it certainly does work for me.

Who said Common Lisp programs cannot be small?

So, how much disk space does your average CL image eat up? A hundred megs? Fifty? Twenty? Five, perhaps, if you’re using LispWorks with a tree-shaker? Well then, how about this?

[nathell@chamsin salza2-2.0.4]$ ./cl-gzip closures.lisp test.gz
[nathell@chamsin salza2-2.0.4]$ gunzip test
[nathell@chamsin salza2-2.0.4]$ diff closures.lisp test 
[nathell@chamsin salza2-2.0.4]$ ls -l cl-gzip 
-rwxr-xr-x 1 nathell nathell 386356 2008-08-09 11:08 cl-gzip

That’s right. A standalone executable of a mini-gzip, written in Common Lisp, taking up under 400K! And it only depends on glibc and GMP, which are available by default on pretty much every Linux installation. (This is on a 32-bit x86 machine, by the way).

I used the most recent version of ECL for compiling this tiny example. The key to the size was configuring ECL with --disable-shared --enable-static CFLAGS="-Os -ffunction-sections -fdata-sections" LDFLAGS="-Wl,-gc-sections". This essentially gives you a poor man’s tree shaker for free at a linker level. And ECL in itself produces comparatively tiny code.

I build this example from Salza2’s source by loading the following code snippet:

(defvar salza '("package" "reset" "specials" 
"types" "checksum" "adler32" "crc32" "chains"
"bitstream" "matches" "compress" "huffman"
"closures" "compressor" "utilities" "zlib"
"gzip" "user"))

(defvar salza2 
  (mapcar (lambda (x) (format nil "~A.lisp" x))
          salza))

(defvar salza3 
  (mapcar (lambda (x) (format nil "~A.o" x))
          salza))

(defun build-cl-gzip ()
  (dolist (x salza2)
          (load x)
          (compile-file x :system-p t))
  (c:build-program 
   "cl-gzip" 
   :lisp-files salza3
   :epilogue-code 
     '(progn
       (in-package :salza2)
       (gzip-file (second (si::command-args))
                  (third (si::command-args))))))

(build-cl-gzip)

(Sadly enough, there’s no ASDF in here. I have yet to figure out how to leverage ASDF to build small binaries in this constrained environment.)

This gave me a standalone executable 1.2 meg in size. I then proceeded to compress it with UPX (with arguments --best --crp-ms=999999) and got the final result. How cool is that?

I am actively looking for a new job. If you happen to like my writings and think I might be just the right man for the team you’re building up, please feel free to consult my résumé or pass it on.

Update 2010-Jan-17: the above paragraph is no longer valid.

cl-morfeusz: A ninety minutes’ hack

Here’s what I came up with today, after no more than 90 minutes of coding (complete with comments and all):

MORFEUSZ> (morfeusz-analyse "zażółć gęślą jaźń")
((0 1 "zażółć" "zażółcić" "impt:sg:sec:perf")
 (1 2 "gęślą" "gęśl" "subst:sg:inst:f")
 (2 3 "jaźń" "jaźń" "subst:sg:nom.acc:f"))

This is cl-morfeusz in action, a Common Lisp interface to Morfeusz, the morphological analyser for Polish.

It’s a single Lisp file, so there’s no ASDF system definition or asdf-installability for now. I’m not putting it under version control, either. Or, should I say, not yet. When I get around to it, I plan to write a simple parser and write a Polish-language version of the text adventure that started it all.

Meanwhile, you may use cl-morfeusz for anything you wish (of course, as long as you comply with Morfeusz’s license). Have fun!

Update 2010-Jan-17: With the advent of UTF-8 support in CFFI, the ugly workarounds in the code are probably no longer necessary; I don’t have time to check it right now, though.

You win some, you lose some, you talk some

After my shameful performance in the previous tournament, this weekend saw my greatest achievement in tournament Scrabble to date: that of advancing to the quarterfinals of the Cup of Poland. For the record, here are the final standings. In the quarterfinal, I lost both games to Tomasz Zwoliński (the former Champion of Poland), who went on to win the Cup.

On Thursday, I will be delivering a presentation about the dark side of programming: error handling and how to cope up with Murphy’s law. The talk will last around 30 minutes and be held within TechAula, a place to hear about exciting and revolutionary technologies in software engineering. Feel invited to register and show up.

(Postscriptum 11 July: By public demand, the slides from my talk are now available for download.)

Another metapost

No, I am not going to write about the programming language for generating vector graphics. This is not a real post, but rather a note to self to write ones on certain topics once I get ready for that. And as for today’s title, I just couldn’t resist the pun. ;-)

I’ve been rewriting the Poliqarp Java (GUI) client for the last two weeks or so. The point is to convert it to use the new protocol, and to take the opportunity of turning a messy, kludgey and bit-rotting pile of code into a neatly decoupled, cleanly designed, robust and comprehensible utility. And the further I get, the more I see how much it’s worth it. I strongly believe that at the end of this path, upon remergence with the mainline, Poliqarp will become better than ever before.

Thus, when I have something to show (I hope to deliver a preliminary working version, though not yet feature-complete, within the upcoming week or so), I will probably brag about the design solutions I’ve taken. While I’m at it, I will possibly also write the long-delayed description of the new build system.

If there is anybody out there besides me who actually cares for that, stay tuned!

Today’s lesson: Mind the symlinks

Probably every day I keep learning new things, without even realizing it most of the time. The vast majority of them are minor or even tiny tidbits of knowledge; but even these might be worth noting down from time to time, especially when they are tiny pitfalls I’d fallen into and spent a couple of minutes getting out. By sharing them, I might hopefully prevent someone else for slipping and falling in.

So here’s a simple Unix question: If you enter a subdirectory of the current directory and back to .., where will you end up? The most obvious answer is, of course, “in the original directory”, and is mostly correct. But is it always? Let’s see.

nathell@breeze:~$ pwd
/home/nathell
nathell@breeze:~$ cd foobar
nathell@breeze:~/foobar$ cd ..
nathell@breeze:~$ pwd
/home/nathell

So the hypothesis seems to be right. But let’s try doing this in Python, just for the heck of it:

nathell@breeze:~$ python
Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> print os.getcwd()
/home/nathell
>>> os.chdir("foobar")
>>> os.chdir("..")
>>> print os.getcwd()
/var

Whoa, hang on! What’s that /var doing there? Of course the one thing I didn’t tell you is that foobar is not really a directory, but rather a symlink pointing to one (/var/log in this case).

The corollary is that the shell builtin cd is not the same as Unix chdir() (it is easily checked that both Perl and C exhibit the same behaviour). In fact, the shell builtin has an oft-forgotten command-line switch, -P, which causes it to follow physical instead of logical path structure.

On a closing note: I have somewhat neglected the blog throughout the previous month, but I hope to revive it soon. It is not unlikely that such irregularities will recur.

Recently read #1: Akhmatova meets Bashō (Vasil Bykaŭ, “The Wall”)

(Introductory note: This post marks the beginning of a new series on this blog, aptly titled “Recently read.” Every now and then I will try to verbalize afterthoughts inspired by the books I happen to read, and post them here. I hope these recommendations or anti-recommmendations might turn out to be useful for someone.)

Give me
a kiss to build a dream on,
and my imagination
will thrive upon that kiss;
sweetheart,
I ask no more than this —
a kiss to build a dream on.

Thusly starts the Fallout 2 intro — a mini-movie that can be considered a piece of art in its own right. Louis Armstrong sings these words in an abandoned underground cinema, wherein a movie is displayed, touching on nostalgia for pre-War times as well as severe dangers that lurk on the surface of the earth. And then come these words…

“War, war never changes.”

Just like throughout the entire Fallout saga, these words reverberated in my mind as I read “The Wall,” a collection of short stories by Vasil Bykaŭ, the late Belarussian writer.

For the war indeed does not change. And wherever it appears, it carries around such an amount of destruction and utter wrongness that it is next to unimaginable for a generation grown up in a relatively peaceful place and time such as ours. In fact, even the words “utter wrongness” do not do justice to what was once an unescapable reality. There is only one way to find appropriate words: to show it, show it without overlooking anything, show it dryly and aloofly in all its hideousness.

And Bykaŭ does. There is not a single word of moralizing in these stories. There are no high words, and barely even a human thought beyond fear for life. There is pure depiction; and yet every word in this depiction stands firm and cannot be removed without losing the level of detail called for. This induces associations with Anna Akhmatova and her famous “Requiem,” which begins as follows:

Это было, когда улыбался
только мертвый, спокойствию рад.

“Это было.” “It happened.” These simple words are immensely powerful. And the same two words, unspoken, echo throughout the entire book. It happened, and it was like this. Nothing more can, or should, be told.

I’ve mentioned the level of detail; this aspect of these stories deserves longer comment. It is imminent that they would not be quite as powerful were it not for the detail. One can almost sense the chill of a dawn rising up above some godforsaken trench somewhere on the battlefront. Or shudder at the cold dampness of the soil inside it. Or smell the stench of decay rising above a corpse shot several days ago. Or feel the almost palpable fear floating in the crossfire of danger, one on the enemy side, the other shaped as one’s own commandment. Or the gloom with which a small group of soldiers sets out to dig their final resting place before committing suicide, lest worse fate befall them.

These stories are almost “haikuistic,” so to speak, in that each one of them resembles a very thinly cut and faithfully portrayed slice of reality from which there is no escape. This shows most strongly in case of the shortest ones, like “The Hill,” which are just several pages long, but it arguably holds even for the longest text in the book, the opening novella, “Love Me, Soldier.” (In which, by the way, Falloutesque associations are particularly strong: imagine a Belarussian sergeant who finds a fellow countrygirl hiding in a village in Austria, at the very end of the war, and falls in love with her. Doesn’t that sound like “a kiss to build a dream on?” Well, in Bykaŭ’s world, good dreams never come true.)

I quoted Akhmatova’s “it happened.” Yet, perhaps, the most striking and saddening impression from “The Wall” as a whole, and one that sets it apart from other war literature, is that this should really read “it still happens.” For the war has ended, but it takes long for a nation to recover from the scars it left; especially the Belarussian nation, who have been held captive by various regimes for too long and have never actually experienced freedom that we take for granted. Once a wound has been healed, it is all too easy to reopen it. And fear for speaking one’s own language remains the same, war or no war.

No wonder Bykaŭ’s writings are still censored in his homeland. Thanks to the Internet, though, the full Polish text of all the stories is available online. Highly recommended.

Inward ripeness

How soon hath Time, the subtle thief of youth,
Stol’n on his wing my three and twentieth year!
My hasting days fly on with full career,
But my late spring no bud or blossom shew’th.
Perhaps my semblance might deceive the truth
That I to manhood am arrived so near;
And inward ripeness doth much less appear,
That some more timely-happy spirits endu’th.
Yet be it less or more, or soon or slow,
It shall be still in strictest measure ev’n
To that same lot, however mean or high,
Toward which Time leads me, and the will of Heav’n:

All is, if I have grace to use it so,
As ever in my great Task-Master’s eye.

John Milton is said to have composed this sonnet on his twenty-fourth birthday, and his thoughts (including, but not limited to, the criticism of the achievements so far) are very much in line with mine on my very own 24th birthday. One wonders what is the programming equivalent of “Paradise Lost.”

cl-netstrings

I’ve just packaged up the Common Lisp netstring handling code that I wrote a week ago into a neat library. Unsurprisingly enough, it is called cl-netstrings and has its own home on the Web. It’s even asdf-installable! I wonder whether this one turns out to be useful for anybody besides me…

The other thing I’ve been working on is a new build system for Poliqarp. But that’s the story for another post — most probably I will write about it when it gets out of a state of constant flux.

Update 2010-Jan-17: cl-netstrings is now hosted on GitHub; I’ve updated the link.

Best OS ever

If you are reading this on a box that does not have an impressive amount of RAM (say, 512 MB or less) and is running a fairly recent Linux, then for goodness sake, drop everything you are doing right now and follow the instructions in this entry. I’m going to show you how to make your system use the memory in a more efficient way, yielding an effect almost equivalent to increasing its amount — with no expenses whatsoever! Sounds good? Read on.

You see, there’s this Linux kernel module for kernels 2.6.17 and up (that’s what the phrase fairly recent in the previous paragraph macroexpands to), called Compcache. It works by slicing out a contiguous chunk of your RAM (25% by default, but it’s settable, of course) and setting it up as a swap space with highmost priority. The trick is that pages that are swapped out to this area are compressed using the LZO algorithm, which provides very fast compression/decompression while maintaining a decent compression ratio. In this way, more unused pages can fit in memory, and less of them are swapped out to disk, which can considerably cut down disk swap usage. I’ve enabled it in my system and it doesn’t seem to cause any problems, while providing a visible efficiency boost. Here’s how I did it on a freshly-installed Ubuntu Hardy:

I installed the Ubuntu package build-essential, then downloaded Compcache from its site, extracted it, entered its directory and compiled it by saying make. So far, so easy.
Unfortunately, one cannot say make install — creating a flexible cross-distro install target is admittedly hard. So I installed it by hand, ensuring that my system enables it automatically on boot-up.
I created a directory /lib/modules/2.6.24-16-generic/ubuntu/compcache/ and copied the four kernel modules (compcache.ko, lzo1x_compress.ko, lzo1x_decompress.ko, and tlsf.ko) created by the compilation to that directory.
Next, I ran depmod -a to make the modules loadable by modprobe.
I edited the file /etc/modules and added a line at the end, containing the single word compcache.
I copied the shell scripts use_compcache.sh and unuse_compcache.sh that come with compcache to /usr/local/bin.

I created an executable script /etc/init.d/compcache with the following contents:

#!/bin/sh
case "$1" in
  start)
    /usr/local/bin/use_compcache.sh ;;
  stop)
    /usr/local/bin/unuse_compcache.sh ;;
esac

The last step was to create a symlink /etc/rc2.d/S02compcache pointing to that script.

I then rebooted the system and verified that the new swapspace is in use:

nathell@chamsin:~$ cat /proc/swaps
Filename        Type        Size    Used    Priority
/dev/sdb2       partition   996020  0       -1
/dev/ramzswap0  partition   128896  111396  100

With the final release of Hardy installed on my main box and compcache optimizing its memory usage, I do not hesitate to call this combo the best OS I have ever had installed.

And no, I don’t own a Mac. :-/

Hacking away with JSON-RPC

Let’s try:

(let ((s (socket-stream 
          (socket-connect "localhost" 10081 
                          :element-type '(unsigned-byte 8)))))
  (write-netstring "{\"method\":\"ping\",\"params\":[],\"id\":1}" s)
  (finish-output s)
  (princ (read-netstring s))
  (close s))
{ "result": "pong" }
--> T

Yay! This is Common Lisp talking to a JSON-RPC server written in C. This means that I have now the foundations for rewriting Poliqarp on top of JSON-RPC (according to the protocol spec I have recently posted) up and running, and all that remains is to fill the remainder.

Well, to be honest, this is not exactly JSON-RPC. First off, as you might have noticed, the above snippet of code sends JSON-RPC requests as netstrings. This is actually intentional, and the reasons for adopting this encoding have been described in detail in the spec (it basically boils down to the fact that it greatly simplifies reading from and writing to network, especially in C). I wrote some crude code to handle netstrings in CL — now it occurred to me that it might actually be worthwhile to polish it up a little, write some documentation and put on CLiki as an asdf-installable library. I’ll probably get on to this quite soon.

Second, the resulting JSON object does not have all the necessary stuff. It contains the result, but not the error or id (as mandated by the JSON-RPC spec). This is actually a deficiency of the JSON-RPC C library I’m currently using. It places the burden of constructing objects that are proper JSON-RPC responses on the programmer, instead of doing that itself. This will be easy to sort out, however, because the library is small and adheres to the KISS principle. More of a problem is that the licensing of that library is unclear; I emailed the maintainers to explain the status.

Forgetting

It has just occurred to me that the best way to throwing things out of one’s mind is to let it be absorbed by something else. I guess this is oft-overlooked fact, even though it seems to be quite obvious. In particular, forcing oneself not to think about something is not a wise strategy, since it leads to mental strain and thinking more and more, eventually yielding dejectedness that can be hard to get over.

And what could be more absorbing than debugging a SIGSEGV buried deeply in the innards of some library early in the morning? ;-)

ECLM 2008

What is there left for me to do in this life?
Did I achieve what I had set in my sights?
Am I a happy man, or is this sinking sand?
Was it all worth it?—was it all worth it?
— Queen

Gusts of moderate wind are blowing at my face through the open window of a train from Amsterdam to Warsaw as I write these words. The last buildings of Amsterdam have vanished a while ago, giving ground to the damp, low countryside of the Netherlands — not quite fascinating sight to be watching — so I decided to fire up my laptop and write down some impressions while they are sharp and vivid — impressions from the European Common Lisp Meeting that was held in Amsterdam yesterday.

I was there with Maciek and Richard. Amsterdam did not receive us warmly, pouring some mild yet cold rain on us, but our hosts — Lispniks from the Hague, Gabriele and Victor from Streamtech — turned out to be really nice guys. I’m not going to go into a very detailed description of the social aspects of our trip, instead focusing on the conference itself. And that is definitely a topic worth talking about for a long time.

The first man to speak was Jeremy Jones of Clozure Associates, talking about InspireData and how they did it in Lisp. Although they also seem to be the people behind Clozure CL the implementation of Common Lisp, InspireData, the product their presentation was about, seems to have been written in LispWorks. It is a quite interesting application for browsing datasets in many interesting ways and draw conclusions for them. Jeremy started off with a demonstration presenting the key ideas of InspireData and what it can do, and this almost instantly hooked the attention of most of the gathered Lispers; mine, at least, definitely. First off, it seems to be quite a nice success story of a real-world application of Lisp, well worth learning about and mentioning where it deserves a mention. Second, one of its great features shown by the demo is that one can copy HTML tables from a Web browser and paste them as InspireData datasets. Given that Poliqarp now has statistical features and can export its data to HTML, I wonder whether it is possible to couple it with InspireData to interactively explore linguistic material in an absorbing way. That’s certainly a topic worthy of further research.

And last but not least, Jeremy outlined the points they did wrong and those they got right. Among those latter were two letters that now constitute a huge part of my professional life: QA. He just couldn’t emphasize enough how crucial the fact that they had a serious quality assurance process from the very beginning proved to yield the final quality of the product. That’s the lesson I’m now quickly learning. When I learned that InspireData was mostly tested by hand by a skilled QA team, I felt somewhat proud of being able to automate large parts of the process at Sentivision. I’m very curious where this path will lead me to. Let’s hope for the best!

The next speaker was Nicolas Neuss of University of Karlsruhe, talking about Femlisp, a framework for solving partial differential equations in Common Lisp. I have little to say about this one, since I lack the mathematical background needed to fully comprehend and appreciate the topic; it’s just not my kettle of fish. Undoubtedly, though, Femlisp seems to be filling its niche in a neat way, as the demonstrations showed.

After a coffee break, Stefan Richter came up with the one presentation that I’ve been looking forward to the most; that of using Common Lisp for large, scalable Internet systems. After all the talks were over, Maciek dubbed it a “very nice anti-FUD presentation” and I could not agree more. I didn’t learn many new things from it, but the author clearly knows how to attempt to convince non-Lispers to try out Lisp. The talk started off with outlining the typical designs of Web apps and portals, starting with simple one-server scenarios that don’t scale well and progressing in the direction of more scalable and extensible ones. Stefan then pointed out that in some mainstream languages like Java there exists a mature and proven infrastructure for employing such designs. And then came the key point — that this is the case also for Common Lisp! All the necessary tools are there, ready to use Right Now and free for the most part; they’re just not as mature in some cases. This is not much of a problem, though, given the incremental nature of Lisp development: any problem at hand is typically fixable much faster than in case of other languages. The only weird thing was that the author advocated using continuation-based Web frameworks (such as Weblocks or UnCommon Web) just a couple of minutes after discouraging using sticky sessions.

Next came Killian Sprotte with a speech about PWGL, the program for computer-aided music composing. I have very mixed feelings about it. Notice that I didn’t use the word “presentation” — there was no presentation at all. Yes, that’s right. The speaker was just talking and showing off various things in the musical engine. Now, having no presentation accompanying the talk is not necessarily a bad thing in itself; but without one, it’s a little harder to draw attention of the audience and a whole lot easier to deliver a chaotic talk instead of a cleanly-structured and well-organized one. Such was the case with this speech. Some features were shown, but with a fair amount of obscurity and boredom thrown in, leaving me with a rather low overall impression.

As for PWGL itself, some of the ideas employed in it seem a bit peculiar (for want of a better word) to me. As befits a Lisp program, it is an extensible utility that actually allows users to program music just as one programs software. But the way that programming is done… well, think of a graphical editor for Lisp programs. An editor in which to write, say, a factorial function, you right-click an initially blank sheet, select defun from a pop-up with a complicated set of menus and submenus… and kaboom! up comes a box divided into several sub-boxes. They correspond to — what else they could? — the name of the function, list of arguments, and a body. You can draw boxes representing computations, drag them around and link them with arrows — this is supposed to build complicated expressions out of simpler ones. And there is a huge library of musical tools, all available for the convenience of a programmer. Or, should I say, a composer.

Sounds cool? Maybe — for a newbie. I can’t really say. As someone who has high experience with Lisp and programming in general, I can only speak for myself. And for me all this click-and-drag-programming seems to be an unnecessarily tedious, obscure and error-prone way of doing things. Stuff like score editors, chord editors or various transformations is admittedly cool, but for lower-level matters the kind of visualization PWGL offers (and it obviously has its rough edges) seems to get in the way rather than staying out of it. But perhaps that’s just me?

By the time the fourth talk ended, most Lispers were already hungry, so a lunch break followed. I talked to some guy (I don’t remember his name, alas) who’s working on porting Clozure CL to 64-bit Windows. This is great news — when the port’s complete, it has high chances of becoming the free Common Lisp implementation of choice for many Windows Lisp hackers.

Juan José García-Ripoll then talked about ECL, another CL implementation that is characterized by a fairly small memory and disk footprint, while still managing to achieve decent performance (via compilation to C) and good standard compliance. It was good to see that ECL is still quite alive and getting better and better with each release. Just for the heck of it, I attempted in the evening to reproduce the problem I had with ECL a while ago on a fresh CVS checkout. I managed to reproduce it (for the curious, it was an issue with ECL failing to build after having been configured with the option --disable-shared). So I reported the bug to Juan, and he promised to look into it within the next days. And I must say that reporting bugs IRL to open source projects’ maintainers is a very nice experience. :-)

And then came a really big surprise, and I mean a nice surprise. It took the form of Kristofer Kvello of Selvaag, a Norwegian-based house-building company, and his presentation on House Designer, a Common Lisp program for aiding in designing residences, as the name suggests. Yet another example of a success story in an area CL can really excel at. Basically, what House Designer can do is that you give it a sketch, containing a rough description of the shape of a flat or residence and layout of rooms, and out comes a very detailed project with all sorts of bells and whistles: the program automatically figures out what the number of windows should be and where they should be located, the number and location of electric outlets, the optimal types of walls, layout of water installation and what not. It’s transfixing when you think of the sheer amount of tedious labour it automates, taking into account all of the professional knowledge about designing houses accumulated over years, some parts of which a human can easily omit. And it’s been Lisp all the lifetime of this project, and it’s Lisp all the way down (except for the GUI in Java)! Very, very impressive!

Marc Battyani’s talk about programming FPGAs in Lisp probably should not have been stacked so late in the venue. I mean, the topic seems to be quite interesting (though a bit low-level for my interests), but there was something about Marc’s way of talking and showing things that sent me off dozing almost instantaneously. I’d been a bit tired after the many hours of sitting and listening to speeches, especially after having woken up at six o’clock, and so I somewhat regret missing large parts of the talk. It’s nice to know, though, that it is possible to do such things with Lisp. Seems to have a high hack value, as in: “Why do it this way? Because we can!”

And what better end of a conference could one ask for than a rant by Kenny Tilton? If you have only encountered Kenny on the Usenet (some of the crème de la crème of his postings is meticulously collected by Maciek) and think he’s one heck of a freak, you definitely should listen to him live. Here was another talk without slides — just talking and demonstrating stuff — but this time, it was a totally different thing. Kenny sure knows how to attract the attention of the audience and how not to let it loose throughout an hour’s worth of talking. And he changes topics with mastery, using digressions to a great effect to avoid the boredom slipping in, caused by bragging about one thing all the time. There was Cells in that talk, there was teaching of algebra, and there was high-speed driving through the streets of New York. I only hope someone has recorded that to put it online.

So, this was it. There was much talk afterwards, there was much beer, there was much socializing, there was much rejoicing. I saw a real XO-1 and played with it for a while, and boy, isn’t it cute! And then we all came back. And here I am, sitting at my desk in Warsaw (it’s the next day already; I really wish my laptop had a better battery), finishing up this longish blog entry and asking myself: was this 50 euro well spent?

Yes, it was a worthwhile experience, hahhahahahahhaaaa!
(evil chuckle à la Kenny)
It was worth it!

Poliqarp’s new protocol

The first version of the document I’ve been writing about a couple of days ago is now ready for public review. I’ll be making an initial attempt at the implementation once I return from the European Common Lisp Meeting ‘08 and write a report.

I’m not playing this stupid game anymore

Not until the next tournament, that is. My achievements in the 12^th Scrabble Championship of Warsaw can be described as “mediocre” at best; four won, one drawn and seven lost games mean that my general rating will drop down by two points or so. Oh well. Everybody knows it’s a stupid game. ;-) At least I’ve managed to get a decent small score, with an average of 377 points per game.

Random resolutions for the indefinite future:

Get a final draft of the C++09 standard when it’s ready and acquaint myself with it as closely as possible. I strongly dislike C++ (and I’m not alone in this — see the Frequently Questioned Answers about C++ for very detailed criticisms); however, I’ve long wanted to learn that language better just to know all the strengths and weaknesses of the enemy. The ideal moment for this will be when the new standard is out; this will give me the advantage of not having to unlearn the things changed by the standard, while staying on a cutting and competitive edge.
Get a copy of Federico García Lorca’s poems translated into Polish by Jerzy Ficowski. I have only a very vague knowledge of Lorca (just his Romance of the Spanish Civil Guard (Romance de la Guardia Civil Española)), but I very much like what little I know.

Recipe for a successful presentation

LaTeX + Beamer (for typesetting the presentation in a visually pleasant, clean, simple and consistent way) + KeyJNote (for presenting it stylishly to the audience) = a recipe for success. In particular, KeyJNote, which I found only yesterday, seems to be a fine and tremendously useful piece of software, despite being very young. The only annoyance I have found in it is that it doesn’t respond to Alt-Tab when in fullscreen mode. On the typographical side, I used the progressbar Beamer theme and the Torunian Antiqua font, both to great effect.

While I’m at this topic, Jan Rychter has recently posted a great guide to giving presentations, especially short ones. I heartily recommend it to those of you who speak Polish (is there actually any non-Polish-speaking person reading this?)

Update 2010-Jan-17: KeyJNote is now called Impressive.

Ubuntu post-installation tricks

Yesterday, my level of frustration with my old operating system at work exceeded a critical point, and I installed a fresh daily build of the not-yet-released Ubuntu 8.04 in place of it. Then, in addition to usual post-installation chores like setting up mail, hardware, etc., I performed a couple of steps to make the system more pleasurable to use. Here’s what I did, just in case someone finds this useful.

First, I tweaked the font rendering. This was one area that has long been a PITA for Linux users (at least for me, since 2000 or so), but as far as Ubuntu is concerned, they introduced a change to Freetype somewhere along the way between Feisty and Gutsy which, when set up properly, makes the font rendering on LCD displays far superior for me to that of, say, Windows XP, in particular at small font sizes. The way to enable it is to enable sub-pixel rendering, and set the hinting level to “slight.” This results in an rendering very close to what the author of Texts Rasterization Exposures managed to achieve.
I installed the package msttcorefonts to get Microsoft’s free-as-in-beer set of core TrueType fonts, including Times New Roman, Arial, Georgia, etc. There are very many sites out there on the Web that were designed with these fonts in mind, and this is one of the few areas Microsoft doesn’t completely suck at.
Next I enabled bitmap fonts. The way to do this is to become root, cd to /etc/fonts/conf.d, remove the symlink named 70-no-bitmaps.conf, and make a symlink pointing to /etc/fonts/conf.avail/70-yes-bitmaps.conf instead. This would come in handy in the next step.
Which was installing my favourite console font. Unfortunately, it doesn’t come preinstalled with the Gnome-based Ubuntu, but it was no big deal. The font is named console8x16 and it comes with Kubuntu’s (and KDE’s) default terminal emulator, Konsole. So I downloaded an appropriate package (manually, without the help of APT, because all I wanted was the font, not the package itself). I then installed Midnight Commander (which I use a lot, if only for its great vfs feature, which allows to access, inter alia, Debian/Ubuntu packages as if they were directories), grabbed the file console8x16.pcf.gz, installed it in /usr/share/fonts/X11/misc, changed to that directory, ran mkfontdir and mkfontscale, logged out and restarted the X server.
The last step was to use this font for Emacs, too. So I installed Emacs, created the file ~/.Xdefaults containing the single line
```
Emacs*font: -misc-console-medium-r-normal--16-160-72-72-c-80-iso10646-1
```
and ran xrdb ~/.Xdefaults.

Then I got round to configuring Emacs itself. But that’s a story for another post.

YAVP

Jezus, the high elven wizard, saved the world with his brave efforts and became a great ruler while saving himself 34 times.
He scored 24999532 points and advanced to level 50.
He survived for 0 years, 123 days, 0 hours, 11 minutes and 39 seconds (176207 turns).
Jezus visited 127 places.
His strength score was modified by +26 during his career.
His learning score was modified by +17 during his career.
His willpower score was modified by +10 during his career.
His dexterity score was modified by +7 during his career.
His toughness score was modified by +25 during his career.
His charisma score was modified by +9 during his career.
His appearance score was modified by +6 during his career.
His mana score was modified by +13 during his career.
His perception score was modified by +15 during his career.
He was unnaturally aged by 76 years.
He was the champion of the arena.
He was a member of the thieves guild.
He made a little water dragon very happy.
He defeated the arch enemy of a mighty karmic wyrm.
He adhered to the principles of the Cat Lord and thus rose to great fame.
He saved Khelavaster from certain death.
He left the Drakalor Chain after completing his quest and became a great leader and famous hero.

Yay! Now that I have finished ADOM for the first time ever, after something like six years of trying, I can finally get back to work with peace of mind. :-)

The TeX Hackery

After a longish while of inactivity, I finally got around to finishing the draft spec of a next-generation protocol for Poliqarp, the be-all-end-all corpus concordance tool that I maintain. The spec is being written in LaTeX, and it has a number of subsections that describe particular methods of the protocol. Each one of those is further divided into sub-subsections that describe the method’s signature, purpose, syntax of request, syntax of response, and an optional example. I thought to write a couple of macros to help me separate the document’s logic from details of formatting, so that I could say:

\synopsis/() -> {version : int; extensions : string*}/

and have it expanded into:

\paragraph{Synopsis}
\verb/{version : int; extensions : string*}/

Being a casual LaTeX user who hardly ever writes his own macros, I first thought to use LaTeX’s command-defining commands, \newcommand and \renewcommand. However, I quickly ran into the limitation that the argument of commands defined in such a way can only be delimited by curly braces, which I could not use because they might appear in the argument itself.

I googled around and found that this limitation can be overcome by using \def instead, which is not a LaTeX macro but rather an incantation of plain TeX, and allows to use arbitrary syntax for delimiting arguments. Having found that, my first shot was:

\def\synopsis/#1/{\paragraph{Synopsis}\verb/#1/}

which, obviously enough, turned out not to work, producing errors about \verb ended by an end-of-line.

“What the heck?” I thought, and resorted to Google again, this time searching for tex macros expanding to verb. This yielded an entry from some TeX FAQ, which basically states that the \verb is a “fragile” command, and as such it cannot appear in bodies of macros. Ook. So it can’t be done?

“But,” I thought, “TeX is such a flexible and powerful tool, there must be some way around this!” And, as it would turn out, there is. Yet more googling led me to this thread on comp.text.tex, where someone gives the following answer for a similar question:

\def\term#{% 
   \afterassignment\Term \let\TErm= }% 

\edef\Term{\noexpand\verb \string}}

Now this is overkill. Why in the world am I forced to stuff such incomprehensible hackery into my document just to perform a seemingly simple task?! Easy things should be easy — that’s one of the principles of good design.

Reluctantly, I copied it over, and attempted to adjust it to my needs. After a number of initial failed attempts, I thought that I might actually attempt to understand what all these \afterassignment’s, \noexpand’s and \edef’s are for, so I downloaded the TeXbook and dived straight in.

I spent another fifteen minutes or so reading bits of it and trying to understand tokens, macros, when they are expanded and when merely carried over, etc. But a sparkle of thought made me replace the whole complicated thingy with a simple snippet that actually worked.

\def\synopsis{\paragraph{Synopsis}\verb}

That’s right. This superficially resembles a C preprocessor macro, and works because I was lucky enough to have \verb appear last in the definition, thus allowing the “arguments” of \synopsis to be specified just like arguments to \verb and fit at exactly right place. I’m almost certain that it does not always work this way, but for now it’ll suffice.

Oh well. TeX is undoubtedly a fine piece of software that provides splendid results if used right. But I can’t get over the impressions that there are a great deal more idiosyncracies like this in it than in, say, Common Lisp, even though the latter’s heritage tracks back to as early as 1958 and is a whopping twenty years longer than TeX’s. (On the side note, as it turns out, someone has already written a Lisp-based preprocessor for TeX macros. Gotta check it out someday.)

As for the TeXbook itself: it is a fine piece of documentation that I will definitely have to add to my must-read list, though it admittedly has a math-textbookish feel to it. First, however, I want to finish “Shaman’s Crossing” by Robin Hobb (which I will probably brag about in a separate post once I’m finished with it) and tackle Christian Queinnec’s “Lisp in Small Pieces”.

Introduction

So, there. Inspired by the newly-started blogs of some of my acquaintances, I had this thought that I might actually have a word or two on a number of subjects, and that it might even be worth sharing. And here I am, typing this introductory entry in my Emacs.

I guess the first thing one usually writes about in a blog is introducing himself to the public, so for those of you who have arrived here through some links on the Web and don’t know me, here goes. I am a 23-year-old (soon to be 24, though) programmer geek, living in Warsaw, Poland. In 2006, I graduated from Warsaw University, where I majored in computer science, and am now working full-time as a senior software engineer at Sentivision. I have a homepage (currently in Polish only, though I probably will translate it to English some day). These days, I tend to use Common Lisp for most of my programming work, though I also occassionally use Python, Ruby, C, Perl, OCaml, Haskell, Java, and a handful of other languages.

This is going to be a blog about my personal interests. This means mostly programming, with a fair share of posts about books, poetry, Scrabble, music, biking, cats, and a bunch of other topics thrown in. The new entries will most likely be added irregularly, whenever I feel like sharing a thought. My mother tongue is Polish, but I will try to maintain this blog in English just to polish up my English writing skills (no pun intended) and for greater worldwide understandability.

As you might have noticed, there is no possibility of leaving comments. This is a side-effect of the fact that this blog is, technically, just a bunch of static HTML pages (automatically generated with Blosxom), and is in line with my idea of blog comments: I view them as a way of providing direct feedback to the author, not as a publically available message board with discussions having a heavy tendency to drift off topic. So, should you like to comment on some post, feel free to drop me an email; I’ll be happy to respond to interesting mails in the blog. I can be contacted at dj at danieljanus dot pl.

Update 2010-Jan-17: Several things have changed — in particular, Sentivision doesn’t exist anymore — but I haven’t done any editing in this post other than updating the links.