cl build

A script to update/install the latest versions of all the most important Common Lisp packages.

### About

clbuild is a shell script helping with the download, compilation, an invocation of Common Lisp applications. It defaults to SBCL but otherwise tries to be somewhat independent of your local environment.

clbuild was originally written by Luke Gorrie. (Idea from jhbuild by James Henstridge, a Gnome hacker).

The last time someone updated this web page, the following libraries and applications were supported as part of the main project list:

acl-compat adw-charting alexandria arnesi babel beirc bordeaux-threads cffi chipz chunga cl+ssl cl-base64 cl-dot cl-fad cl-irc cl-pdf cl-ppcre cl-typesetting cl-vectors cl-webdav cl-who cl-yacc clim-alerts climacs climplayer closer-mop closure closure-common closure-html clsql clx cxml cxml-rng cxml-stp drakma fiveam flexi-streams flexichain graphic-forms gsharp hunchentoot ieee-floats ironclad iterate ltk lw-compat mcclim md5 mel-base midi mw-equiv net-telent-date parse-number plexippus-xpath postmodern psgraph puri redshank rfc2388 rt salza salza2 series skippy slime spatial-trees split-sequence trivial-features trivial-gray-streams trivial-sockets trivial-utf-8 url-rewrite usocket vecto zip zpb-ttf zpng

And the following projects were part of the wnnp list (work-needing and prospective projects):

cells cells-gtk cffi-net cl-opengl clawk clg cljl eclipse ftd gtk-ffi lisa parse-html ph-maths pod-utils regex s-base64 s-http-client s-sysdeps s-utils s-xml s-xml-rpc xuriella zebu

Please send patches bug reports to clbuild-devel@common-lisp.net (list information).

### News

2008-01-20 In addition to the main projects list, we now have a list of projects that are not downloaded by default, called WNPP projects (work-needing and prospective projects). Acceptance criteria for WNPP projects are less strict than for main projects, allowing us to add more projects than previously.

2007-10-07 David's repository moved from http://www.lichteblau.com/blubba/clbuild to http://common-lisp.net/project/clbuild/clbuild. Project members should be able to push into this repository.

### FAQ

How does clbuild differ from asdf-install?

clbuild includes a list of selected libraries and applications and their well-known locations, and downloads the newest version or CVS/SVN/darcs repositories if possible. It can also build SBCL for you and includes convenient commands to start various applications.

In contrast, asdf-install can install any software listed on Cliki. It always downloads release tarballs. (It is up to the user to hunt down the trustworthy PGP keys as a protection agains malicous wiki entries.)

My favourite application is not supported. How can I add it?

Just add it to `clbuild/wnpp-projects`. Please send a note to the mailing list so that we can include it, too. (See below for notes on the dependencies file.)

Why did you get rid of all tarball-only downloads?

We use version-controlled download mechanisms to avoid losing local changes when updating.

clbuild's source directory is meant as an area where hackers can do their work, and we wouldn't feel confident recommending it for that purpose if we had to delete and move directories or extract tarballs in there.

Instead, we have the clbuild mirror with darcs conversions of tarball-only projects.

(Future versions of clbuild might do that darcs conversion on-the-fly somewhere in a new directory clbuild/import, and then use darcs pull from that directory into clbuild/source. The downside of that approach would be that different users (and separate clbuild checkouts by the same user) would have incompatible darcs checkouts for the same project.)

It doesn't load my ~/.sbclrc!

Yes, that's the default. But you can change it in clbuild.conf using the USER_INIT configuration variable.

"clbuild lisp" doesn't seem to use my monster.core!

Yes, use "clbuild preloaded" if you want that.

Can I start the lisp with swank already preloaded? (Or: "clbuild slime" doesn't seem to load my monster.core)

Yes, you can change that behaviour in clbuild.conf using the START_SLIME_USING_CORE configuration variable. To preload swank, you can run "clbuild dumpcore slime" before doing that.

Can I run emacs and slime without going through "clbuild slime"?

Yes, just run "clbuild slime-configuration". It prints a .emacs excerpt. Copy&paste it into your .emacs.

Is the "dependencies" file autogenerated?

Yes, using "clbuild record-dependencies", which requires a complete check-out of all projects, including wnpp-projects.
You don't have to run this command when sending us patches for new projects. Anyone merging such a patch will rebuild the dependencies before comitting anyway.

The "dependencies" file is broken!

This file is an educated guess and sometimes not perfect. We prefer to err on the side of too many dependencies rather than too few.

However, the following situations are not necessarily bugs:

- circular dependencies: (cffi depends on trivial-features, which depends on cffi itself. That is not a problem.
- non-transitive dependencies: (if A depends on B, which depends on C, it does not follow that A depends on C.)

Make sure to distinguish between project dependencies and system dependencies.

The following process computes the "project dependencies" of a project FOO:

- look for all files named clbuild/source/FOO/*.asd and load them. (Except for some special projects that force us to scan recursively for asd files.)
- for each system BAR that we found a file FOO/bar.asd for, compute the systems that BAR depends on. (ASDF provides no function that I am aware of to find those dependencies, so we scan the in-order-to slot to make an educated guess what those dependencies are.) As an extra exception, we sometimes add dependencies at this point, for example to force mcclim to always depend on cffi, no matter whether the caller currently has gtkairo enabled or not.
- for each such system that is a dependency, map the system back to its asd file, and if that asd file is found somewhere in clbuild/source/BAZ/, consider it to be part of project BAZ
- The projects found in the last step are the dependencies of FOO.

This process implies that dependencies between projects are not transitive.

For example, the project `babel' depends on the project `rt' because the system BABEL-TEST depends on the system RT.

But the system HUNCHENTOOT depends only on BABEL, not BABEL-TEST, so the project `hunchentoot' does not depend on the project `rt'.

Think about "clbuild update hunchentoot" as "install everything I need to use/hack all parts of hunchentoot", which doesn't include BABEL-TEST or RT. "clbuild update babel" means "install everything I need to use/hack babel", which includes BABEL-TEST and hence RT.

Why is clbuild written in shell? Lisp is so much better.

There are many answers to this question. Here are just a few of them:

- Luke originally wrote clbuild as a shell script.
- clbuild is not a mere shell script. At least `recompile' and `record-dependencies' are now written in Lisp.
- We basically just call out to darcs, git, cvs, and svn anyway.
- Thanks to the shell script, we can install lisp projects before having built a clean SBCL, and without having to worry whether the Lisp in $PATH is good enough.
- Building SBCL from a shell script seems like a natural thing to do.
- Other projects have tried to explore different parts of the design space for lisp software installers. There is cl-librarian written in Lisp, clget written in Perl, and so on.

Does it work on cygwin?

It used to work on cygwin. Beware that cygwin support is tested only occasionally and might not be complete or functional anymore. (In any case, you need to recompile SBCL with an ASDF patch for shortcut support first.) To start clbuild's slime, make sure to use the Windows version of Emacs, not cygwin's Emacs.

### Download and Usage

clbuild is maintained in darcs.

    $ darcs get http://common-lisp.net/project/clbuild/clbuild

You might want to make the shell script executable:

    $ cd clbuild
    clbuild$ chmod +x clbuild

clbuild depends on various helper applications. Try `check` to make sure they are installed:

    clbuild$ ./clbuild check
    Checking for helper applications...
    found /usr/bin/X11/cvs
    found /usr/bin/X11/svn
    found /usr/bin/X11/darcs
    found /usr/bin/X11/wget
    found /bin/tar
    found /bin/mktemp
    Success: All helper applications found.

    Checking Lisp startup...
    Success: Lisp starts up using "sbcl"

    Looking for installable systems...
    0 systems definition files registered

Use `list` to search for projects:

    clbuild$ ./clbuild list web
    i cl-webdav              WebDAV server
    i closure                CLIM-based web browser
    i hunchentoot            web server formerly known as TBNL

Run `update` to download the userland:

    clbuild$ ./clbuild update --main-projects
    cvs co flexichain...
    cvs co mcclim...
    ... lots of output not shown ...
    update complete
    46 systems definition files registered

You do not have to download all default projects (and in particular, wnpp-projects are not part of the default download anyway). Instead, you can specify a particular project, and download it with its dependencies only:

    clbuild$ ./clbuild update climacs
    clbuild update climacs
    climacs depends on: climacs clx flexichain mcclim slime spatial-trees
    include dependencies in update? (Y/n)y
    cvs co climacs...
    cvs co clx...
    ... lots of output not shown ...

To reduce start up times, you can dump a core file including all main projects (or the projects specified on the command line, just like with update):

    clbuild$ ./clbuild climacs   # slow, from fasls
    clbuild$ ./clbuild dumpcore climacs
    ...
    clbuild$ ./clbuild climacs   # fast, using monster.core

### Building SBCL

You can also download and recompile SBCL using `buildsbcl` or `buildworld`:

clbuild$ ./clbuild buildsbcl

### Command line arguments

For other options, see `help`:

    clbuild$ ./clbuild help
    Usage:
      check         check availability of all necessary helper applications

      list [PATTERN]  list all projects, or projects matching PATTERN

      update [--dependencies|--no-dependencies] PROJECT_NAME
                    download/update this project
      update [--resume]
                    download/update main projects.  With --resume, consider
                    only projects that a previous update run failed to fetch.
      skip PROJECT_NAME
                    mark this project as done for the purposes of update --resume

      recompile [PROJECT_NAME] recompile main applications (or just PROJECT_NAME)
      dumpcore [PROJECT_NAME]  recompile and dump a core file for faster startup
      build [PROJECT_NAME]     update && dumpcore

      diff          show local changes (for all version-controlled projects)
      check-urls    compared installed repository urls agains current locations
      clean-links   remove broken symlinks in systems/
      update-missing  download only projects not present yet
      register-asd PROJECT  add .asd file symlinks for PROJECT

      buildsbcl [XC_HOST]  download/update and compile SBCL
      world         buildsbcl && build

      clean         delete all compiled object files
      mrproper      delete all downloaded source and fasl files

      slime         run the Superior Lisp Interaction Mode in a fresh Emacs
      lisp          run Lisp in the terminal

      record-dependencies   rebuild dependency information file

    Starting applications:

      run APPLICATION   run this application
      run --help        show help for applications

    If you do 'world' or 'buildsbcl' then SBCL will be installed in
    target/ and used for future commands. If you don't run these commands
    (or you remove target/) then clbuild uses the 'sbcl' in your PATH.

    For configuration options (including for non-SBCL lisps), see clbuild.conf.

### Application startup

    clbuild$ ./clbuild run --help
    clbuild run APPLICATION [ARGS...]

      run listener      run the McCLIM listener
      run gsharp        run the Gsharp score editor
      run climacs       run the Climacs text editor
      run closure [HOME_PAGE_URL]   run the CLOSURE web browser
                        (required Debian packages: gif2png,libjpeg-progs)
      run beirc         run the Beirc IRC client
      run climplayer    run the CLIMPlayer music player
                        (required Debian packages: mplayer, fileschanged, fam)
      run demodemo      run some random CLIM examples
      run clim-alerts   run CLIM alerts
      run eclipse [DPY] run the eclipse window manager

      run hunchentoot   run the Hunchentoot web server test
      run webdav DIR    run the CL-WEBDAV server, serving directory DIR
                        (required Debian packages: libssl-dev)

      run parse-xml FILENAME
                        check XML for well-formedness
      run validate-xml FILENAME
                        check XML for validity
      run validate-relax-ng [--compact yes] XML-FILENAME SCHEMA-FILENAME
                        check XML for validity against a Relax NG Schema
      run html-to-xhtml HTML-FILENAME OUTPUT-FILENAME
      run xhtml-to-html XML-FILENAME OUTPUT-FILENAME
                    convert between HTML 4 and XHTMl

      run vecto-demo    generate a test image using vecto