Making Monolithic Fasls with ASDF3

I have been trying to get familiar with some of the new features in ASDF3. One of these features is the so-called asdf/bundle package, which provides the ability to make self-contained "monlithic" fasl files either for an individual system, or for a system including all its depended-upon systems.

This looks like a clean way to build either a runtime or a development image. By starting up a fresh empty image then loading just the desired monofasls, it is possible to benefit from all the goodness of Quicklisp and ASDF, and yet to prepare an executable image which itself is free of those bootstrap utilities (which themselves can then be included or not, in a deliberate manner).

Using gendl as an example, let's prepare a monolithic fasl which contains all its depended-upon systems plus any of its own components.

Setting the Stage


First we will use Quicklisp to fetch the asdf systems (or Quicklisp "dists") for :gendl and everything it depends on:

(ql:quickload :gendl :verbose t)

This brings down the systems (if they are not already there in your ql:*quicklisp-home*) and ensures that they can be compiled and loaded successfully, at least in your current development environment. 

Before generating the monolithic fasl, out of curiosity, let's first take a look at what the depended-upon systems are:

(pprint (asdf:component-depends-on 'asdf:monolithic-compile-bundle-op :gendl))

This yields:

((ASDF/BUNDLE:FASL-OP #<SYSTEM "cl-who">

#<SYSTEM "base"> 

#<SYSTEM "alexandria">

#<SYSTEM "bordeaux-threads">

#<SYSTEM "cl-fad">

#<SYSTEM "cl-lite">

#<SYSTEM "iterate">

#<SYSTEM "cl-pdf">

#<SYSTEM "cl-typesetting">

#<SYSTEM "geom-base">

#<SYSTEM "puri">

#<SYSTEM "cl-ppcre">

#<SYSTEM "nibbles">

#<SYSTEM "ironclad">

#<SYSTEM "acl-compat">

#<SYSTEM "htmlgen">

#<SYSTEM "aserve">

#<SYSTEM "cl-base64">

#<SYSTEM "trivial-features">

#<SYSTEM "babel">

#<SYSTEM "trivial-backtrace">

#<SYSTEM "cl-html-parse">

#<SYSTEM "gwl">

#<SYSTEM "gwl-graphics">

#<SYSTEM "yadd">

#<SYSTEM "robot">

#<SYSTEM "tree">

#<SYSTEM "tasty">))

Presumably these are in the correct order to satisfy dependencies. Notice that "gendl" itself is not included in this list. That's because "gendl" does not have any of its own local components --- it is just a container system which :depends-on other gdl components.


Generating the actual Monofasl

Now we invoke the operation to make the actual monolithic fasl:

(asdf:operate 'asdf:monolithic-compile-bundle-op :gendl)

Done.


Now let's take a look at where the fasl file itself should have landed:

(asdf:output-files 'asdf:monolithic-compile-bundle-op :gendl)     

Sure enough, this yields:

(#P"/Users/[...]/gendl/gendl--all-systems.dx64fsl") 

As you can see, this is going into the toplevel of the location derived from ASDF's effective output-translations. To see what those output-translations are going to be explicitly, we can always do something like:

(asdf:apply-output-translations

    (asdf:system-source-directory :gendl))

Building an Executable Image

The next step in the build process will be to start a fresh image, load this monofasl, then dump the image.  Note that ASDF3 already does have some ability to dump the currently-running image after loading a monolithic fasl, using e.g.

(asdf:operate 'asdf:program-op :gendl)

While this facility may already be sufficient for many needs, it does not yet give hooks into the multitude if implementation-dependent options for creating executable images on the various different CL platforms. For example Allegro CL has at least three ways to generate executable images, each with dozens of options. So making the actual executable image will still, for the time being, often require calling into the native CL implementation's way of doing it.

The next posting will discuss some of the techniques we are using currently to tackle this step, as well as our general approach for doing simultaneous and automated builds, on many combinations of CL and OS platforms, including: Allegro CL (SMP and non-SMP, modern-mode and ANSI mode, 32-bit and 64-bit), LispWorks, SBCL, and CCL, running on MacOS, Linux, and Windows.