[LMH]The Cold Loader and a Story

[Steve Krueger]

You may know that a load band is merely a saved virtual memory image.  Explorers need a LOD band and a MCR band to boot.  You can easily make a LOD band by running an appropriate save-band command.

If you think about that for a moment, you may wonder how the Lisp system code is initially placed into a band.  It is clear that once you have a band you can make a new band by loading some more files or making other changes in your working storage and then converting it to a load band by running save-band (or whatever the exact function name is).

But, that initial load band is a problem.  Taking files of Lisp code and turning it into a load band, is a much different problem than the other operations on bands.  MIT solved this problem with the cold loader.

The cold loader takes Lisp code that is cross-compiled for a LispM environment, and reads it into a new virtual memory image.  TI used the cold loader for two purposes: to build load bands from scratch so that they would not contain anything extraneous; and for fundamental changes like the VM2 work that changed the tag encoding, the instruction set and the format of several object types.

To construct the virtual memory image, the cold loader formats objects into the memory image using the object format of the target.  By making changes in the definition and functions of the cold loader, the target band may be very different than the environment in which it was run.

Now if you think about a cold loader for a while, you realize that Lisp presents a formidable challenge in that eval-when-load forms can be arbitrarily complex.  They can call function, macros, create any data type.  The cold loader was able to handle most of the simple cases, but anything very complicated is just pushed onto the cold-load-list variable.  The idea is that these forms will all be eval'ed by (mapcar #'eval cold-load-list) when the new load band is first run.

The environment of a new cold load band is very fragile.  The biggest problem is that only a small, essential part of the lisp environment was loaded with the cold loader.  In particular, the flavor system and the error system weren't loaded. The window systme and file system were simple partial implementations of those functions.  If anything run in this environment tries to call a function that wasn't loaded, it crashes.  Without the error system, the error handler didn't work.

So, the contents of the cold-load-list can prove fatal.  Files in the cold load were carefully written and the order in which they were loaded by the cold loader was carefully crafted.  A fairly innocent change to a cold loaded file, could make its cold-load-list items depend on other files that were not loaded into the cold load, or that were loaded later in the cold load and so had not had cold-load-list initializations performed.  The result was always a crash.

You may actually see a comment in some of the system source files that mention that it is a cold load file.  Now you know what that means.

Now for a story:

When we first were working to bring up the lispm software on Explorer (I), the microcode and lisp sources had been adapted to the TI hardware.  It took weeks before we got through the microcode initialization to the point of the first disk read from the cold-load load band.  And weeks still until it first successfully called the initial-lisp-function.  Every new instruction used seemed to have microcode bugs, so it took days before we executed much code at all.  At that point, we began debugging the execution of the cold load list. Some of the cold-load-list items referred to non-existant functions or to uninitialized modules.  Finally, we got out to the lisp environment.  I have a photo of the first words printed on the Explorer (I) screen.  I'll scan and upload it at some point.

The cold load contained a function that would read in the rest of the system files.  Again the order was quite significant, as was the proper use of eval-when-load forms.  Again it tooks weeks of work.  Getting the cold-load file system to work with the TI NuPI was a challenge as the NuPI wasn't fully functional.  Each distinct macroinstruction seemed to require debugging the microcode.  It probably took a couple of weeks to load the first file.  Each new file took days.  The name of each file would be printed as it loaded.  We got one, then two then three files listed.  Eventually six then ten files were listed.

Then one day, the file names just kept coming.  There were cheers in the system integration lab as it went further and further, then it reached the last line of the screen and said "** More **".  It took a moment for use to realize that we couldn't proceed.  The keyboard wasn't yet complete.  Then we burst out laughing.

After a bit of celebration we devised a work around to this problem.  We built the cold-load band with the keyboard buffer pre-filled with a space.

A few names for posterity: The core microcode integration team for Explorer (I) was Wayne Gibson, Phil Mueller and myslf.  TI's work on the cold loader was by Rick Blair.  Our microcode debugger was called Seymour and was written by Paul Fuqua.  The hardware connection to an Explorer was via a hardware box that tapped into the processor to give good visibility to the internal memories and registers, called a Spyport.  Glenn Manuel built the Spyport.