[LMH]Data file mirroring

[James A. Crippen]

Okay, here's the start of a plan.

For right now mirrors will just have to be simple manual downloading
of a directory tree (or tarball thereof) on Kappa. The new Kappa will
support rsync which should be implemented to reduce bandwidth usage
and make mirroring automatically schedulable by cron.

We can have as many mirrors as people want to provide. I'll keep track
of them all on a web page somewhere, and direct people to try to use
the one closest to them. I *won't* give access to this data directly
from my site once a sufficient number of mirrors exist. The reason why
should be obvious by now, given my repetitive whining.

A description of the contents of the data archive follows. Criticism
is welcome.

First we have the directory itself. It's divided into a handful of
subdirs. The bands, disks, eproms, and tapes directories should be
autoexplanatory. The docs directory will contain those annoyingly
large files full of PDFs from various sources (patents, scans,
etc). In the interest of relieving pressure on Al Kossow's network I
think we should mirror all the PDFs that he has graciously provided.
He has enough other computer crazies pestering his web server already,
I guess.

  expl-data/
  expl-data/bands/
  expl-data/disks/
  expl-data/docs/
  expl-data/eproms/
  expl-data/source/
  expl-data/tapes/
  expl-data/tools/

First, the bands. In there you'll find the three bands you've come to
know and love. Maybe we'll see more show up here eventually as other
people submit good test candidates from working Exploders. Note that
I'm trying to standardize on a '.LOAD' convention. I'm not sure if
that was standard on Exploders, but we've got two using it so why not
the third?

  expl-data/bands/N928.LOAD.bz2
  expl-data/bands/EXP1.LOAD.bz2
  expl-data/bands/expl-6.1-custom.LOAD.bz2

Next, the disks. Right now we only have a couple of disk images. These
will do for now. Perhaps in the future we'll see more show up here
too. Currently these three are somewhat obscurely named. I'd like to
have the name of the machine that these came from to name them after.
Idea being, we have a standard format of 'machine-scsiXdX.dsk.bz2',
the first X being the SCSI chain and the second being the drive
number.

  expl-data/disks/scsi0d0.dsk.bz2
  expl-data/disks/scsi0d1.dsk.bz2
  expl-data/disks/scsi2d0.dsk.bz2

Now the docs. The docs are split up into a couple of subdirs. I'm not
going to exhaustively list the files here. The directory names should
be obvious. The papers directory has various research papers that seem
important, like MIT AIM-528. In the ssdn2 directory are the text
chapters, the subdir contains the HTML version. The scanned version of
the SSDN2 is in the scans dir. The visidoc directory contains that
pile of manuals from visidoc that I have floating around here
somewhere. I will be making the SSDN2 text and HTML versions available
from my site, an exception to the rest of this.

  expl-data/docs/patents/smbx/
  expl-data/docs/patents/ti/
  expl-data/docs/papers/
  expl-data/docs/scans/
  expl-data/docs/ssdn2/
  expl-data/docs/ssdn2/html
  expl-data/docs/visidoc/

The eproms directory I'll leave alone at the moment, but if we start
to get more I'd like to subdivide it by machine serial number or name
or some similar unique identifier.

The source directory contains the Explorer System Software source
tarball. If we get more dumps from other machines then I'll probably
just keep diffs from this base tarball. Eventually we'll end up with a
canonical set of sources for use on an emulation system, and even more
eventually these will end up in some world-readable CVS repository or
something for all to keep synchronized with. But I get ahead of
myself. (Sigh. I like to dream.)

Tapes. What machine did these tape images come from? I want to name
them. Also, I've fixed the truncated names on my copies, putting the
appropriate .tap on the end.

Tools. Tools? Well, I have at least one that is useful. JWZ's
strip-explorer-fonts.lisp as edited by Carl Shapiro in 2001. That
should be part of the collection since it's vital to working with the
sources tarball when on a non-Exploder.

Okay, that's all I've thought of. I considered keeping released
versions of Exploiter in here, but it didn't seem right. These belong
in their own archive, separate from the data, or at least that's the
way it seems to me. Although probably mirroring them the same way
would be a good idea. I expect that Alastair will keep copies of
ancient versions, but that the publically available archive should be
kept down to only a few release versions. But we'll see how that all
works out once the Exploiter is active in CVS.

I haven't actually done all of this yet. I will later, but I need some
sleep tonight. I'm thinking of taking care of it tomorrow during the
day, about 12 hours from now.

'james

-- 
James A. Crippen <james at unlambda.com> Lambda Unlimited
61.2204N, -149.8964W                     Recursion 'R' Us
Anchorage, Alaska, USA, Earth            Y = \f.(\x.f(xx))(\x.f(xx))