Infrastructure update (was Re: [LMH] Ooh, look! List traffic!)

[Nyef]

On Sun, Mar 07, 2004 at 07:15:39PM -0900, James A. Crippen wrote:
> 
> I really think given the current `debates' going on in the wider world
> about software that we should stick to the original plan of having
> something working before we start pestering TI for rights. I'd really
> rather not have some lawyer after me with the DMCA in his hand right
> now, and I think my name, Alastair's, and John's are the three most
> well known out there. I suppose if there's still nothing really
> working (enough to give a working Lisp interpreter) in a couple more
> years people could start doing whatever they want. But I suspect that
> both JM and Nyef would like to avoid lawyers as much as I would right
> now.

Right now our biggest safeguards is not having working emulators and
not having the ROM files easily accessible. Even if they know we're
working on it, it's not such a big deal when only a select few people
(most of whom would be in the market for the real thing -anyway-) can
actually do anything with it.

As it stands, I suspect that when we get everything close to working
we should have the emulators and ROMs hosted on different sites by
different people. And the ROM sites should be hard to get to. Which,
admittedly, is just about how things are now. You basically have to
ask someone who already has the ROMs and disk images to share.

At that point we might consider writing a few academic-looking papers
on the design of the emulators and the process of their creation, and
try and keep everything as innocent-looking as possible. Or not.

> > You mean there is no documentation on low level programming?! Oh boy... This
> > may just be a big hard and scary thing to do and actually doing it may prove
> > us right ;-) Have the results of this reverse-engineering been published? I
> > can't find much about it.
> 
> Alastair has done an incredible amount of reverse engineering
> already. I have complete confidence in his skills, even if he
> doesn't. ;-)

My skills I have confidence in. My patience and continued interest,
I don't. James, you aren't the only one who has trouble finishing
projects. ^_-

And, actually, -some- of the low-level programming information has been
turning up. Usually shortly after I figure it out for myself. Amazing
synchrony there. Most of what we don't have documentation for now is the
internals of the CPU, particularly the ALU setup, and the memory boards.
Scans for the SIB, NuPI, CPU, and ethernet boards are available, as are
scans of parts of the maintenance manuals.

>              I never have the time to help, which is horribly
> unfortunate, because I'd absolutely love to do so. But if anyone else
> is really genuinely interested I recommend them reading through all of
> the threads in which he's posted details, and then reading what he's
> got done so far very carefully. Then ask him for recommendations on
> where to start.

For exploiter: Read all of SSDN2, ucode/def-elroy.lisp, ucode/defop.lisp
ucode/lroy-qcom.lisp, ucode/lroy-qdev.lisp, ucode/lroy-templates.lisp,
the list archives (particularly the Bug in SSDN2 posts), and then run
exploiter and start reading through the output trying to line it up to
the source tarball (remembering that the load band you use may be out of
date with respect to the sources we have).

For nevermore: Do the exploiter list (except maybe the bit about running
exploiter) and read E1-proc-gen.pdf (except for the stuff about
macroinstructions), ucode/ravfmt.lisp, ucode/ravsym.lisp, and anything
in the list archives by Steve Krueger. Also hit up the bitsavers.org
document archive and grab all the TI Explorer docs. Read anything that
sounds like it is about hardware rather than software. Run nevermore
for a while by single-stepping through instructions. Figure out how to
make the nevermore disassembler work against prom-memory instead of
reading from the prom_combined file (that none of you have). Disassemble
the boot prom, figure out what it does (having an emulator here helps a
-lot-, that's why I wrote nevermore in the first place).

Actually, fixing up the disassembler would be a good first project for
someone to get involved in nevermore. Making it work on a .mcr file
would just be a bonus (now that we know the file format). Wait until I
release a new version before starting, though, since I sortof changed
some of the data formats...

The game plan, if anyone wants to fix the disassembler, is to wait until
the new version of nevermore, fix the disassembler as it is now, then I
will release my current disassembly and symbol table files, then whoever
is working on the disassembler can see about adding .mcr file support for
Explorer I microloads (since they will then know what the file format is).

> The real trick is not to see the huge amorphous mass of undefined
> hardware and microcode, but to look at the system one instruction at a
> time. Nyef's had really great success with this approach.

That's because this approach is the one that works for me. As an example,
instead of looking at the second instruction in the first function in
the lisp load band and saying "Oh no, CALL-1-DEST-INDS is a function
call instruction, that means we have to implement function calling next",
what I do is look at it and say "Hrm... CALL-1-DEST-INDS is a function
call instruction, what is it calling in this case? Oh, a DTP-FUNCTION?
And what state do I have to set up to start in on the next function? Oh,
I need to set up an args pointer, a locals pointer, reserve space for the
locals, and set up the new function pointer. Oh, and make sure that the
emulator dies with an error message if we try to call anything other than
a DTP-FUNCTION. And don't bother saving the information for the return
instructions, I'll deal with that when I get to them. I wonder what neat
things I'll learn trying to step through this next function?" I stuck all
of this in under a check specifically for the CALL-1-DEST-INDS instruction
used in that first function. Any other call instruction would stop the
emulator with an undefined opcode message.

I hack through things this way, just handling the bare minimum of any
situation that I don't understand, until I have enough information to
go back and do it right. It's not exactly an approach to writing
software, it's more an approach to learning about something that
creates software as part of the learning process. The upside is that
it leaves a record of the understanding gained. The downside is that
that record is relatively difficult to interpret.

About how long after we get this working do you think it'll be before
someone tries to cross-build SBCL from it? ^_^

--Alastair