[LispM-Hackers] Outageness

[Nyef]

On 22 Feb 2002, James A. Crippen wrote:

> Nyef <nyef@softhome.net> writes:
>
> > While unlambda was gone I noticed that E3 is allocating _four times_ as
> > much memory as it needs during startup (compare the size of an instance of
> > e3MemMap to the size of the load band), which runs my system close to out
> > of swap space (possible fixes include: doing a check against the different
> > memory regions, emulating part of the memory management hardware (page
> > hash table and such), (ab)using the UN*X mmap() syscall, and simply
> > declaring it a feature).
>
> mmap() is Not Allowed because it's not portable to, say, PalmOS.  Or
> (possibly) to Windows.  It's too un*x specific.

I didn't think you'd go for it (I wouldn't), but I had to at least mention
it. (Would it work on MacOS? Would it work on the Amiga? Probably not...)

> As for emulating the PHT, I think that would be an ugly mess to deal
> with.  Certainly it could be done, but I think that it would hurt
> performance quite a bit, as well as being unneccesary (since the host
> operating system probably knows a lot more about efficient swapping
> than we do).

Of course it would hurt performance... on boxes with 256 megs of memory or
more. Not everyone has those. _Especially_ not running PalmOS as well.

This is something that everyone who emulates a CPU with an MMU in a style
that precludes using a hardware assist (i.e. mmap()) has to face: It's
going to be slower than emulating it without the MMU. It gets to the point
where the emulators have N-way set-associative caches to speed up the MMU
translation (even when the MMU is _off_), just like the real thing,
because it's actually the cheapest alternative.

There's another catch: The GC implementation described in SSDN2 relies on
some features of the system MMU to implement the read barrier. Of course,
if you use a different type of GC, you can probably work around that...

> If you say that it's four times the necessary amount, could it be
> because someone is allocating with bytes->words in mind when the
> actual allocation unit is already words?  Haven't looked but that
> might be a stupid mistake that could cause this.

No, the address space appears to be word indexed, and is allocated on
those terms. The problem is that all 128MB of the address space is being
allocated, which is severe overkill, since we only need about 32MB of it
to start.

> BTW, it's basically impossible to run E3 with less than 32MB of
> memory.  This is the size of the load band and we expect to at least
> have that much.  In the future once we can look at the running system
> perhaps we can do some magic with a sparse array, or something.

Right now it's basically impossible to run E3 with 192MB of memory (RAM +
Swap). I have a little more than that, and it still chokes (funny messages
from the kernel VM system and everything).

> > There's a pile of textified chapters of ssdn2 that aren't on unlambda.com
> > up at http://www.dridus.com/~nyef/lispm/ssdn2/ and now that there appears
> > to be a point again I may find time to do a few more soon...
>
> I'm still caught up with taking care of networking things, so I'll
> get to that soon, though not instantly.

Okay, just making sure that you know that they're there.

> 'james

---------------------------
All programming can be viewed as an exercise.
---------------------------
Alastair Bridgewater
e-mail: nyef@softhome.net