[LispM-Hackers] Outageness

[John Morrison]

Hi;

It seems I really should've documented this better.

On Saturday 23 February 2002 07:24 am, Robert Swindells wrote:
> James A. Crippen wrote:
> >Robert Swindells <rjs@fdy2.demon.co.uk> writes:
> >> James A. Crippen wrote:
> >> >Nyef <nyef@softhome.net> writes:
> >> >> While unlambda was gone I noticed that E3 is allocating _four times_
> >> >> as much memory as it needs during startup (compare the size of an
> >> >> instance of e3MemMap to the size of the load band), which runs my

Paul and Steve, please correct any misstatements of fact I make...

Just the facts as I know them:

(1) The load band (pretty much) contains only the memory pages that
were allocated when the "real" Explorer saved the band.

(2) The pages (except for the first low-memory ones) are NOT in any
useful order.  

(3) There are *only* 2^25 words (approx = 128MB) in the Explorer
address space.

(4) Most modern operating systems do NOT allocate physical pages
behind such large memory objects until they are "touched" in some
way. 

Thus:

(1) e3 allocates the memory for this address space out of the host OS
heap (the bare iron version will obviously not do this that way).

(2) e3 reorders the pages as it loads them into this address space
where each page "belongs."

(3) e3 no longer touches every word in the address space (it used to,
and that's when I said "whoops").

You should NOT be seeing working sets any bigger than the load band
size.

> >> >> system close to out of swap space (possible fixes include: doing a
> >> >> check against the different memory regions, emulating part of the
> >> >> memory management hardware (page hash table and such), (ab)using the

If we used mmap, we'd have to implement the VM hardware and
performance would not be as good.  (Mind you, I am well aware that I
am probably going to have to do something funky with the bare iron x86
port involving the MMU in some way.)

> >> >> UN*X mmap() syscall, and simply declaring it a feature).
> >> >
> >> >mmap() is Not Allowed because it's not portable to, say, PalmOS.  Or
> >> >(possibly) to Windows.  It's too un*x specific.
> >>
> >> There is a Win32 equivalent of mmap that works fine. I can provide an
> >> example if it allows us to use mmap() under UNIX.
> >>
> >> Data is in memory anyway on PalmOS, but you would run into the 32k limit
> >> on an individual record long before you hit any other limits.
> >
> >I knew that, honest. :-)  I was just using that as an example.
> >
> >JM has some better argued reasons for avoiding mmap().  We had this
> >argument two years ago IIRC.

I hope this explains the current scheme, even if it does not settle
the matter in everybody's minds.

-jm

-- 
==== John Morrison
==== MAK Technologies Inc.
==== 185 Alewife Brook Parkway, Cambridge, MA 02138
==== http://www.mak.com/
==== vox:617-876-8085 x115
==== fax:617-876-9208
==== jm@mak.com