[LispM-Hackers] Outageness

[ford@objs.com]

John Morrison wrote:

> Hi;
>
> It seems I really should've documented this better.
>
> On Saturday 23 February 2002 07:24 am, Robert Swindells wrote:
> > James A. Crippen wrote:
> > >Robert Swindells <rjs@fdy2.demon.co.uk> writes:
> > >> James A. Crippen wrote:
> > >> >Nyef <nyef@softhome.net> writes:
> > >> >> While unlambda was gone I noticed that E3 is allocating _four times_
> > >> >> as much memory as it needs during startup (compare the size of an
> > >> >> instance of e3MemMap to the size of the load band), which runs my
>
> Paul and Steve, please correct any misstatements of fact I make...

Paul knows this stuff much better than I, but I can confirm assumptions #1 & #3.
I'm not sure about #2.  There was quite a bit of work on the boot process to
speed things up.  For instance, I don't think most of the pages were copied
from the band into memory until they were touched.

Steve

> Just the facts as I know them:
>
> (1) The load band (pretty much) contains only the memory pages that
> were allocated when the "real" Explorer saved the band.
>
> (2) The pages (except for the first low-memory ones) are NOT in any
> useful order.
>
> (3) There are *only* 2^25 words (approx = 128MB) in the Explorer
> address space.
>
> (4) Most modern operating systems do NOT allocate physical pages
> behind such large memory objects until they are "touched" in some
> way.
>
> Thus:
>
> (1) e3 allocates the memory for this address space out of the host OS
> heap (the bare iron version will obviously not do this that way).
>
> (2) e3 reorders the pages as it loads them into this address space
> where each page "belongs."
>
> (3) e3 no longer touches every word in the address space (it used to,
> and that's when I said "whoops").
>
> You should NOT be seeing working sets any bigger than the load band
> size.
>
> > >> >> system close to out of swap space (possible fixes include: doing a
> > >> >> check against the different memory regions, emulating part of the
> > >> >> memory management hardware (page hash table and such), (ab)using the
>
> If we used mmap, we'd have to implement the VM hardware and
> performance would not be as good.  (Mind you, I am well aware that I
> am probably going to have to do something funky with the bare iron x86
> port involving the MMU in some way.)
>
> > >> >> UN*X mmap() syscall, and simply declaring it a feature).
> > >> >
> > >> >mmap() is Not Allowed because it's not portable to, say, PalmOS.  Or
> > >> >(possibly) to Windows.  It's too un*x specific.
> > >>
> > >> There is a Win32 equivalent of mmap that works fine. I can provide an
> > >> example if it allows us to use mmap() under UNIX.
> > >>
> > >> Data is in memory anyway on PalmOS, but you would run into the 32k limit
> > >> on an individual record long before you hit any other limits.
> > >
> > >I knew that, honest. :-)  I was just using that as an example.
> > >
> > >JM has some better argued reasons for avoiding mmap().  We had this
> > >argument two years ago IIRC.
>
> I hope this explains the current scheme, even if it does not settle
> the matter in everybody's minds.
>
> -jm
>
> --
> ==== John Morrison
> ==== MAK Technologies Inc.
> ==== 185 Alewife Brook Parkway, Cambridge, MA 02138
> ==== http://www.mak.com/
> ==== vox:617-876-8085 x115
> ==== fax:617-876-9208
> ==== jm@mak.com
>
> http://lists.unlambda.com/mailman/listinfo/lispm-hackers

--
Steve Ford
Object Services and Consulting, Inc.
225 State Rd
West Grove, PA 19390
(610)345-0290
(610)444-4445 FAX