Prev: Message Queue
Next: Vista and CSIDL_COMMON_APPDATA - readonly and _hidden_ directory
From: David Schwartz on 1 Mar 2010 20:09
On Mar 1, 2:10 pm, David Given <d...(a)cowlark.com> wrote:

> I have an odd requirement when porting a program. It wants to create
> memory mapped view of files aligned to 4kB boundaries (it's using the
> Unix mmap() call).

Where does it get 4kB from? Does it call some function? Or is this
hard-coded?

> Mapping files into the address space is easy enough in Windows, but all
> the documentation says that I can only do so in 64kB pages.

Correct.

> The thing is, this just isn't true: I can examine a running process and
> see lots of mapped areas aligned at 4kB boundaries. Indeed, if I map a
> 64kB block of anonymous memory and write to the first byte of it, I can
> see it turn into a 4kB range and a 60kB range, so the kernel's obviously
> allocating 4kB pages lazily.

These are two completely different kinds of mappings. One is high-
level mappings that specify what happens on a page fault. The other is
low-level that tracks what pages of memory are actually currently
mapped.

> So, what does the kernel know that I don't? Is it actually possible to
> do this? If so, it would save me a huge amount of work writing mmap()
> simulation code using file I/O calls and loading the data into actual RAM..

Unless you have a really unusual situation, you should be able to do
one of two things:

1) Modify the original code to call some function to get the "page
size" and return the allocation granularity under Windows.

2) Write your own 'mmap' 'munmap' functions that fake 4kB granularity
by mapping with 64kB granularity and returning a pointer inside the
mapping if needed. So if a mapping is requested for 12kB into a file,
create a mapping at offset zero and return a pointer 12kB into the
mapping. Keep a table of the pointers you've returned and on an
'munmap' call, remove the actual mapping (if it's no longer needed).

DS
From: Ulrich Eckhardt on 9 Mar 2010 08:12
David Given wrote:
> On 02/03/10 01:09, David Schwartz wrote:
>> 2) Write your own 'mmap' 'munmap' functions that fake 4kB granularity
>> by mapping with 64kB granularity and returning a pointer inside the
>> mapping if needed.
>
> I can't modify the original code --- I just have a binary --- so I'm
> actually doing something similar to (2).
>
> The problem is that in some cases the code isn't giving me the option to
> choose my address --- it'll ask for a 500MB file to be mapped at
> 0x12345000.

The address given to mmap() is but a hint. mmap() is free to ignore it and
any program assuming it is used is broken anyway. Complain to the vendor!

> So I must somehow make the data in the file available at 0x12345000, no
> more, no less. My only option so far is to map 500MB of anonymous memory
> at 0x12340000 and then load the file data into it at the target address.
> I'd really rather not do this, as it's wasteful and slow, and also
> means that I can't implementable shared writeable mappings (although
> luckily I don't think the code wants to do this).

This is getting hackish now, but even if you just reserve RAM, you can still
load pages on demand. For that you can remove read/write access bits from
the pages and hook into page faults for your process. I guess
reimplementing the other code would be a better idea though.

Just curious, what library is using mmap() calls with a fixed page size and
on MS Windows?

Uli

--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

 | 
Pages: 1
Prev: Message Queue
Next: Vista and CSIDL_COMMON_APPDATA - readonly and _hidden_ directory