From: Ilya Zakharevich on
On 2010-07-29, Ben Morrow <ben(a)morrow.me.uk> wrote:
> ...but then your pointers are more than (32|16) bits wide. A DOS/Win16
> __far pointer is a perfectly respectable 32bit pointer.

This depends on your GDT/LDT. E.g., on OS/2 LDT is organized so that
8K of 64KB segments cover the first 512MB of virtual memory in a 4GB
segment (0x053, IIRC). So the same 32-bit-wide bitmaps can be used as
far 16-bit or near 32-bit pointers - which leads to very simple thunks
between 16-bit and 32-bit code.

Yours,
Ilya
From: Ilya Zakharevich on
On 2010-07-31, Peter J. Holzer <hjp-usenet2(a)hjp.at> wrote:
>> E.g., Generally speaking, on "well-designed" 32-bit architecture, one
>> would be able to address 8GB of memory (4GB of data, and 4GB of
>> code).

> And we could split off the stack into a different segment, too and then
> address 12 GB of memory.

Not with C. The same subroutine should accept stack data pointers and
heap data pointers.

> the code also much, much smaller than 4GB

This is not what I experience with my system (which has less than 4GB
memory, though). The monsters like Mozilla take more text memory that
data memory (unless one loads a LOT of HTML into the browser). (This
might be related to 64KB granularity of system allocator in OS/2 [in
units of virtual memory, not real memory] - e.g, you order 1000 small
chunks of shared memory, and you use 64MB of virtual space.) And many
things related to DLLs are loaded into a code-like memory ("shared
address space region").

So maybe I'm wrong: the usage of "shared address space region" may be
large, but not everything which is put there is code. But to be
absolutely sure, I must run some tools on a memory state dump...

> I see the smiley but I'd like to clarify for our young readers that
> 32bit Linux uses near pointers. On the 386, a far pointer would be 48
> bits.

.... but only if you round up to a multiple of 8bit; otherwise 46bit.

>> [*] AFAIK, Solaris (tries to?) separate code from data AMAP. On
>> Solaris/i386, are they in different segments?

> I don't think so. Sun likes Java. Java uses JIT compilers. JIT compilers
> and separated address spaces for code and data don't mesh well.

Do not see how this would be related. AFAI suspect (basing on the
sparse info I have seen), the only way to load code on solaris is to
write an executable module on disk, and dlopen() it.

Yours,
Ilya
From: Ben Morrow on

Quoth Ilya Zakharevich <nospam-abuse(a)ilyaz.org>:
> On 2010-07-31, Peter J. Holzer <hjp-usenet2(a)hjp.at> wrote:
> >[Ilya wrote]
>
> >> [*] AFAIK, Solaris (tries to?) separate code from data AMAP. On
> >> Solaris/i386, are they in different segments?
>
> > I don't think so. Sun likes Java. Java uses JIT compilers. JIT compilers
> > and separated address spaces for code and data don't mesh well.
>
> Do not see how this would be related. AFAI suspect (basing on the
> sparse info I have seen), the only way to load code on solaris is to
> write an executable module on disk, and dlopen() it.

mprotect(2). I suspect you can also allocate writeable-and-executable
memory with an mmap of /dev/zero.

Ben

From: Ilya Zakharevich on
On 2010-08-01, Peter J. Holzer <hjp-usenet2(a)hjp.at> wrote:
> So a compiler could put stuff like
>
> * return addresses
> * non-array automatic variables which don't have their address taken
> * function arguments and return values
> * temporary variables
>
> into the stack segment and automatic variables which do have their
> address taken into the data segment.

> Among other things this means that return addresses are not accessible
> with a pointer and can't be overwritten by a buffer overflow.

Cool argument!

> Mozilla is a monster, but it still uses only about 40 MB of code memory,
> which is about 1% of 4 GB:

I suspect your system has 4K virtual address space granularity. Mine
has 64K. (And it has an order of magnitude less memory.)

What is important is the ratio of data/text. In your case, it is
less than 10. (With more memory, you run more of OTHER monsters. ;-)

> instead of 3.96GB for data. Doesn't seem like much of an improvement.
> (especially if you consider that on most 32-bit OSs the address space is
> limited to 2 or 3 GB anyway - lifting that limit would have a much
> larger effect).

No, the effect would be the opposite: 40M/2G is LARGER than 40M/4G. ;-)

>> AFAI suspect (basing on the sparse info I have seen), the only way to
>> load code on solaris is to write an executable module on disk, and
>> dlopen() it.

> That would absolutely kill the performance of a JIT compiler. If
> Solaris/x86 uses separate code and data segments (which I doubt) then
> there is probably some way (maybe with mmap) to map a region of memory
> into both the data and the code segment. More likely they use a common
> address space and just use mprotect to prevent execution of data which
> isn't meant to be code.

BTW, I looked, and OS/2 uses different segments for code and data.
(Although the paging data for them is almost identical, so they are
more or less aliases of each other.)

Yours,
Ilya
From: Ilya Zakharevich on
On 2010-08-02, Peter J. Holzer <hjp-usenet2(a)hjp.at> wrote:
>> What is important is the ratio of data/text.
>
> No. What is important is the ratio between code and the usable address
> space.

I see (below) that we discuss different scenarios.

>> In your case, it is less than 10. (With more memory, you run more of
>> OTHER monsters. ;-)

> Yes, but those other monsters get their own virtual address space, so
> they don't matter in this discussion.

They do on OS/2: the DLL's-related memory is loaded into shared
address region. (This way one does not need any "extra"
per-process-context patching or redirection of DLL address accesses.)

> No, you misunderstood. If you now have an address space of 2 GB for
> code+data, and you move the code to a different segment, you win 40MB
> for data. But if the OS is changed to give each process a 4 GB address
> space, then you win 2 GB, which is a lot more than 40 MB.

I do not see how one would lift this limit (without a segmented
architecture ;-). I expect that (at least) this would make context
switch majorly costlier...

Thanks,
Ilya