Prev: How to split huge file located on nfs using some system call.
Next: mblk
From: David Schwartz on 10 Sep 2009 08:24
On Sep 9, 8:47 am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:

> But all of this is actually completely besides the point, because the
> library doesn't use MAP_FIXED. According to the comments in
>
>         valgrind-3.4.1/coregrind/m_aspacemgr/aspacemgr-linux.c
>
> this is supposed to work such that either the 'hinted' mapping is
> placed at the intended address (by the kernel) or that valgrind picks
> a suitable replacement address. The file linked-to by the OP doesn't
> contain MAP_FIXED, either.

I think you're drawing a distinction between the library and the
program that I'm not. In any event, I'm a bit confused now. The OP
specifically stated that an mmap call with MAP_FIXED was failing, and
that it was used in a case where it is not guaranteed to succeed. If
the library doesn't call mmap with MAP_FIXED and the program doesn't,
why is mmap getting called with MAP_FIXED? If valgrind is doing it, it
would be interesting to know the circumstances -- you may be right, it
maybe be a bug in valgrind.

DS
From: Moi on 10 Sep 2009 14:02
On Thu, 10 Sep 2009 05:24:47 -0700, David Schwartz wrote:

> On Sep 9, 8:47 am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:
>
>> But all of this is actually completely besides the point, because the
>> library doesn't use MAP_FIXED. According to the comments in
>>
>>         valgrind-3.4.1/coregrind/m_aspacemgr/aspacemgr-linux.c
>>
>> this is supposed to work such that either the 'hinted' mapping is
>> placed at the intended address (by the kernel) or that valgrind picks a
>> suitable replacement address. The file linked-to by the OP doesn't
>> contain MAP_FIXED, either.
>
> I think you're drawing a distinction between the library and the program
> that I'm not. In any event, I'm a bit confused now. The OP specifically
> stated that an mmap call with MAP_FIXED was failing, and that it was
> used in a case where it is not guaranteed to succeed. If the library
> doesn't call mmap with MAP_FIXED and the program doesn't, why is mmap
> getting called with MAP_FIXED? If valgrind is doing it, it would be
> interesting to know the circumstances -- you may be right, it maybe be a
> bug in valgrind.

relevant part of post++

.....

file_header hdr;
hdr.base_address = base;
hdr.file_size = 0;
read(fd, &hdr, sizeof hdr);
base = (char*)hdr.base_address;
allocated_size = size = ALIGN(hdr.file_size, page_size);
mapped_size = size > max_file_size ? size : max_file_size;
void* p = mmap(base, mapped_size, PROT_READ|PROT_WRITE,
MAP_VARIABLE|MAP_SHARED|MAP_FILE, fd, 0);
if (p == (char*)MAP_FAILED) {
error_code = errno;
TRACE_MSG(("post_file::open: mmap failed: base=%p, size=%ld: %
s\n",
base, mapped_size, strerror(error_code)));
::close(fd);
return false;
}
base = (char*)p;
n_locked_pages = 0;

....
, so it reads the intended map addres and size from the file before
mapping it to that "fixed" address.
Both of these _could_ of course contain garbage, causing EINVAL.

AvK

From: Rainer Weikusat on 10 Sep 2009 14:55
Moi <root(a)invalid.address.org> writes:
> On Thu, 10 Sep 2009 05:24:47 -0700, David Schwartz wrote:
>> On Sep 9, 8:47�am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:
>>
>>> But all of this is actually completely besides the point, because the
>>> library doesn't use MAP_FIXED. According to the comments in
>>>
>>> � � � � valgrind-3.4.1/coregrind/m_aspacemgr/aspacemgr-linux.c
>>>
>>> this is supposed to work such that either the 'hinted' mapping is
>>> placed at the intended address (by the kernel) or that valgrind picks a
>>> suitable replacement address. The file linked-to by the OP doesn't
>>> contain MAP_FIXED, either.
>>
>> I think you're drawing a distinction between the library and the program
>> that I'm not. In any event, I'm a bit confused now. The OP specifically
>> stated that an mmap call with MAP_FIXED was failing, and that it was
>> used in a case where it is not guaranteed to succeed. If the library
>> doesn't call mmap with MAP_FIXED and the program doesn't, why is mmap
>> getting called with MAP_FIXED? If valgrind is doing it, it would be
>> interesting to know the circumstances -- you may be right, it maybe be a
>> bug in valgrind.
>
> relevant part of post++
>
> ....
>
> file_header hdr;
> hdr.base_address = base;
> hdr.file_size = 0;
> read(fd, &hdr, sizeof hdr);
> base = (char*)hdr.base_address;
> allocated_size = size = ALIGN(hdr.file_size, page_size);
> mapped_size = size > max_file_size ? size : max_file_size;
> void* p = mmap(base, mapped_size, PROT_READ|PROT_WRITE,
> MAP_VARIABLE|MAP_SHARED|MAP_FILE, fd, 0);
> if (p == (char*)MAP_FAILED) {
> error_code = errno;
> TRACE_MSG(("post_file::open: mmap failed: base=%p, size=%ld: %
> s\n",
> base, mapped_size, strerror(error_code)));
> ::close(fd);
> return false;
> }
> base = (char*)p;
> n_locked_pages = 0;
>
> ...
> , so it reads the intended map addres and size from the file before
> mapping it to that "fixed" address.

This isn't a mapping to a fixed address, for the simple reason that
MAP_FIXED is not among the MAP_-flags to the mmap call (MAP_VARIABLE
is #defined to 0). It is a request to mmap the file at the address
given as base if possible and to mmap it elsewhere if not. Valgrind
intercepts this call and should perform exactly this task: If mapping
at this address isn't possible because it is occupied by a
valgrind-owned mapping, treat this as so-called 'floating mmap', ie
map in some possible location.

The way valgrind seems to treats this is that it always tries to do a
fixed mapping (code is in coregrind/m_syswrap/syswrap-generic and
coreground/m_aspacemgr/aspacemgr-linux) it just selects and address it
'knows' to be good. Quote from am_get_advisory (comments added)

switch (req->rkind) {
case MFixed: /* MAP_FIXED */
if (fixedIdx >= 0) {
*ok = True;
return req->start;
} else {
*ok = False;
return 0;
}
break;
case MHint: /* address w/o MAP_FIXED */
if (fixedIdx >= 0) {
*ok = True;
return req->start;
}
if (floatIdx >= 0) {
*ok = True;
return nsegments[floatIdx].start;
}
*ok = False;
return 0;
case MAny: /* no address in original call */
if (floatIdx >= 0) {
*ok = True;
return nsegments[floatIdx].start;
}
*ok = False;
return 0;
default:
break;
}
From: Moi on 10 Sep 2009 15:13
On Thu, 10 Sep 2009 20:55:07 +0200, Rainer Weikusat wrote:

> Moi <root(a)invalid.address.org> writes:
>> On Thu, 10 Sep 2009 05:24:47 -0700, David Schwartz wrote:
>>> On Sep 9, 8:47 am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:
>>>
>>>> But all of this is actually completely besides the point, because the
>>>> library doesn't use MAP_FIXED. According to the comments in
>>>>
>>>>         valgrind-3.4.1/coregrind/m_aspacemgr/aspacemgr-linux.c
>>>>
>>>> this is supposed to work such that either the 'hinted' mapping is
>>>> placed at the intended address (by the kernel) or that valgrind picks
>>>> a suitable replacement address. The file linked-to by the OP doesn't
>>>> contain MAP_FIXED, either.
>>>
>>> I think you're drawing a distinction between the library and the
>>> program that I'm not. In any event, I'm a bit confused now. The OP
>>> specifically stated that an mmap call with MAP_FIXED was failing, and
>>> that it was used in a case where it is not guaranteed to succeed. If
>>> the library doesn't call mmap with MAP_FIXED and the program doesn't,
>>> why is mmap getting called with MAP_FIXED? If valgrind is doing it, it
>>> would be interesting to know the circumstances -- you may be right, it
>>> maybe be a bug in valgrind.
>>
>> relevant part of post++
void* p = mmap(base,
>> mapped_size, PROT_READ|PROT_WRITE,
>> MAP_VARIABLE|MAP_SHARED|MAP_FILE, fd, 0);
>> if (p == (char*)MAP_FAILED) {

>
> This isn't a mapping to a fixed address, for the simple reason that
> MAP_FIXED is not among the MAP_-flags to the mmap call (MAP_VARIABLE is
> #defined to 0). It is a request to mmap the file at the address given as
> base if possible and to mmap it elsewhere if not. Valgrind intercepts
> this call and should perform exactly this task: If mapping at this
> address isn't possible because it is occupied by a valgrind-owned
> mapping, treat this as so-called 'floating mmap', ie map in some
> possible location.

My bad.
I had not even checked the flags, seeing that arg1 was non null.
I stand corrected.

AvK
From: David Schwartz on 10 Sep 2009 21:23
On Sep 10, 11:55 am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:

> The way valgrind seems to treats this is that it always tries to do a
> fixed mapping (code is in coregrind/m_syswrap/syswrap-generic and
> coreground/m_aspacemgr/aspacemgr-linux) it just selects and address it
> 'knows' to be good. Quote from am_get_advisory (comments added)

Nice detective work. So we might have a workaround for the OP -- if
the mmap fails, try it again without specifying an address.

DS
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: How to split huge file located on nfs using some system call.
Next: mblk