Prev: Disabling RPC MIDL rundown generation
Next: Cheat anti-debuging application
From: gartaud on 10 Mar 2010 14:25

Hello.

I'm looking for definitive answers to those two questions:

1. When using a persisted memory mapped file, is there a guarantee made by
the OS that the content of the memory mapped file will always be written to
disk even if the application terminates unexpectedly? In other words, if we
write to a view but do not flush it and then the application crashes, are we
guaranteed that what was written to the view will make it to the file on disk?

2. When using a non-persisted memory mapped file backed by the system page
file for inter-process communication, is it possible to avoid that the
content of the memory mapped file be unnecessarily written to the system page
file? In other words, assuming there is enough physical memory on the system
so that paging out is not needed while using the memory mapped file, is it
possible to ask the OS not to write the content of the memory mapped file to
the system pagefile once were are done with it but simply discard it?

Thanks!
From: Hector Santos on 10 Mar 2010 14:52
gartaud wrote:

> Hello.
>
> I'm looking for definitive answers to those two questions:
>
> 1. When using a persisted memory mapped file, is there a guarantee made by
> the OS that the content of the memory mapped file will always be written to
> disk even if the application terminates unexpectedly? In other words, if we
> write to a view but do not flush it and then the application crashes, are we
> guaranteed that what was written to the view will make it to the file on disk?


No, you have to flush it. At least that has been our practice. For a
365x24x7 system, we make exclusive use of mapped files and if thye
customer's machine aborted or some idiot killed the process in some
automated maintenance thing and didn't follow our own automated
shutdown/restart procedure, they ended up calling support about file
integrity issues requiring them to run our repair tool.

In the last few years or so, we decided to minimize the support issues
and now flush every (record) write to avoid such issues. We watched
for complaints of performance. None, and thats probably because
machines and hard drives are much faster than the time our software
was first design many moons ago.

So you should measure it yourself for your needs.

> 2. When using a non-persisted memory mapped file backed by the system page
> file for inter-process communication, is it possible to avoid that the
> content of the memory mapped file be unnecessarily written to the system page
> file? In other words, assuming there is enough physical memory on the system
> so that paging out is not needed while using the memory mapped file, is it
> possible to ask the OS not to write the content of the memory mapped file to
> the system pagefile once were are done with it but simply discard it?

Can't answer that one definitively.

--
HLS
From: gartaud on 10 Mar 2010 15:44
Thanks Hector for the quick feedback on the first question!

-Gabriel

"Hector Santos" wrote:

> gartaud wrote:
>
> > Hello.
> >
> > I'm looking for definitive answers to those two questions:
> >
> > 1. When using a persisted memory mapped file, is there a guarantee made by
> > the OS that the content of the memory mapped file will always be written to
> > disk even if the application terminates unexpectedly? In other words, if we
> > write to a view but do not flush it and then the application crashes, are we
> > guaranteed that what was written to the view will make it to the file on disk?
>
>
> No, you have to flush it. At least that has been our practice. For a
> 365x24x7 system, we make exclusive use of mapped files and if thye
> customer's machine aborted or some idiot killed the process in some
> automated maintenance thing and didn't follow our own automated
> shutdown/restart procedure, they ended up calling support about file
> integrity issues requiring them to run our repair tool.
>
> In the last few years or so, we decided to minimize the support issues
> and now flush every (record) write to avoid such issues. We watched
> for complaints of performance. None, and thats probably because
> machines and hard drives are much faster than the time our software
> was first design many moons ago.
>
> So you should measure it yourself for your needs.
>
> > 2. When using a non-persisted memory mapped file backed by the system page
> > file for inter-process communication, is it possible to avoid that the
> > content of the memory mapped file be unnecessarily written to the system page
> > file? In other words, assuming there is enough physical memory on the system
> > so that paging out is not needed while using the memory mapped file, is it
> > possible to ask the OS not to write the content of the memory mapped file to
> > the system pagefile once were are done with it but simply discard it?
>
> Can't answer that one definitively.
>
> --
> HLS
> .
>
From: m on 10 Mar 2010 21:07
Note that some systems are configured not to use the page file at all.
AFAIK, there is no way to dictate to the OS, besides FlushViewOfFile(), how
it manages the buffering of memory mapped files and the conditions under
which it will flush are OS version and configuration dependent. IMHO
memory mapped files are most useful for read-only data files and for
read-write or write-only, unbuffered overlapped IO is more easily
controlled.

"gartaud" <gartaud(a)discussions.microsoft.com> wrote in message
news:5FD5F31A-0749-411E-9A56-7CA6350D7ED7(a)microsoft.com...
> Thanks Hector for the quick feedback on the first question!
>
> -Gabriel
>
> "Hector Santos" wrote:
>
>> gartaud wrote:
>>
>> > Hello.
>> >
>> > I'm looking for definitive answers to those two questions:
>> >
>> > 1. When using a persisted memory mapped file, is there a guarantee made
>> > by
>> > the OS that the content of the memory mapped file will always be
>> > written to
>> > disk even if the application terminates unexpectedly? In other words,
>> > if we
>> > write to a view but do not flush it and then the application crashes,
>> > are we
>> > guaranteed that what was written to the view will make it to the file
>> > on disk?
>>
>>
>> No, you have to flush it. At least that has been our practice. For a
>> 365x24x7 system, we make exclusive use of mapped files and if thye
>> customer's machine aborted or some idiot killed the process in some
>> automated maintenance thing and didn't follow our own automated
>> shutdown/restart procedure, they ended up calling support about file
>> integrity issues requiring them to run our repair tool.
>>
>> In the last few years or so, we decided to minimize the support issues
>> and now flush every (record) write to avoid such issues. We watched
>> for complaints of performance. None, and thats probably because
>> machines and hard drives are much faster than the time our software
>> was first design many moons ago.
>>
>> So you should measure it yourself for your needs.
>>
>> > 2. When using a non-persisted memory mapped file backed by the system
>> > page
>> > file for inter-process communication, is it possible to avoid that the
>> > content of the memory mapped file be unnecessarily written to the
>> > system page
>> > file? In other words, assuming there is enough physical memory on the
>> > system
>> > so that paging out is not needed while using the memory mapped file, is
>> > it
>> > possible to ask the OS not to write the content of the memory mapped
>> > file to
>> > the system pagefile once were are done with it but simply discard it?
>>
>> Can't answer that one definitively.
>>
>> --
>> HLS
>> .
>>
From: Alexander Grigoriev on 10 Mar 2010 22:09

"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message
news:uXMrJtIwKHA.3952(a)TK2MSFTNGP06.phx.gbl...
> gartaud wrote:
>
> No, you have to flush it. At least that has been our practice. For a
> 365x24x7 system, we make exclusive use of mapped files and if thye
> customer's machine aborted or some idiot killed the process in some
> automated maintenance thing and didn't follow our own automated
> shutdown/restart procedure, they ended up calling support about file
> integrity issues requiring them to run our repair tool.
>
> In the last few years or so, we decided to minimize the support issues and
> now flush every (record) write to avoid such issues. We watched for
> complaints of performance. None, and thats probably because machines and
> hard drives are much faster than the time our software was first design
> many moons ago.
>

I think you're mistaken. Whatever the application puts to the mapped memory
will be flushed to the file, even if the application dies. You problem was
more likely because an application killed in a wrong moment would leave
inconsistent state in the file.


 |  Next  |  Last
Pages: 1 2 3 4 5 6 7
Prev: Disabling RPC MIDL rundown generation
Next: Cheat anti-debuging application