2 questions about memory mapped files
in [Win32 Kernel]
|
From: Hugo gleaves on 18 Mar 2010 10:53 "Pavel Lebedinsky [MSFT]" wrote: > > But in any case, the data is flushed when the file mapping object is > > ultimately closed. > > There are two ways a modified mapped page can be written to disk: > > 1. As a result of an explicit FlushViewOfFile call. > > 2. After a page has been trimmed or unmapped from all working sets, > the mapped page writer thread will (eventually) write it to disk. > The OS makes no guarantees as to how long this will take, but > eventally all such pages will be written to their backing storage, > unless the system crashes or loses power, or the backing storage > becomes unavailable (e.g. network is disconnected). > > Closing the file mapping object has no effect on when the pages will > be written out. > > -- > Pavel Lebedinsky/Windows Fundamentals Test > This posting is provided "AS IS" with no warranties, and confers no rights. > > > . > It is also worth stressing that one must ideally use FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING for the file handle used when creating the mapping. It is my understanding that when one does this, the FlushViewOfFile will only return to the caller AFTER ever single modified page has been written to the physical disk drive. Hugh
From: Alexander Grigoriev on 18 Mar 2010 21:24 FILE_FLAG_WRITE_THROUGH may slow the flush EXTREMELY. "Hugo gleaves(a)hotmail.com>" <hugh<underbar> wrote in message news:7727D7E0-838D-4EE9-BDC5-7DD6045008B5(a)microsoft.com... > > > "Pavel Lebedinsky [MSFT]" wrote: > >> > But in any case, the data is flushed when the file mapping object is >> > ultimately closed. >> >> There are two ways a modified mapped page can be written to disk: >> >> 1. As a result of an explicit FlushViewOfFile call. >> >> 2. After a page has been trimmed or unmapped from all working sets, >> the mapped page writer thread will (eventually) write it to disk. >> The OS makes no guarantees as to how long this will take, but >> eventally all such pages will be written to their backing storage, >> unless the system crashes or loses power, or the backing storage >> becomes unavailable (e.g. network is disconnected). >> >> Closing the file mapping object has no effect on when the pages will >> be written out. >> >> -- >> Pavel Lebedinsky/Windows Fundamentals Test >> This posting is provided "AS IS" with no warranties, and confers no >> rights. >> >> >> . >> > > It is also worth stressing that one must ideally use > FILE_FLAG_WRITE_THROUGH > and FILE_FLAG_NO_BUFFERING for the file handle used when creating the > mapping. It is my understanding that when one does this, the > FlushViewOfFile > will only return to the caller AFTER ever single modified page has been > written to the physical disk drive. > > Hugh >
From: Jonathan de Boyne Pollard on 18 Mar 2010 23:30 > >>> >>> even if an app dies and the OS begins to automatically flush, the >>> logoff will be forced to hang until its done. >>> >> M. Lebedinsky has explained that that is a falsehood. WINLOGON >> doesn't do any magic at logout to flush dirty pages. >> > He didn't say that exactly. > Yes, xe did. Go and read all of what xe wrote. > [...] I simply logged out and logged back in, the file was there [...] > Now try attempting to demonstrate the forcible hang and flush of dirty pages to disc that you claimed would occur, instead of something quite different.
From: Hugo gleaves on 19 Mar 2010 12:14 "Jonathan de Boyne Pollard" wrote: > > > >>> > >>> even if an app dies and the OS begins to automatically flush, the > >>> logoff will be forced to hang until its done. > >>> > >> M. Lebedinsky has explained that that is a falsehood. WINLOGON > >> doesn't do any magic at logout to flush dirty pages. > >> > > He didn't say that exactly. > > > Yes, xe did. Go and read all of what xe wrote. > > > [...] I simply logged out and logged back in, the file was there [...] > > > Now try attempting to demonstrate the forcible hang and flush of dirty > pages to disc that you claimed would occur, instead of something quite > different. > > . > Jonathan the only circumstances I know of, that can lead to missing data with memory mapped files and updates to them are: 1) OS crash 2) Power failure 3) Forced reset of machine In all other cases you will not lose data, even if you logout/shutdown with active processes that have unflushed mapped pages. I have tested this with logout and shutdown, I just told you that and you dismiss the test. You claimed above that Mr Lebedinsky said that this was an untrue statement "even if an app dies and the OS begins to automatically flush, the logoff will be forced to hang until its done." He did not say that, so why don't you go and reread his posts. I'm not looking for a bickering contest Jonathan, the OP asked an important question and it is important that we all try to stick with facts and not let our ego intrude. Hugh
From: Hector Santos on 19 Mar 2010 16:50
Hugo, I think the point of all this is, IMTO, that you have need to be more proactive to minimize the potential abnormal and/or ungraceful shutdown event issues by: - Add more explicit flushing logic. - Implement automated WM_QUERYENDSESSION, WM_ENDSESSION graceful exiting logic where necessary. The OP asked a basic question: "are we guaranteed that what was written to the view will make it to the file on disk?" The only guarantee anyone can have is one of expected design behavior. But there is also a 2nd part of the question which adds a time factor - an expectation it will be flushed at some future point in time. IMTO, this introduces an engineering unknown that some MMF extensive applications can't afford. One should not rely on waiting until the OS and/or network drivers cache management to flush based on some dirty page trigger point or until processes (graceful) exit. While I agree ideally, lost of data is very minimal and today, hardware, OS, networks, hard drivers are better, more reliable and faster, in practice, it mandated a more direct control of the situation to minimize the potential lost of data due to events outside your control. Finally, there was another factor why more explicit flushing helps- the need to better support OFA (Open File Agent) and the introduction of VSS for hot backups. For VSS, unless your process is VSS aware, flushing helps with the integrity of the snapshots. -- HLS Hugo gleaves(a)hotmail.com> <hugh wrote: > > "Jonathan de Boyne Pollard" wrote: > >>>>> even if an app dies and the OS begins to automatically flush, the >>>>> logoff will be forced to hang until its done. >>>>> >>>> M. Lebedinsky has explained that that is a falsehood. WINLOGON >>>> doesn't do any magic at logout to flush dirty pages. >>>> >>> He didn't say that exactly. >>> >> Yes, xe did. Go and read all of what xe wrote. >> >>> [...] I simply logged out and logged back in, the file was there [...] >>> >> Now try attempting to demonstrate the forcible hang and flush of dirty >> pages to disc that you claimed would occur, instead of something quite >> different. >> >> . >> > > Jonathan the only circumstances I know of, that can lead to missing data > with memory mapped files and updates to them are: > > 1) OS crash > 2) Power failure > 3) Forced reset of machine > > In all other cases you will not lose data, even if you logout/shutdown with > active processes that have unflushed mapped pages. > > I have tested this with logout and shutdown, I just told you that and you > dismiss the test. > > You claimed above that Mr Lebedinsky said that this was an untrue statement > "even if an app dies and the OS begins to automatically flush, the logoff > will be forced to hang until its done." > > He did not say that, so why don't you go and reread his posts. > > I'm not looking for a bickering contest Jonathan, the OP asked an important > question and it is important that we all try to stick with facts and not let > our ego intrude. > > Hugh > > > > > > |