Can extra processing threads help in this case?
in [MFC]
|
From: Peter Olcott on 22 Mar 2010 20:39 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:%23pueS5hyKHA.5288(a)TK2MSFTNGP05.phx.gbl... > Peter Olcott wrote: > >> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >> message news:exGTQtgyKHA.3884(a)TK2MSFTNGP06.phx.gbl... >>> Peter Olcott wrote: > > >>> No. If its zero or not changing and I know your process >>> is not, it means that your process working set is not >>> demanding more than it can handle or OTHER processes >>> have not chewed up memory, limiting your available >>> memory. >> >> OK so zero page faults does not mean that virtual memory >> is not being used? >> (1) YES zero page faults means that virtual memory is not >> active on this process >> (2) Not (YES zero page faults means that virtual memory >> is not active on this process) >> >> Which is it (1) or (2) ??? Any hem hawing will be >> taken as intentional deceit > > > None of the above: > > Your process AT THAT MOMENT does not need PAGE anything > because > it was already in your WORKING SET. > > Look, YOU SIMPLE PROGRAM IS ALWAYS USING VIRTUALIZE > MEMORY! ALWAYS! You just contradicted yourself, but, there was no intentional deceit. (1) It is ALWAYS using virtual memory (2) There was an instance where is was not using virtual memory > > Please answer these questions: > > > Did you try the memory load program? I will go back and carefully study these threads at the point in time where my strategy to always make sure that I have twice as much RAM as the total load on the system requires fails. I expect this to be never. > > > // File: V:\bin\memload.cpp > > #include <stdio.h> > #include <windows.h> > > void main(char argc, char *argv[]) > { > MEMORYSTATUS ms; > ms.dwLength = sizeof(ms); > GlobalMemoryStatus(&ms); > printf("Memory Load: %d%%",ms.dwMemoryLoad); > } > > Did you compile your application to use 64BIT? Did you > convert all your 32 BIT variables to 64BIT? > > Did you read the expert Mark R? > > Pushing the Limits of Windows: Physical Memory > http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx > > Pushing the Limits of Windows: Virtual Memory > http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx > > Pushing the Limits of Windows: Paged and Nonpaged Pool > http://blogs.technet.com/markrussinovich/archive/2009/03/26/3211216.aspx > > -- > HLS
From: Hector Santos on 22 Mar 2010 20:44 Peter Olcott wrote: >> It says nothing about one execution; it says that under >> certain conditions, paging is not >> an issue. It does not say anything about using multiple >> threads on multiple cores within >> a single process. > > This is the earlier issue where you claimed that my thinking > that I needed to have my data resident in RAM was absurd and > based on ignorance. I do need to have my data resident in > RAM and indeed my data is resident in RAM for extended > periods, and there is no ignorance associated with this > thinking. But is ignorance because your PROCESS MEMORY is VIRTUAL MEMORY! Look, for people who have PCs with 1GB or 2GB of RAM, the PROCESS STILL GETS 4GB. Where is it "GHOST RAM" coming from? -- HLS
From: Peter Olcott on 22 Mar 2010 20:45 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:661gq5l01bg7rf539dgotj1b69fiq3re18(a)4ax.com... > See below... > On Mon, 22 Mar 2010 18:46:24 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>message >>news:exGTQtgyKHA.3884(a)TK2MSFTNGP06.phx.gbl... >>> Peter Olcott wrote: >>> >>>> You tell me all about pages faults, yet the process >>>> monitor >>>> reports zero page faults, and you continue to claim >>>> that >>>> its >>>> all about page faults, and virtual memory. >>> >>> >>> Its not a claim - its a fact. >>> >>>> Pages faults indicate victual memory usage right? >>> >>> >>> It shows when your PROCESS is asking too much the can >>> provide to you all in memory - it has to virtualize it. >>> >>>> A lack of page faults indicates a lack of virtual >>>> memory >>>> usage right? >>> >>> >>> No. If its zero or not changing and I know your process >>> is >>> not, it means that your process working set is not >>> demanding more than it can handle or OTHER processes >>> have >>> not chewed up memory, limiting your available memory. >> >>OK so zero page faults does not mean that virtual memory >>is >>not being used? > *** > OF COURSE virtiual memory is being used; there is NO OTHER > KIND OF MEMORY for a process. > What it means is that all of the virtual memory has > remained resident, something that > before you published this results was not something that > was demonstrable. You have > demonstrated that it is not being paged out and the pages > reused, at least under your test > scenario. > *** >>(1) YES zero page faults means that virtual memory is not >>active on this process > **** > But given that there is only virtual memory, you cannot > assert that zero page faults mean > it is not being used, only that the virtual pages have > remained in memory, a useful piece > of knowledge. And if you had a clue about memory-mapped > files, this would tell you that > using a named, shared segment would improver performance > of multiple processes using MMF > to get the data in, and it might also mean you wouldn't > see the several-minute startup > transient. You certainly wouldn't see it on the second or > higher processes. And of course you know that a second thread would work just fine because you know that my process is not memory bandwidth intensive. > > Perhaps you can explain whay you man by "virtual memory is > not active". Alas, for your If there are no pages going in and out of physical memory it is active in the same way that a parked car is active. Maybe you call it active if the engine is still running, even if its not going anywhere. > way of forming the question, virtual memory is ALWAYS > active, and that implies the Right and it is even active if it is disabled, because we all know that it is ALWAYS active. > potential for TLB thrashing. Do you know what a TLB is, > the role it serves, and how it is > managed? Of what TLB thrashing might be? To simplify the > task, I will tell you that TLB > stands for Translation Lookaside Buffer. The rest is up > to you. If you didn't know what > at TLB is when you asked the question, it proves that you > had no clue about why the > question as stated is nonsense. > **** >>(2) Not (YES zero page faults means that virtual memory >>is >>not active on this process) > **** > Virtual memory HAS To be "active" because there isn't any > other kind of memory available > to a process. This is inherent in every operating system. > **** >> >>Which is it (1) or (2) ??? Any hem hawing will be taken >>as intentional deceit > **** > THe question is ill-formed because the terms are being > used incorrectly and in some cases > in ways that suggest that the answer could be "there is no > virtual memory being used" > which is a nonsensical statement. There is ONLY virtual > memory being used, it just > happens that it is not paged out or the pageouts retain > their in-memory images so any page > faults are "soft" (meaning the data does not have to be > read off the disk, because it can > be found already in memory, in an un-reused page). > > You can't hem or haw an ill-formed question, whether the > goal is or is not to be > deceitful. he question makes no sense as stated and the > two alternative answers are both > nonsensical. It is almost but not quite as bad as the > "have you stopped beating your > wife?" style questions, for which any answer that is "yes" > or "no" is damning. So I will > state that the question is ill-formed and the two options > presented as answers are > nonsensical, and that is the complete TRUTH. There is no > reason to be deceitful here, you > asked a question for which the correct answer is: > > "This demonstrates that Windows retains pages in memory > when there is no need to page them > out and reuse the page frames" and that is the ONLY > truthful and correct answer. > > Unfortunately, it is not either of the nonsensical > alteratives you allow. if you don't > undetstand why the question is nonsensical as stated and > both alternative answers are > nonsensical, you are demonstrating that you really, truly > are clueless about how operating > systems work. > **** >> >>> >>> You have NO control over this UNLESS you explicitly told >>> windows to use NO-CACHING, NO BUFFER I/O memory. >>> >>> Why do you refuse to believe this? Every Windows >>> programmer has to know this to some degree. >>> >>>> How does this not prove that my data is in memory and >>>> thus you are wrong when you say that my data is not >>>> resident in memory? >>> >>> >>> It doesn't. >>> >>> Allow a Technical Fellow Engineer at Microsoft explain >>> it >>> all to you: >>> >>> Pushing the Limits of Windows: Physical Memory >>> http://blogs.technet.com/markrussinovich/archive/2008/07/21/3092070.aspx >>> >>> Pushing the Limits of Windows: Virtual Memory >>> http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx >>> >>> Pushing the Limits of Windows: Paged and Nonpaged Pool >>> http://blogs.technet.com/markrussinovich/archive/2009/03/26/3211216.aspx >>> >>> Are you going to argue with Mark too? >>> >>> -- >>> HLS >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 22 Mar 2010 21:02 Peter Olcott wrote: >>> Which is it (1) or (2) ??? Any hem hawing will be >>> taken as intentional deceit >> >> None of the above: >> >> Your process AT THAT MOMENT does not need PAGE >> anything because it was already in your WORKING SET. >> >> Look, YOU SIMPLE PROGRAM IS ALWAYS USING VIRTUALIZE >> MEMORY! ALWAYS! > > You just contradicted yourself, but, there was no > intentional deceit. > (1) It is ALWAYS using virtual memory > (2) There was an instance where is was not using virtual > memory I don't see that as a contradiction at all. Your process gets 4GB Virtual Memory. There was a moment in space and time that your program did not need the OS to page data into your WORKING SET which is the virtual memory in active use by your program. For example, program has allocated 2GB. It may look like your program has access to 2GB and thats the overall idea, but its virtualized. When you reference something in the 2GB that is not in the WORKING SET, then you get a PAGE FAULT which tells the system to go get from the page.sys file the data you need. Look it says it right here in MSDN: http://support.microsoft.com/kb/555223 In modern operating systems, including Windows, application programs and many system processes *ALWAYS* reference memory using virtual memory addresses which are automatically translated to real (RAM) addresses by the hardware. Only core parts of the operating system kernel bypass this address translation and use real memory addresses directly. Virtual Memory is always in use, *EVEN* when the memory required by all running processes does not exceed the amount of RAM installed on the system. REPEAT THE FIRST SENTENCE IN EACH PARAGRAPH 1000 TIMES! How ignorance can you be? When did you start using Windows? or program for it? -- HLS
From: Hector Santos on 22 Mar 2010 21:04
Peter Olcott wrote: > And of course you know that a second thread would work just > fine because you know that my process is not memory > bandwidth intensive. yes, we know that. The simulator, real code with shared memory and multiple threads, proved this and if you took the time to explore it, you will see for yourself. -- HLS |