Can extra processing threads help in this case?
in [MFC]
|
From: Joseph M. Newcomer on 13 Apr 2010 12:52 You have failed repeatedly to use any form of "sound reasoning" to support ANY of your claims; the latest was the estimate that it took 1 seek to update a record in a database and the time was therefore 9ms. There is no sound basis for this assertion. So don't tell US about "sound reasoning". Remember, you're the guy who wnated to "turn off virtual memory" in spite of the fact that this statement makes no sense whatsoever, and no form of sound reasoning could ever justify this as an action. When we gave you sound reasoning (e.g., citing exact page number of an Intel manual, and you started frothing at the mouth about how it was requiring that you read 10,000 pages of information. And you would not accept the fact that everyone was telling you were spouting nonsense. I saw no instance of sound reasoning there. Or about virtual memory, or transacted databases, or the use of named pipes, or TCP/IP. No sound reasoning, just basing decisions on hearsay, badly-written 'man' pages, wishful thinking, etc. joe On Mon, 12 Apr 2010 20:13:40 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:dbv6s519osh3qdv2l063sv98n5kbbd6t2m(a)4ax.com... >> See below... >> On Sat, 10 Apr 2010 08:47:00 -0600, Jerry Coffin >> <jerryvcoffin(a)yahoo.com> wrote: >> >> Note that THREE people with real experience in building >> complex systems have now told you >> your design is wrong. You are 1 of 4 votes claiming it is >> good. Has it occurred to you >> that if several independent experts agree the design is >> bad, that just maybe, it is bad? >> joe > >Without sounding reasoning to back it up it might as well be >three liars. I don't know who you are, you could possibly be >an imposter. > >When it comes to discerning truth only sound reasoning forms >a sufficient criterion measure. > >> **** >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 13 Apr 2010 13:07 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:sv79s55iuj6h2uhtrfaur1dlnd907su2no(a)4ax.com... > You have failed repeatedly to use any form of "sound > reasoning" to support ANY of your > claims; the latest was the estimate that it took 1 seek to > update a record in a database > and the time was therefore 9ms. That is not what I said. I said that it would take at least this long if the drive's (average) seek time is 9 ms, and all writes are forced to disk immediately. > There is no sound basis for this assertion. So don't > tell US about "sound reasoning". Remember, you're the guy > who wnated to "turn off virtual > memory" in spite of the fact that this statement makes no > sense whatsoever, and no form of > sound reasoning could ever justify this as an action. > When we gave you sound reasoning > (e.g., citing exact page number of an Intel manual, and > you started frothing at the mouth > about how it was requiring that you read 10,000 pages of > information. And you would not > accept the fact that everyone was telling you were > spouting nonsense. I saw no instance > of sound reasoning there. Or about virtual memory, or > transacted databases, or the use of > named pipes, or TCP/IP. No sound reasoning, just basing > decisions on hearsay, > badly-written 'man' pages, wishful thinking, etc. > joe Much of what I have said is valid reasoning, and it is often based on premises that turn out to be false. Until I know exactly how a thing works I guess how it could work and retain this as my working hypothesis until refuted.
From: Jerry Coffin on 13 Apr 2010 13:22 In article <012bc856-a14b-41ad-99a3-ae05393bbfc0 @z4g2000yqa.googlegroups.com>, sant9442(a)gmail.com says... > > Good example Jerry. I would of done a few things differently. Yup -- a perfect example of the kind of thing I was trying to explain to Peter yesterday. _Every_ design has some shortcomings (or trade- offs, anyway), and if you don't recognize them (or at least have some idea of them), you haven't thought about the design enough to really understand it. > First, The critical section skews the scheduling for the threads, > preferencing the higher priority threads. Removing it provides a > closer measurement of natural quantum based time slicing and you get > different results. Removing it certainly changes the results because you're changing the simulated work load. Basically, what's happening is that GetTickCount is a blocking call. While that's executing, the higher priority thread is blocked, so the lower priority thread gets a chance to run. My intent, however, was to show how the scheduler works with a task that would never block if it was all single threaded. I'll admit that's not a very realistic work load, but it IS the workload I wanted. My work load was intended to show the basic algorithm implemented by the scheduler, and I think it does that quite well. You've done (IMO) a much better job of simulating a real workload though, so your results are probably closer to what you can/could expect when running real programs. In fact your work load shows exactly *why* most schedulers just run the highest priority task that's read to run: almost no real work load is always ready to run. That lets the really simple scheduler design actually allocate CPU time quite a bit more fairly (or at least evenly) than it initially seems like it would. The real question is which work load is a more accurate simulation of Peter's OCR engine. Mine simulates what Peter has *said* -- that it's completely CPU bound. In all honesty, that's probably not correct, so yours is probably a more accurate simulation of how it's likely to work in reality. > Second, I would get the baseline count rates (msecs/count increment) > for equal priority threads, then change the priority of one to > compare rate differences. Good point, and one I intended (but forgot) to mention -- though I didn't post the code both ways, I actually ran what you suggest. If you simply comment out the "SetThreadPriority", the two threads, of course, run at the same priority. It probably wouldn't hurt, however, to run them once with the same priority, and once with the priorities adjusted. -- Later, Jerry.
From: Joseph M. Newcomer on 13 Apr 2010 13:27 See below... On Mon, 12 Apr 2010 20:09:08 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:q6u6s5pl2aueul9l9bor6olbcqvhptcani(a)4ax.com... >> See below.. >> On Mon, 12 Apr 2010 09:31:48 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> > >>>A single queue with two handlers will not by itself >>>provide >>>the prioritization that I need. >> **** >> So add more handlers. You seem to have missed the idea >> that "time slicing" allows you to >> have an arbitrary number of handlers! > >Four different types of jobs, one of these is to have (as >much as possible) absolute priority over all the others, >every job must be processed in strict FIFO order within its >priority. The whole system should be as efficient as >possible. **** How does MQMS make this "as efficient as possible" and still scale to cover massive incoming sequences of requests? **** > >I don't think that SQMS using threads can do that as well as >MQMS using processes because Linux threads are reported to >not work as well as Linux processes. I don't know a good way >to make SQMS work well with multiple processes. The whole >purpose of the MQ is to make communicating with Multiple >processes simple. *** You seem to have some hangup where you think threads have to be in the same process; I have constantly said threads are threads whether they are in the same process or different processes. I just don't feel like making this elaborate artificial distinction every time I raise the topic. I'm sorry you are stuck using a 1960s OS (which is all linux is), but if you want to talk about obsolete technology, this is not the newsgroup to do it. Personally, I prefer to use a much more modern operating system, which we call "Windows", and one of my criteria for shopping for an ISP is to use one that would run a Windows-based server for me. Because I know how to develop software for that platform. **** > >>>I still see four different queues as a better solution for >>>a >>>single core processor. It is both simpler and more >>>efficient. One of the types of jobs will take 210,000 ms >>>and >>>this job absolutely positively can not screw up my maximum >>>100 ms real time threshold for my high priority jobs. >>>Joe's >>>solution is simply broken in this case. >> **** >> Try to find a bright 10-year-old to help you with the >> complex arithmetic involved here. > >This is the ruse that deceitful people use in an attempt to >conceal their deceitfulness. > >> How is it that having your 210,000 ms job lose a timeslice >> to your 10ms job "screws up" >> anything? Duh! But I guess you never heard of "time >> slicing" so you can be forgiven. > >The cache spatial locality of reference will likely be >ruined. **** Really? You have measurements that indicate this is an actual problem? Or did the Tarot cards reveal this? Did you account for the thousands of interrupts-per-second that real operating systems deal with and their impact on cache management? Did you wonder about what happens when the Web server run? You get this incredibly low-level performance buzzword-lock conditions, and think you have a solution based upon nothing at all except what you call "sound reasoning" and the rest of us know is merely augury, since we know what REALLY happens in REAL operating systems; I guess it is because PFOS takes no interrupts when an "important" process is running and therefore there can be no cache pollution from the interrupt handlers, file system operations, etc. Also, no processing of I/O, no handling of the mouse, no handling of network traffic, etc., which is going to be REAL interesting when that 3-minute job runs...no mouse movement, no keyboard response at all, for 3 minutes, then just MAYBE a brief flurry while the next 3-minute job is started, then no response for another 3 minutes... The last I saw a measure on one of my operating systems, it was processing 1400 interrupts/second while doing real work. GET SOME DAMNED DATA AND QUIT FIXATING ON TRIVIA!!!! In the Real World, we call decisions based on actual measurement "sound reasoning". We do not consider the I Ching, Ouija boards, or Tarot cards as "sound reasoning", yet you have provided no evidence you have a clue as to how you are making these various decisions. So we can only surmise, based on their disconnection from reality, that you are trying some kind of psychic methods. **** > >Depending upon the duration and frequency of the time slices >this may not make much of a difference. **** You are truly clueless. joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 13 Apr 2010 13:36
See below... On Mon, 12 Apr 2010 19:53:40 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:t1u6s5908ajiqes3830sqmn8h0f6fnucm9(a)4ax.com... >> See below... >> On Sun, 11 Apr 2010 21:11:40 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>>The only way this site is going to ever get too long of a >>>queue is if too many free jobs are submitted. Do you >>>really >>>think that this site is ever going to be making $10.00 per >>>second? If not then I really don't have to worry about >>>queue >>>length. In any case I will keep track of the average and >>>peak loads. >>> >> **** >> Sorry, if the interarrival time even EQUALS the expected >> processing time, the queue grows >> to infinite size, no matter what. >> joe > >If jobs come in at exactly the same rate that it takes them >to be processed including every little nuance of process >overhead, then the queue grows to infinite length? I don't >see how this can occur. Could you explain it, or at least >point me to a link that explains it? **** It is one of the fundamental results in queueing theory that we proved, mathematically, using "sound reasoning" (because queueing models have closed-form analytic solutions) in the first week of the queueing theory section of the O.R. course. Go get a book on elementary queueing theory. This is considered one of the important results because it is so counterintuitive, which just proves how "intuitive reasoning" is not "sound reasoning". It has been more than 40 years since I proved that theorem, and I no longer recall the details of the proof, but the result was easy to remember. (Actually, the exercise goes something like "compute the maximum queue length if the interarrival time exactly equals the processing time" and it has a singularity that means it goes infinite; furthermore, if you build a discrete-event simulation model, the program can graph the queue size, and before it runs out of memory, it will show the size climbing to infinity. In the simplest form, you just use a counter, and therefore, at least in those days when we submitted the programs on punched cards, the simulation ran out of time and was kicked off the machine with the curve still climbing. We had to do this to show that the singularity was not just a mathematical artifact that didn't actually work out in practice!) joe **** > >> **** >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm |