Details about how process scheduler works?
in [Unix Programming]
|
From: Peter Olcott on 16 Apr 2010 15:13 "Stefan Monnier" <monnier(a)iro.umontreal.ca> wrote in message news:jwvvdbrkzrz.fsf-monnier+comp.unix.programmer(a)gnu.org... >> To sum it all up. One set of jobs is to be provided (as >> mush >> as possible) with a 500 ms response time and the >> remaining >> sets of jobs a 24 hour turn-around is good enough. I do >> want >> to provide these jobs with the very fastest response time >> that can be provided without impacting the high priority >> jobs at all. > > I see, 3 classes of jobs, one of which (the "1st class") > should really > suffer as little disruption as possible. Handling the 2 > "best effort" > (or "slow") ones will be the easy part in the sense that > it's what Unix > schedulers do all the time, so you may need to twiddle > with the niceness > to favor the paying slow ones over the free ones but > that's about it. > > OTOH for the 1st class of jobs, you'll have to work > harder. There are > several problems to solve: > - CPU scheduling: you may want to try one of the > soft-realtime > scheduling as suggested by someone else for that. This > will ensure > those jobs always get the CPU in preference to the other > jobs. > But beware: this kind of scheduling has *very* high > priority, i.e. not > just higher than your other jobs but also higher than > most of the OS's > system processes, so you may get into trouble at high > loads. Yes so my original attempt would be to simply lower the priority of the lower priority jobs exactly one unit. > - Scheduling within the web-server: if you have only one > web-server, it > may prove difficult to make sure that the many "slow" > clients don't > slow down the "1st class" clients. I have no experience > there, so I'll > just stop (maybe you can just run 2 web servers). > - Scheduling other resources: if you get many many "slow" > requests, even > if they don't get much CPU time, they may eat up your RAM > and cause > the machine to swap/thrash, so make sure your web-server > is configured > to limit the number of "slow" requests that are being > serviced at any > given time. If necessary I will persist these to disk. If I get too many free job requests I will limit these somehow. > >> My goal of making an online application provide the >> response time of >> an application installed directly on the user's computer >> is already >> close enough to impossible that I know for sure that even >> if my >> system is infallibly perfectly as fast as possible, that >> this will at >> best only be marginally good enough. > > Computer scientists are accustomed to do the impossible on > a daily basis. > > > Stefan
From: Stefan Monnier on 17 Apr 2010 11:01 > Yes so my original attempt would be to simply lower the > priority of the lower priority jobs exactly one unit. AFAIK in Linux and other Unix kernels, all the soft-real time scheduling classes have higher priority than the "normal" priority. I.e. you can only lower the priority of a process with "nice", which does not guarantee that it will only get the CPU if the other processes are done. Stefan
From: Peter Olcott on 17 Apr 2010 11:26
"Stefan Monnier" <monnier(a)iro.umontreal.ca> wrote in message news:jwv633qjffx.fsf-monnier+comp.unix.programmer(a)gnu.org... >> Yes so my original attempt would be to simply lower the >> priority of the lower priority jobs exactly one unit. > > AFAIK in Linux and other Unix kernels, all the soft-real > time scheduling > classes have higher priority than the "normal" priority. > I.e. you can > only lower the priority of a process with "nice", which > does not > guarantee that it will only get the CPU if the other > processes are done. > > > Stefan The idea of raising the priority of the high priority job requires superuser privileges. I was thinking that it might be a good idea (for security reasons) to tend to avoid processes with superuser privileges. Maybe there is an easy completely safe way to do this, I don't know. In any case my first guess attempt would be so simply see how reducing the priority of the low priority worked under different combinations of system load. Someone else also suggested that I might be able to do this same sort of thing with thread priorities. |