From: Peter Olcott on

"Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message
news:as6dnVw3GMrIGVnWnZ2dnUVZ_v2dnZ2d(a)posted.sasktel...
> On 04/13/2010 08:18 AM, Peter Olcott wrote:
>> "Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message
>
>>> Have you considered reading up on the design of current
>>> linux/unix
>>> OS's rather than asking random questions on newsgroups
>>> based on
>>> what random people on the internet say?
>
>> Yes I bought a dozen books for this purpose. The one that
>> covers this topic has not arrived yet. Also the earlier
>> version of this book that is available online does not
>> seem
>> to go into the details that I need.
>>
>> http://oreilly.com/catalog/linuxkernel/chapter/ch10.html
>
> Generally speaking it's a bad idea to code applications
> based on
> internal knowledge of the OS algorithms.
>
> The section you describe gives a substantial amount of
> detail on the
> workings of the scheduler used by linux until relatively
> recently. It
> certainly describes how priorities, timeslices, etc. are
> handled.
>
> What information are you looking for that isn't described
> in that section?
>
> Chris

Ultimately what I am looking for is a way to provide
absolute priority to one kind of job over three other kinds
of jobs.

Each one of these jobs is invoked by another process through
some form of IPC. I am trying to derive the best design to
achieve these requirements.


From: Chris Friesen on
On 04/13/2010 09:04 AM, Peter Olcott wrote:

> Ultimately what I am looking for is a way to provide
> absolute priority to one kind of job over three other kinds
> of jobs.

If you really mean "absolute" priority then you should consider one of
the soft-realtime scheduler policies--SCHED_FIFO or SCHED_RR. (Of
course by default on current linux systems even those are limited to 95%
of the system, though that limit is tuneable.)

Your earlier statements suggested you wanted to give tuneable
percentages though. On current linux the best way to do that would be
to create two different scheduler groups. Give the first group a weight
corresponding to 80% and the other group a weight corresponding to 20%,
put the first kind of job in the first group and the other jobs in the
second group. This is highly specific to linux though.

Chris
From: Peter Olcott on

"Chris Friesen" <cbf123(a)mail.usask.ca> wrote in message
news:3tednYgc4c8iDlnWnZ2dnUVZ_rydnZ2d(a)posted.sasktel...
> On 04/13/2010 09:04 AM, Peter Olcott wrote:
>
>> Ultimately what I am looking for is a way to provide
>> absolute priority to one kind of job over three other
>> kinds
>> of jobs.
>
> If you really mean "absolute" priority then you should
> consider one of
> the soft-realtime scheduler policies--SCHED_FIFO or
> SCHED_RR. (Of
> course by default on current linux systems even those are
> limited to 95%
> of the system, though that limit is tuneable.)
>
> Your earlier statements suggested you wanted to give
> tuneable
> percentages though. On current linux the best way to do
> that would be
> to create two different scheduler groups. Give the first
> group a weight
> corresponding to 80% and the other group a weight
> corresponding to 20%,
> put the first kind of job in the first group and the other
> jobs in the
> second group. This is highly specific to linux though.
>
> Chris

The first process is a web server that has been adapted so
that it can directly interface with four OCR processes or
one OCR process with four threads.

There has also been a big debate over whether or not I
should implement the priority of the four types of OCR jobs
using threads or processes.

I am thinking that something as simple as lowering the
priority of three of the four OCR jobs might provide
everything that I need since this is a server dedicated for
the sole purpose of running the web server / OCR processes.
Either that or raising the priority of the web server and
one of the four OCR processes.


From: Chris Friesen on
On 04/13/2010 10:10 AM, Peter Olcott wrote:

> There has also been a big debate over whether or not I
> should implement the priority of the four types of OCR jobs
> using threads or processes.

Technically nice() and setpriority() both only work on processes. There
is no standard way to specify the nice level of a single thread within a
process. In linux if you specify PRIO_PROCESS and a "who" of 0, the
current thread will be changed but there is no guarantee this will work
on other systems.

> I am thinking that something as simple as lowering the
> priority of three of the four OCR jobs might provide
> everything that I need since this is a server dedicated for
> the sole purpose of running the web server / OCR processes.
> Either that or raising the priority of the web server and
> one of the four OCR processes.

That would be the simplest. Worth a try.

Chris
From: David Schwartz on
On Apr 12, 8:48 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote:

> Someone in another group said that the idea that you
> mentioned in your last sentence had been universally
> abandoned several decades ago because there was too much OS
> overhead involved with this.

That's how progress goes. Ideas are adopted, abandoned, and then
rediscovered. How much OS overhead matters in the scheduler, compared
to how important it is that you do the most important work, has
changed significantly, mostly due to the increasing complexity of CPU
topology.

> I came here to confirm or deny
> the truth of this alternative view.  Of course I could be
> misparaphrasing what they said.

It's going to depend on people's areas of experience. Someone who
works with Windows a lot will have a different idea of what's common
scheduler behavior than someone who only works with Linux. Linux's
scheduling algorithm was recently completely replaced and the way
priority works was completely changed.

In most typical cases, the observable behavior will be the same. If
the process with the highest priority always runs/pre-empts, then
processes with close static priorities will wind up getting
proportional CPU anyway as their dynamic priorities cause them to pre-
empt each other. Processes with vastly differing static priorities
will wind up giving the vast majority of the CPU to the higher-
priority process (assuming it can use it) with either model.

I honestly think your requirements are not unusual and you will have a
hard time getting things to not work if you use the process priority
model. But the best approach is going to be to test on the hardware
you plan to use. Schedulers vary from OS to OS more than just about
anything else.

DS