Do named pipes have disk I/O ?? [Unix Programming]

Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?

From: Peter Olcott on 7 Apr 2010 17:21

"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
news:We6vn.1063$MH1.937(a)news.usenetserver.com...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>If I am wrong (and I really don't think that I am, I have
>>benchmarking to support my hypotheses) I can always use
>>the
>
> Peter,
>
> Many of the folks you are arguing with create operating
> systems
> or highly sophisticated application stacks for a living.
> You
> should listen a bit more carefully.
>
> Your benchmarks, such as you've described to this point,
> tell you basically _nothing_; Trust me, the current
> system
> that I've architected and built (192 cores, 1TB memory)
> requires
> very substantial and extremely sophisticated modelling and
> run-time performance analysis to determine the memory
> access
> characterstics. Your simple test, which several have
> shown
> to be defective, tells you exactly nothing about the
> memory
> or cache subsystems performance.
>
>>simple mechanism that you proposed. I would benchmark
>>these
>>against each other. Since I will make sure that there are
>>no
>>locks between processes that makes all of these
>
> You don't get this choice. For example, the OS will take
> locks
> on your behalf. If you use threads, so will the
> standard C library.
>
> I think you need to do the following:
>
> 1) re-evaluate your requirements.
>
> - Do you really need real-time response for a batch
> OCR service?
Yes.

> - Do you really need to pre-empt a longer running
> job? Why?
Maybe. Because it could possibly make the high priority
jobs break their real-time limits.

> - Do you seriously think you could run a
> pay-for-service server
> with a one single-core box (hint: your SLA will
> suck).

I don't know the acronym, but, I know it will work because I
know all of the operational parameters. I should be able to
service 100 high priority jobs per second.
>
> 2) Start listening to the professionals who are
> attempting to help
> you, instead of blindly disregarding, ignoring or
> misrepresenting
> their input.

When I hear the reasons then I can agree with those reasons
that are derived from sound reasoning. For example the way
that David switched me from threads to processes.

>
> scott

From: Peter Olcott on 7 Apr 2010 17:22

"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
news:5g6vn.1064$MH1.197(a)news.usenetserver.com...
> Chris Friesen <cbf123(a)mail.usask.ca> writes:
>>On 04/07/2010 12:32 PM, Peter Olcott wrote:
>>
>>> Likewise with my scheduling of my
>>> processes as compared to the OS scheduling them for me.
>>
>>The OS has knowledge of what else the system is doing that
>>you don't have.
>>
>>> Is there any way to tell the hardware cache to load
>>> specific
>>> data?
>>
>>Generally yes, but it requires assembly code.
>
> And generally it is not a good idea. The processor has a
> better
> idea of what to prefetch than most application
> programmers.
>
> scott

I am not most application programmers.

From: David Schwartz on 7 Apr 2010 17:24

On Apr 7, 11:32 am, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote:

> If I am wrong (and I really don't think that I am, I have
> benchmarking to support my hypotheses) I can always use the
> simple mechanism that you proposed. I would benchmark these
> against each other. Since I will make sure that there are no
> locks between processes that makes all of these
> complications moot. In any case if a lock was needed, then
> this is an operation that would not be interrupted.

You can't use benchmarks that way. Your naive assumptions may be right
99% of the time, but in the 1% where they're wrong, you can deadlock
completely. Benchmarks won't detect that.

> Either approach may produce acceptable performance, one
> might be better than the other. It is like compression
> algorithms the generic ones can not perform as well as the
> specialized ones because the specialized ones know more of
> the underlying details. Likewise with my scheduling of my
> processes as compared to the OS scheduling them for me.

If your goal is to write an application that happens to perform
acceptably under the conditions you test it under, then fine. But if
your goal is to design an application that will reliably meet your
requirements, you are going about it all wrong.

> Is there any way to tell the hardware cache to load specific
> data?

That's yet another step in the wrong direction. The hardware cache has
a lot more information than you do -- trying to override its decision
is likely a huge mistake.

DS

From: Peter Olcott on 7 Apr 2010 17:23

"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
news:%g6vn.1065$MH1.582(a)news.usenetserver.com...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>
>>David has mostly convinced me that my 3.5 minute job is
>>best
>>off as its own process. The only issue is that the minimum
>>sleep period seems to be one second, I could really use it
>>to be 100 ms. I might have to build my own sleep system
>>for
>>this process.
>
> man nanosleep.

Yeah I did this before I asked this question, and thought
that it might not also apply to processes.

>
> In addition, learn how to use man -k (e.g. man -k sleep).
>
> scott

From: Peter Olcott on 7 Apr 2010 17:25

"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
news:Zh6vn.1066$MH1.320(a)news.usenetserver.com...
> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>
>>People define systems programming in different ways. I
>>need
>>to speak to people that do the kind of programming that
>>you
>>have referred to, to make sure that my designs work the
>>way
>>that I expect them to under the covers, as implemented in
>>the kernel.
>>
>>Here is a specific concrete example:
>>I am assuming that all pwrite(), pread() and append()
>>operations are atomic specifically because the kernel
>>forces
>>these to be executed sequentially. Is this the means that
>>the kernel uses to make these operations atomic?
>
> The kernel does not "force them to be executed
> sequentially".
>
> The kernel does use mutual exclusion techniques to ensure
> that
> the current file position pointer is only updated
> atomically.
>
> scott

If it does not do it by sequencing (the easy way) Then it
uses some sort of locking mechanism (the slower more
difficult way)?