Do named pipes have disk I/O ?? [Unix Programming]

Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?

From: Peter Olcott on 8 Apr 2010 16:06

"Jens Thoms Toerring" <jt(a)toerring.de> wrote in message
news:826pq6FaobU1(a)mid.uni-berlin.de...
> In comp.unix.programmer Peter Olcott
> <NoSpam(a)ocr4screen.com> wrote:
>> "David Schwartz" <davids(a)webmaster.com> wrote in message
>> news:4a7d53c3-1d2a-49ff-84d7-def8b6b67678(a)g11g2000yqe.googlegroups.com...
>> > On Apr 8, 9:50 am, "Peter Olcott"
>> > <NoS...(a)OCR4Screen.com>
>> > wrote:
>
>> > > (2) Is there a completely certain way that a write to
>> > > a file
>> > > can be flushed to the disk that encompasses every
>> > > possible
>> > > memory buffer, including the hard drives onboard
>> > > cache? I want
>> > > to be able to yank the power cord at any moment and
>> > > not get
>> > > corrupted data other than the most recent single
>> > > transaction.
>
>> > That's a surprisingly tricky question. Generally,
>> > 'fsync'/'fdatasync' is supposed to do this. But the
>> > choice of
>> > filesystem can be important as well. You may actually
>> > have to
>> > disable the hard drive's write cache, which can really
>> > hurt
>> > performance.
>> >
>> > It may be a better solution simply to use two machines
>> > and to
>> > have each one backup the other's transactions. Simply
>> > report the
>> > transaction is reported to the other machine before
>> > progressing.
>
>> Only one machine is available.
>> What specific "C" commands should be used for reading and
>> writing, I am guessing open, read, and write because they
>> are unbuffered.
>
> They don't use the buffer that gets used by the C
> functions like
> fwrite(), fprintf() etc. But they also only pass the data
> on to
> the kernel, which has it's own buffers. Moreover, you can
> switch
> off buffering for the C functions with setvbuf() and
> friends (or
> you can force flushing of the buffer with fflush() at a
> conve-
> nient time). As David write to get the kernel to flush its
> buf-
> fers to the disk you then will need fsync()/fdatasync(),
> even
> when using write() or with disabled buffer for the C
> functions.
> But that still leaves the disks own write cache. Perhaps
> it
> would be prudent to invest in a UPS if data integrity is
> of
> utter importantance (and then keep everyone away from the
> power
> cables;-)
> Regards, Jens

I just checked the third party provider provides UPS. Most
experts seem to say not to worry about the disk drive's
onboard cache if UPS is available. It looks like the only
alternative that can be counted on for this would be to
disable write caching for the drive.

> --
> \ Jens Thoms Toerring ___ jt(a)toerring.de
> \__________________________ http://toerring.de

From: Scott Lurndal on 8 Apr 2010 16:27

Rainer Weikusat <rweikusat(a)mssgmbh.com> writes:
>scott(a)slp53.sl.home (Scott Lurndal) writes:
>
>[...]
>
>> You don't get this choice. For example, the OS will take locks
>> on your behalf. If you use threads, so will the standard C
>> library.
>
>A kernel running on a multiprocessor will need to use sufficient
>interlocking between different entities executing kernel code in order
>to provide the documented semantics of the system call
>interface. Things are a little different for the C library code,
>though: Since some UNIX(*) interfaces, in particular, fork, have
>different semantics for single-threaded and multi-threaded processes,
>the C library must not silently turn a single-threaded process into a
>multi-threaded one.

I'm not sure I see where taking a lock in, for example, the stdio
portion of the C library "turns a single-threaded process into a
multi-threaded one". Functions in libc don't a priori know that
a particular call comes from a process with more than one thread.

scoty

From: Rick Jones on 8 Apr 2010 16:20

In comp.unix.programmer Peter Olcott <NoSpam(a)ocr4screen.com> wrote:
> It looks like the only alternative that can be counted on for this
> would be to disable write caching for the drive.

Depending on how "conservative" the OS happens to be, or perhaps how
often it got beaten-up on some otherwise uninteresting disc
microbenchmark :) JBOD write caching may be disabled by default.

rick jones
--
a wide gulf separates "what if" from "if only"
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

From: Scott Lurndal on 8 Apr 2010 16:28

"Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>
>"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
>news:5g6vn.1064$MH1.197(a)news.usenetserver.com...
>> Chris Friesen <cbf123(a)mail.usask.ca> writes:
>>>On 04/07/2010 12:32 PM, Peter Olcott wrote:
>>>
>>>> Likewise with my scheduling of my
>>>> processes as compared to the OS scheduling them for me.
>>>
>>>The OS has knowledge of what else the system is doing that
>>>you don't have.
>>>
>>>> Is there any way to tell the hardware cache to load
>>>> specific
>>>> data?
>>>
>>>Generally yes, but it requires assembly code.
>>
>> And generally it is not a good idea. The processor has a
>> better
>> idea of what to prefetch than most application
>> programmers.
>>
>> scott
>
>I am not most application programmers.
>

You shouldn't feed straight lines like that. I suspect that
you'll find little disagreement with that statement here.

scott

From: Scott Lurndal on 8 Apr 2010 16:30

"Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>
>"Scott Lurndal" <scott(a)slp53.sl.home> wrote in message
>news:Zh6vn.1066$MH1.320(a)news.usenetserver.com...
>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> writes:
>>>
>>>People define systems programming in different ways. I
>>>need
>>>to speak to people that do the kind of programming that
>>>you
>>>have referred to, to make sure that my designs work the
>>>way
>>>that I expect them to under the covers, as implemented in
>>>the kernel.
>>>
>>>Here is a specific concrete example:
>>>I am assuming that all pwrite(), pread() and append()
>>>operations are atomic specifically because the kernel
>>>forces
>>>these to be executed sequentially. Is this the means that
>>>the kernel uses to make these operations atomic?
>>
>> The kernel does not "force them to be executed
>> sequentially".
>>
>> The kernel does use mutual exclusion techniques to ensure
>> that
>> the current file position pointer is only updated
>> atomically.
>>
>> scott
>
>If it does not do it by sequencing (the easy way) Then it
>uses some sort of locking mechanism (the slower more
>difficult way)?
>

Why would you conclude that a locking mechanism is slower and
more difficult?

scott