Do named pipes have disk I/O ?? [Unix Programming]

Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: Why can't I catch a SIGBUS on my Mac (when strlen(NULL) is used)?

From: David Schwartz on 8 Apr 2010 13:44

On Apr 8, 9:50 am, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote:

> (2) Is there a completely certain way that a write to a file
> can be flushed to the disk that encompasses every possible
> memory buffer, including the hard drives onboard cache? I
> want to be able to yank the power cord at any moment and not
> get corrupted data other than the most recent single
> transaction.

That's a surprisingly tricky question. Generally, 'fsync'/'fdatasync'
is supposed to do this. But the choice of filesystem can be important
as well. You may actually have to disable the hard drive's write
cache, which can really hurt performance.

It may be a better solution simply to use two machines and to have
each one backup the other's transactions. Simply report the
transaction is reported to the other machine before progressing.

DS

From: David Schwartz on 8 Apr 2010 13:45

On Apr 8, 9:56 am, Rainer Weikusat <rweiku...(a)mssgmbh.com> wrote:

> I have no idea what you're up to.

Now you know how I feel.

DS

From: Peter Olcott on 8 Apr 2010 14:17

Only one machine is available.
What specific "C" commands should be used for reading and
writing, I am guessing open, read, and write because they
are unbuffered.

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:4a7d53c3-1d2a-49ff-84d7-def8b6b67678(a)g11g2000yqe.googlegroups.com...
On Apr 8, 9:50 am, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> (2) Is there a completely certain way that a write to a
> file
> can be flushed to the disk that encompasses every possible
> memory buffer, including the hard drives onboard cache? I
> want to be able to yank the power cord at any moment and
> not
> get corrupted data other than the most recent single
> transaction.

That's a surprisingly tricky question. Generally,
'fsync'/'fdatasync'
is supposed to do this. But the choice of filesystem can be
important
as well. You may actually have to disable the hard drive's
write
cache, which can really hurt performance.

It may be a better solution simply to use two machines and
to have
each one backup the other's transactions. Simply report the
transaction is reported to the other machine before
progressing.

DS

From: Peter Olcott on 8 Apr 2010 15:19

Here is an interesting link.
http://mail.nl.linux.org/linux-crypto/2005-10/msg00004.html

"David Schwartz" <davids(a)webmaster.com> wrote in message
news:4a7d53c3-1d2a-49ff-84d7-def8b6b67678(a)g11g2000yqe.googlegroups.com...
On Apr 8, 9:50 am, "Peter Olcott" <NoS...(a)OCR4Screen.com>
wrote:

> (2) Is there a completely certain way that a write to a
> file
> can be flushed to the disk that encompasses every possible
> memory buffer, including the hard drives onboard cache? I
> want to be able to yank the power cord at any moment and
> not
> get corrupted data other than the most recent single
> transaction.

That's a surprisingly tricky question. Generally,
'fsync'/'fdatasync'
is supposed to do this. But the choice of filesystem can be
important
as well. You may actually have to disable the hard drive's
write
cache, which can really hurt performance.

It may be a better solution simply to use two machines and
to have
each one backup the other's transactions. Simply report the
transaction is reported to the other machine before
progressing.

DS

From: Jens Thoms Toerring on 8 Apr 2010 15:40

In comp.unix.programmer Peter Olcott <NoSpam(a)ocr4screen.com> wrote:
> "David Schwartz" <davids(a)webmaster.com> wrote in message
> news:4a7d53c3-1d2a-49ff-84d7-def8b6b67678(a)g11g2000yqe.googlegroups.com...
> > On Apr 8, 9:50 am, "Peter Olcott" <NoS...(a)OCR4Screen.com>
> > wrote:

> > > (2) Is there a completely certain way that a write to a file
> > > can be flushed to the disk that encompasses every possible
> > > memory buffer, including the hard drives onboard cache? I want
> > > to be able to yank the power cord at any moment and not get
> > > corrupted data other than the most recent single transaction.

> > That's a surprisingly tricky question. Generally,
> > 'fsync'/'fdatasync' is supposed to do this. But the choice of
> > filesystem can be important as well. You may actually have to
> > disable the hard drive's write cache, which can really hurt
> > performance.
> >
> > It may be a better solution simply to use two machines and to
> > have each one backup the other's transactions. Simply report the
> > transaction is reported to the other machine before progressing.

> Only one machine is available.
> What specific "C" commands should be used for reading and
> writing, I am guessing open, read, and write because they
> are unbuffered.

They don't use the buffer that gets used by the C functions like
fwrite(), fprintf() etc. But they also only pass the data on to
the kernel, which has it's own buffers. Moreover, you can switch
off buffering for the C functions with setvbuf() and friends (or
you can force flushing of the buffer with fflush() at a conve-
nient time). As David write to get the kernel to flush its buf-
fers to the disk you then will need fsync()/fdatasync(), even
when using write() or with disabled buffer for the C functions.
But that still leaves the disks own write cache. Perhaps it
would be prudent to invest in a UPS if data integrity is of
utter importantance (and then keep everyone away from the power
cables;-)
Regards, Jens
--
\ Jens Thoms Toerring ___ jt(a)toerring.de
\__________________________ http://toerring.de