Flushing file writes to disk with 100% reliability
in [Unix Programming]
|
Prev: Learn about proxy sites and how to use them to open blocked sites unlimited downloads from RapidShare and megaupload and increase the speed of the Internet with new sites for free
Next: How reliable are named pipes?
From: Peter Olcott on 20 Apr 2010 08:47 "Casper H.S. Dik" <Casper.Dik(a)Sun.COM> wrote in message news:4bcd64ac$0$22903$e4fe514c(a)news.xs4all.nl... > Tony Delroy <tony_in_da_uk(a)yahoo.co.uk> writes: > >>On Apr 9, 1:58=A0am, "Peter Olcott" >><NoS...(a)OCR4Screen.com> wrote: >>> Is there a completely certain way that a write to a file >>> can be flushed to the disk that encompasses every >>> possible >>> memory buffer, including the hard drives onboard cache? >>> =A0I >>> want to be able to yank the power cord at any moment and >>> not >>> get corrupted data other than the most recent single >>> transaction. > >>You'd have to check the hard disk programming documents, >>you may be >>able to do direct I/O to ensure your data is written. >>Even if the >>drive's onboard cache has not been flushed, it might have >>enough >>capacitance to flush during a power failure, or use >>non-volatile >>memory that can be flushed when power returns. Above >>that, the answer >>is highly OS dependent, and you've specified absolutely >>nothing about >>your hardware, OS, programming language etc.... > > The OS should shield the programmer from the particulars > of the > hardware. So read the manuals and hope they give you a > promise > you can live with (and not lie to you) > > (I'm somewhat disappointed that fsync() in Linux doesn't > offer anything > if your write cache is enabled) There is a "Flush Cache" ATA command on some SATA drives. From what I was able to find out turning off the write cache is a bad idea too. It wears out the drive much more quickly because this maximizes rather this minimizes drive head movement. I was also able to figure out that groups of transactions could be batched together to increase performance, if there is a very high transaction rate. Turning off write cache would prohibit this. This could still be reliable because each batch of transactions could be flushed to disk together. This could provide as much as a 1000-fold increase in performance without losing any reliability, and depends upon write cache being turned on. > > Casper > -- > Expressed in this posting are my opinions. They are in no > way related > to opinions held by my employer, Sun Microsystems. > Statements on Sun products included here are not gospel > and may > be fiction rather than truth.
From: robertwessel2 on 20 Apr 2010 18:44 On Apr 20, 1:06 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > "Joe Beanfish" <j...(a)nospam.duh> wrote in message > > news:hqkp2c$c98(a)news.thunderstone.com... > > > > > > > On 04/20/10 08:38, Peter Olcott wrote: > >> "Tony Delroy"<tony_in_da...(a)yahoo.co.uk> wrote in > >> message > >>news:7ed1ecad-c641-4d32-bcef-6bd5b3bc5632(a)c20g2000prb.googlegroups.com.... > >> On Apr 9, 1:58 am, "Peter Olcott"<NoS...(a)OCR4Screen.com> > >> wrote: > >>> Is there a completely certain way that a write to a file > >>> can be flushed to the disk that encompasses every > >>> possible > >>> memory buffer, including the hard drives onboard cache? > >>> I > >>> want to be able to yank the power cord at any moment and > >>> not > >>> get corrupted data other than the most recent single > >>> transaction. > > >> --You'd have to check the hard disk programming > >> documents, > >> you may be > >> --able to do direct I/O to ensure your data is written. > >> Even if the > >> --drive's onboard cache has not been flushed, it might > >> have > >> enough > >> --capacitance to flush during a power failure, or use > >> non-volatile > >> --memory that can be flushed when power returns. Above > >> that, the answer > >> --is highly OS dependent, and you've specified absolutely > >> nothing about > >> --your hardware, OS, programming language etc.... > >> -- > >> --Cheers, > >> --Tony > > >> It looks like that OS is not the big problem. The OS can > >> always be bypassed, by working directly with the > >> hardware. > >> The big problem is that for example Western Digital SATA > >> drives simply do not implement the "Flush Cache" ATA > >> command. > > >> Seagate drives do implement this command. It was > >> Seagate's > >> idea to create this command in 2001. Although it may > >> still > >> be possible to simply shut off write caching for these > >> drives, this will wear the drive out much more quickly, > >> and > >> drastically reduce performance. > > > Have you considered solid state hard disks? Server > > quality, not the > > cheap desktop quality ones. > > > IMHO, with a magnetic HD with a journalling filesystem and > > a good UPS > > with software to shutdown before battery runs out are all > > you need. > > Then you won't have to sacrifice speed trying to sync all > > the way to > > the hard media. > > I will be renting my system from my service provider, thus > no choices are available for hardware. Both UPS and backup > generators are provided by my service provider. > > SSD have a limited life that is generally not compatible > with extremely high numbers of transactions. > > Some drives might not even be smart enough to flush their > buffers even when UPS kicks in. I guess that you could force > a buffer flush for every drive by simply writing a file > larger than the buffer. If you make sure that this file is > not fragmented, it might even be fast enough to do this > after every transaction. > > Obviously the best way to do this would be to have a drive > that correctly implements some sort of "Flush Cache" command > such as the ATA command. Does your service provider offer a system with SAS (aka SCSI) disks? Support of the Synchronize-Cache command is pretty universal.
From: David Schwartz on 20 Apr 2010 18:52 On Apr 20, 11:06 am, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > I will be renting my system from my service provider, thus > no choices are available for hardware. Both UPS and backup > generators are provided by my service provider. I think your combination of requirements are impossible to meet. At a minimum, the only way to establish that your system meets the high reliability rates in your requirements is to test failure conditions on the actual hardware that will be used. That will not be possible on rented hardware. You need to go back to the requirements and make them more rational. Don't state them in absolutes, but state them in analyzable form. Decide how much, say, a lost transaction will cost you. That way, you can make a rational decision on whether it's worth, say, an extra $1,000 to drop that chance from .01% to .001% or not. Think about how many transactions per day, how many power failures per year, how many disk failures per year, and so on. Assess how big the vulnerability window is and then you can figure the odds of a failure in the vulnerability window. It will cost money to shrink that window, so you need to know how much it's worth the make rational implementation decisions. DS
From: Peter Olcott on 20 Apr 2010 19:00 <robertwessel2(a)yahoo.com> wrote in message news:e151e29e-0c31-474f-a08a-0cc1a4211e9e(a)x12g2000yqx.googlegroups.com... On Apr 20, 1:06 pm, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > "Joe Beanfish" <j...(a)nospam.duh> wrote in message > > news:hqkp2c$c98(a)news.thunderstone.com... > > > > > > > On 04/20/10 08:38, Peter Olcott wrote: > >> "Tony Delroy"<tony_in_da...(a)yahoo.co.uk> wrote in > >> message > >>news:7ed1ecad-c641-4d32-bcef-6bd5b3bc5632(a)c20g2000prb.googlegroups.com... > >> On Apr 9, 1:58 am, "Peter > >> Olcott"<NoS...(a)OCR4Screen.com> > >> wrote: > >>> Is there a completely certain way that a write to a > >>> file > >>> can be flushed to the disk that encompasses every > >>> possible > >>> memory buffer, including the hard drives onboard > >>> cache? > >>> I > >>> want to be able to yank the power cord at any moment > >>> and > >>> not > >>> get corrupted data other than the most recent single > >>> transaction. > > >> --You'd have to check the hard disk programming > >> documents, > >> you may be > >> --able to do direct I/O to ensure your data is written. > >> Even if the > >> --drive's onboard cache has not been flushed, it might > >> have > >> enough > >> --capacitance to flush during a power failure, or use > >> non-volatile > >> --memory that can be flushed when power returns. Above > >> that, the answer > >> --is highly OS dependent, and you've specified > >> absolutely > >> nothing about > >> --your hardware, OS, programming language etc.... > >> -- > >> --Cheers, > >> --Tony > > >> It looks like that OS is not the big problem. The OS > >> can > >> always be bypassed, by working directly with the > >> hardware. > >> The big problem is that for example Western Digital > >> SATA > >> drives simply do not implement the "Flush Cache" ATA > >> command. > > >> Seagate drives do implement this command. It was > >> Seagate's > >> idea to create this command in 2001. Although it may > >> still > >> be possible to simply shut off write caching for these > >> drives, this will wear the drive out much more quickly, > >> and > >> drastically reduce performance. > > > Have you considered solid state hard disks? Server > > quality, not the > > cheap desktop quality ones. > > > IMHO, with a magnetic HD with a journalling filesystem > > and > > a good UPS > > with software to shutdown before battery runs out are > > all > > you need. > > Then you won't have to sacrifice speed trying to sync > > all > > the way to > > the hard media. > > I will be renting my system from my service provider, thus > no choices are available for hardware. Both UPS and backup > generators are provided by my service provider. > > SSD have a limited life that is generally not compatible > with extremely high numbers of transactions. > > Some drives might not even be smart enough to flush their > buffers even when UPS kicks in. I guess that you could > force > a buffer flush for every drive by simply writing a file > larger than the buffer. If you make sure that this file is > not fragmented, it might even be fast enough to do this > after every transaction. > > Obviously the best way to do this would be to have a drive > that correctly implements some sort of "Flush Cache" > command > such as the ATA command. --Does your service provider offer a system with SAS (aka SCSI) disks? --Support of the Synchronize-Cache command is pretty universal. It looks like the answer is no. It is good to hear that a switch to SCSI will solve this problem, that is what I expected.
From: Peter Olcott on 20 Apr 2010 19:16
"David Schwartz" <davids(a)webmaster.com> wrote in message news:1fedd1f3-df14-4879-a652-5757b5086b70(a)w32g2000prc.googlegroups.com... On Apr 20, 11:06 am, "Peter Olcott" <NoS...(a)OCR4Screen.com> wrote: > I will be renting my system from my service provider, thus > no choices are available for hardware. Both UPS and backup > generators are provided by my service provider. --I think your combination of requirements are impossible to meet. --At a minimum, the only way to establish that your system meets the --high reliability rates in your requirements is to test failure --conditions on the actual hardware that will be used. That will not be --possible on rented hardware. That is one of the reasons why I bought identical hardware. --You need to go back to the requirements and make them more rational. --Don't state them in absolutes, but state them in analyzable form. --Decide how much, say, a lost transaction will cost you. That way, you --can make a rational decision on whether it's worth, say, an extra --$1,000 to drop that chance from .01% to .001% or not. Yeah I already figured that out. The one transaction that I can not afford to lose, is when the customer adds money to their account. I don't want to ever lose the customer's money. The payment processor already provides backup of this. I already figured out a way to provide transaction by transaction offsite backup relatively easily. I will do this as soon as it is worth the effort. I will plan for this in advance to reduce the time to implement it. There is another option that I figured out might work. I could always flush the cache of any drive by following every transaction with a cache sized file write. Since this will be burst mode speed it might be fast enough if the file has no fragmentation. Horribly inefficient, but, a possibly a passable interim solution. --Think about how many transactions per day, how many power failures per --year, how many disk failures per year, and so on. Assess how big the --vulnerability window is and then you can figure the odds of a failure --in the vulnerability window. It will cost money to shrink that window, --so you need to know how much it's worth the make rational --implementation decisions. --DS I did all that. Basically my biggest issue is that I may not charge a customer for a completed job. Worst case I may lose a whole day's worth of charges. I don't think that giving the customer something for free once in a while will hurt my business. As soon as these charges amount to very much money (or sooner) I will fix this one way or another. It looks like the most cost effective solution is some sort of transaction by transaction offsite backup. I might simply have the system email each transaction to me. |