Why is network reading slow?
in [MFC]
|
Prev: Is it a bug in mciSendCommand function?
Next: AFX_MANAGE_STATE() macro disturbs the startup position of the ATL dialog
From: Peter Olcott on 13 Jan 2010 19:52 I can copy a file to and from my Samba network in 20 seconds, yet it takes 80 seconds to compute its MD5 value. The delay is not related to the MD5 processing time because it can compute this in 12 seconds from the local drive. What is going on here?
From: Hector Santos on 13 Jan 2010 21:49 Peter Olcott wrote: > I can copy a file to and from my Samba network in 20 > seconds, yet it takes 80 seconds to compute its MD5 value. > The delay is not related to the MD5 processing time because > it can compute this in 12 seconds from the local drive. > > What is going on here? Probably in how you are reading the file to hash it. Are you doing block I/O or streaming (byte) I/O? To me, that might explain the different timings your describe. -- HLS
From: Hector Santos on 13 Jan 2010 21:57 Hector Santos wrote: > Peter Olcott wrote: > >> I can copy a file to and from my Samba network in 20 seconds, yet it >> takes 80 seconds to compute its MD5 value. The delay is not related to >> the MD5 processing time because it can compute this in 12 seconds from >> the local drive. >> >> What is going on here? > > Probably in how you are reading the file to hash it. > > Are you doing block I/O or streaming (byte) I/O? > > To me, that might explain the different timings your describe. Just to note, the baseline time you are shooting for <= 32 seconds: 20 to read, 12 to process. However, your 12 seconds includes local file I/O time, so whatever that is, your ideal network/processing time should be: 20 + 12 - localIOtime <= 32 -- HLS
From: Peter Olcott on 13 Jan 2010 22:09 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:u3Xi0QMlKHA.1652(a)TK2MSFTNGP05.phx.gbl... > Peter Olcott wrote: > >> I can copy a file to and from my Samba network in 20 >> seconds, yet it takes 80 seconds to compute its MD5 >> value. The delay is not related to the MD5 processing >> time because it can compute this in 12 seconds from the >> local drive. >> >> What is going on here? > > Probably in how you are reading the file to hash it. > > Are you doing block I/O or streaming (byte) I/O? > > To me, that might explain the different timings your > describe. > > -- > HLS I am doing block I/O, and it is very fast on the local drive and much slower on the 1.0 gb LAN, yet file copies to and from the LAN are still fast. (1) File copy to and from the LAN is faster than local drive copies, 20 seconds for LAN, 25 seconds for local. (2) Block I/O is fast on the local drive, 12 seconds for 632 MB. (3) Block I/O is slow on the LAN, 80 seconds for 632 MB. I also tried changing the block size from 4K to 1500 bytes and 9000 bytes (consistent with Ethernet frame size), this did not help.
From: Peter Olcott on 13 Jan 2010 22:18
"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:OsfykVMlKHA.1540(a)TK2MSFTNGP06.phx.gbl... > Hector Santos wrote: > >> Peter Olcott wrote: >> >>> I can copy a file to and from my Samba network in 20 >>> seconds, yet it takes 80 seconds to compute its MD5 >>> value. The delay is not related to the MD5 processing >>> time because it can compute this in 12 seconds from the >>> local drive. >>> >>> What is going on here? >> >> Probably in how you are reading the file to hash it. >> >> Are you doing block I/O or streaming (byte) I/O? >> >> To me, that might explain the different timings your >> describe. > > Just to note, the baseline time you are shooting for <= 32 > seconds: > > 20 to read, 12 to process. > > However, your 12 seconds includes local file I/O time, so > whatever that is, your ideal network/processing time > should be: > > 20 + 12 - localIOtime <= 32 > > -- > HLS It takes 12 seconds to read and process the file locally the first time, it only takes 4 seconds to process the same file again. Both the network and the local machine have equivalent processors and disk drives. The LAN has a 1.0 gb connection. |