Prev: JOGL preload
Next: Popup-dialog steals focus *only from MS Outlook* Why?
From: Boris Punk on 5 Aug 2010 00:46
Why does the write speed slow down the bigger the file gets? For example
creating a 10GB file is fairly fast but get to 20GB+ and the write speed
drops and continues to drop.


From: bugbear on 5 Aug 2010 03:55
Boris Punk wrote:
> Why does the write speed slow down the bigger the file gets? For example
> creating a 10GB file is fairly fast but get to 20GB+ and the write speed
> drops and continues to drop.

I suspect this is an OS issue, not a Java issue.

My (further) guess would be cache limits.

BugBear
From: Lew on 5 Aug 2010 08:28
Boris Punk wrote:
>> Why does the write speed slow down the bigger the file gets? For
>> example creating a 10GB file is fairly fast but get to 20GB+ and the
>> write speed drops and continues to drop.

bugbear wrote:
> I suspect this is an OS issue, not a Java issue.
>
> My (further) guess would be cache limits.

OS issue sounds right. On which OS is this?

My guess is Windows and that the file is fragmented, requiring frequent visits
to the directory entries.

It's not just the OS, it's the file system used by the OS. Does the behavior
occur on different OSes and with different file systems?

--
Lew
From: markspace on 5 Aug 2010 13:50
bugbear wrote:
> Boris Punk wrote:
>> Why does the write speed slow down the bigger the file gets? For
>> example creating a 10GB file is fairly fast but get to 20GB+ and the
>> write speed drops and continues to drop.
>
> I suspect this is an OS issue, not a Java issue.
>
> My (further) guess would be cache limits.


Yup. Or could be anything else, since the OP has shown us no code.
Could be garbage collector, since we don't know how many objects are
created and released during the run of his code.

Could be some part of the Java system. ObjectOutputStream retains
objects, iirc, filling memory and making the garbage collector work harder.

Could be sunspots. Could be cosmic rays. Could be faulty overheating
hardware. Could be some other process running and thrashing the IO.

Or could be anything else, since the OP has shown us no code.
From: Thomas Pornin on 5 Aug 2010 13:52
According to Lew <noone(a)lewscanon.com>:
> It's not just the OS, it's the file system used by the OS. Does the
> behavior occur on different OSes and with different file systems?

And it could be the disk itself, too. Disks are faster near their
beginning; a factor of 2 is commonly observed (at least that's what I
get with my harddisks, e.g. when rebuilding the RAID1 array: starts at
105 MB/s, but slowed down to about 45 MB/s near the end).


--Thomas Pornin
 |  Next  |  Last
Pages: 1 2 3
Prev: JOGL preload
Next: Popup-dialog steals focus *only from MS Outlook* Why?