Prev: Time
Next: CImage trouble on Windows 7
From: AliR on 2 Apr 2010 08:33
I'm not aware of any restrictions.

You said that writing it chunks "might not be possible", do you have a good
reason for that?

Typically no one really writes an entire file out all at once, 90% of the
time the file is written out in chuncks. How do you keep 750GB if data in
memory to begin with?

To answer your own question, I would suggest writing a program that writes
512MB chuncks at a time and see where it gives you an error.

AliR.



"dave" <dave(a)127.0.0.1> wrote in message
news:11nbr5doi0l8kqe8509bru41rj7hgu8hqv(a)4ax.com...
> Does someone have an MFC (of API) code extract that can write a
> single huge file to disc? (~700Gb). This is just to see if it can be
> done :-)
>
> I've made a few attempts using WriteFile() and __int64 int's but the
> problems arise becuase WriteFile() args are all (afaic see) 32 bit
> ints. So it's iompossible to say "write 750000000000 bytes" in the
> bytecount arg. Even to do it in chuncks of 2Gb might not be possible
> as the offset in file pointer args is also 32 bits.
> Just wondered if anyone out there has managed to do this?
> Thanks


From: David Lowndes on 2 Apr 2010 09:55
>So it's iompossible to say "write 750000000000 bytes" in the
>bytecount arg. Even to do it in chuncks of 2Gb might not be possible
>as the offset in file pointer args is also 32 bits.

SetFilePointer has 2 32-bit input parameters lDistanceToMove &
lpDistanceToMoveHigh.

Having said that, SetFilePointerEx accepts 64-bit values directly.

Dave
From: Joseph M. Newcomer on 2 Apr 2010 11:20
Key here is that you have confused "writing a large file" with "a single write operation"
and lumped them into one question. For example

CStdioFile f;
.....open file on f;
CString star(_T("*"));
for(LONGLONG i = 0; i < 700000000000; i++)
f.WriteString(star);
can write a large file. But it never writes more than 1 byte at a time (this will take a
while...)

However, to write it all in a single WriteFile operation, even in Win64, is impossible,
because the low-level ::WriteFile API only takes a DWORD (not a DWORD_PTR or ULONGLONG)
length (I consider this a deep failure in the Win64 API).

Also, you can only have data < 2GB (or in practice, <1GB) in Win64. You cannot have more
than 2GB data, EVER, in a Win32 app running on normal Win32, and even running a
/LARGEADDRESSAWARE app on Win32 booted with the 3GB address space option, you are unlikely
to get 3GB of available data, and running such a program on Win64, you can have at MOST
4GB data (actually, a bit smaller, since you have to allow space for your code and your
stack, and other heap objects; your DLLs will fragment this so the max you can really get
in practice is yet smaller). So it is IMPOSSIBLE to have 700BG of buffer in a Win32
program, and stretching credulity to have it in WIn64 (you had better have > 700GB of
paging file space available, or you can't even get that much address space in Win64).

So can you restate the problem to ask the question you intend?
joe
****
On Fri, 02 Apr 2010 13:04:21 +0100, dave <dave(a)127.0.0.1> wrote:

>Does someone have an MFC (of API) code extract that can write a
>single huge file to disc? (~700Gb). This is just to see if it can be
>done :-)
>
>I've made a few attempts using WriteFile() and __int64 int's but the
>problems arise becuase WriteFile() args are all (afaic see) 32 bit
>ints. So it's iompossible to say "write 750000000000 bytes" in the
>bytecount arg. Even to do it in chuncks of 2Gb might not be possible
>as the offset in file pointer args is also 32 bits.
>Just wondered if anyone out there has managed to do this?
>Thanks
Joseph M. Newcomer [MVP]
email: newcomer(a)flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm
 | 
Pages: 1
Prev: Time
Next: CImage trouble on Windows 7