Prev: container of iterators
Next: VC8 compiler behavior?
From: Vivek on 19 Aug 2006 10:30
Hi,


Newbie alert: Please be nice :)

I'm using IStream (basic_iostream class to be specific) interface and I
am trying to store very large Unicode XML data into a stream and it
seems to be overflowing. The symptom for the overflow *seems* to be
related to the size of the data that I am trying to stream in. If the
amount of data gets too large (for e.g. 128MB+) the data gets corrupted
and for XML, it obviously becomes useless. The amount of memory on my
system is 4GB, so it is NOT a physical memory problem.

If anyone else has experienced any limitations with IStream and would
like to share their experiences with me, I would really appreciate that!

Does anyone have any efficient code examples to utilize IStream::SetSize
to dynamically allocate more memory when required rather than
occupying too much of statically assigned memory? The reason for why I
ask is that the data size that I stream in, is unknown at the time of
creating and using the stream.

-> How to detect an overflow for IStream?
-> What the default size limit for an IStream is?


Regards,

Vivek

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Ismail Pazarbasi on 19 Aug 2006 20:37
On Sat, 19 Aug 2006 16:30:33 +0200, Vivek <vivekd(a)optusnet.com.au> wrote:

> Hi,
>
>
> Newbie alert: Please be nice :)
>
> I'm using IStream (basic_iostream class to be specific) interface and I
> am trying to store very large Unicode XML data into a stream and it
> seems to be overflowing. The symptom for the overflow *seems* to be
> related to the size of the data that I am trying to stream in. If the
> amount of data gets too large (for e.g. 128MB+) the data gets corrupted
> and for XML, it obviously becomes useless. The amount of memory on my
> system is 4GB, so it is NOT a physical memory problem.
>
> If anyone else has experienced any limitations with IStream and would
> like to share their experiences with me, I would really appreciate that!
>
> Does anyone have any efficient code examples to utilize IStream::SetSize
> to dynamically allocate more memory when required rather than
> occupying too much of statically assigned memory? The reason for why I
> ask is that the data size that I stream in, is unknown at the time of
> creating and using the stream.
>
> -> How to detect an overflow for IStream?
> -> What the default size limit for an IStream is?
>
>
> Regards,
>
> Vivek



Hi,

I think you need information about COM's IStream, which is Microsoft
specific issue. I recommend you to ask your question to
microsoft.public.win32.programmer.ole or another OLE related group.

I am not sure whether IStream has a size limit, but I think it's at least
2GB. If you obtain IStream from an Open call from a storage, you may not
alter its size. To allocate some memory and get an IStream pointer for
that block of memory, you can use CreateStreamOnHGlobal.

HTH

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: Alf P. Steinbach on 19 Aug 2006 20:30
* Vivek:
> I'm using IStream (basic_iostream class to be specific) interface and I
> am trying to store very large Unicode XML data into a stream and it
> seems to be overflowing. The symptom for the overflow *seems* to be
> related to the size of the data that I am trying to stream in. If the
> amount of data gets too large (for e.g. 128MB+) the data gets corrupted
> and for XML, it obviously becomes useless. The amount of memory on my
> system is 4GB, so it is NOT a physical memory problem.

Try to output the data in smaller chunks, and flush the stream for each
chunk.


> If anyone else has experienced any limitations with IStream and would
> like to share their experiences with me, I would really appreciate that!

IStream is not a name from the standard library, and it doesn't sound
like a stream you'd be outputting data to: it sounds like an input stream.

--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

From: kanze on 21 Aug 2006 07:32
Vivek wrote:

> I'm using IStream (basic_iostream class to be specific)
> interface and I am trying to store

I'm not sure I understand. The class is std::istream, not
IStream, it is a typedef for an instanciation of
std::basic_istream, not std::basic_iostream, and it doesn't
support writing, only reading.

> very large Unicode XML data into a stream and it
> seems to be overflowing.

And if you are working with Unicode, you probably want
std::wostream, rather than std::ostream.

> The symptom for the overflow *seems* to be
> related to the size of the data that I am trying to stream in. If the
> amount of data gets too large (for e.g. 128MB+) the data gets corrupted
> and for XML, it obviously becomes useless. The amount of memory on my
> system is 4GB, so it is NOT a physical memory problem.

The only limitations to the size are normally those concerning
the maximum size of a file imposed by the operating system.
Something which should be, at the very least, in the order of
Gigabytes.

> If anyone else has experienced any limitations with IStream
> and would like to share their experiences with me, I would
> really appreciate that!

> Does anyone have any efficient code examples to utilize
> IStream::SetSize

std::istream doesn't have a member SetSize. You can use setbuf
to control the buffer size, but this is usually
counter-productive; a good implementation will automatically use
the best buffer size if you let it.

--
James Kanze GABI Software
Conseils en informatique oriente objet/
Beratung in objektorientierter Datenverarbeitung
9 place Smard, 78210 St.-Cyr-l'cole, France, +33 (0)1 30 23 00 34


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]

 | 
Pages: 1
Prev: container of iterators
Next: VC8 compiler behavior?