Prev: Advice on good Application Programming Practices
Next: how to get 2 button press
From: Stanza on 12 Jan 2010 07:02
I need to write a DLL function which somehow returns a large amount of data.
The caller is a VC++ program and passes in the name of a file, and the
function gathers data from the file contents. The gathered data is then
passed back to the caller. The function will have to read through the whole
file, appending to the gathered data as it goes, so it will not know the
total size of the gathered data until it has read the entire file. The size
of the data could be anything from a few bytes to maybe 100 Megabytes.

What is the best way of passing this data back to the caller?

I suppose I could malloc an initial chunk of memory inside the DLL function,
and repeatedly realloc it when the gathered data filled it up, and then
finally pass the pointer back to the caller which would then have the
responsability of freeing it, but would that work between a program and DLL
function?

Is there a better way to do this?

From: Ulrich Eckhardt on 12 Jan 2010 08:02
Stanza wrote:
> I need to write a DLL function which somehow returns a large amount of
> data. [...] The function will have to read through the whole file,
> appending to the gathered data as it goes, so it will not know
> the total size of the gathered data until it has read the entire file.

Use a linked list, which is much easier to extend than using realloc().

void parse_file(std::istream& in, std::deque<record>& out);

Note: You can use std::deque or std::list, they should both work efficiently
in your case.

Uli


--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

From: ScottMcP [MVP] on 12 Jan 2010 09:07
On Jan 12, 7:02 am, "Stanza" <sta...(a)devnull.com> wrote:
> What is the best way of passing this data back to the caller?
>
> I suppose I could malloc an initial chunk of memory inside the DLL function,
> and repeatedly realloc it when the gathered data filled it up, and then
> finally pass the pointer back to the caller which would then have the
> responsability of freeing it, but would that work between a program and DLL
> function?

The caller can free memory allocated by the DLL if (and only if) they
are both compiled to use the DLL-version of the runtime library. That
means they must both use the /MD or /MDd build option. When this is
done they both use the same instance of the memory manager.

If you do not meet this requirement then another solution is to export
a function from the DLL that will free the memory it allocated.
From: Stanza on 12 Jan 2010 13:38
Thanks for the suggestions. If I wanted the caller to be able to read the
gathered data as it was being generated, how could I do that? I'm thinking
of something like (in UNIX) an unnamed pipe, where a thread in the DLL would
write data to the pipe and the caller in another thread would read it from
the pipe, so that there would be interleaved writing and reading, without
the caller having to wait for the whole file to be read (or having the
caller thread tied up doing the actual data gathering from the file inside
the DLL).

From: David Schwartz on 12 Jan 2010 16:45
On Jan 12, 4:02 am, "Stanza" <sta...(a)devnull.com> wrote:

> I need to write a DLL function which somehow returns a large amount of data.
> The caller is a VC++ program and passes in the name of a file, and the
> function gathers data from the file contents. The gathered data is then
> passed back to the caller. The function will have to read through the whole
> file, appending to the gathered data as it goes, so it will not know the
> total size of the gathered data until it has read the entire file. The size
> of the data could be anything from a few bytes to maybe 100 Megabytes.
>
> What is the best way of passing this data back to the caller?

There are two good ways, and you can easily implement both:

1) Provide 'open', 'read', and 'close' type functions. The 'open'
function begins the operation. The 'read' function retrieves data. The
'close' function is called when the operation is done. (Or you can
implicitly close when the last byte is read.) In this case, the caller
supplies the threads to the library, and the library can either do the
"real work" in those threads or just have the 'read' function block
until other library service threads do the work. In this case, the
application handles memory for the buffers, allocating them, passing
them to 'read', and then freeing or reusing them.

2) Provide a way for the application to pass you a callback function.
As you have data that the application needs to "suck in", you call the
application's callback function to give it the data. In this case, the
library supplies the threads that call the callback function. The
library manages the buffers and passes a pointer and length to the
callback function.

Either way works great, and it's not hard at all to offer both (since
95% of the code is the same). You can support "natively" whichever
method most closely fits how your library thinks and "emulate" the
other with a thin wrapper.

DS
 |  Next  |  Last
Pages: 1 2
Prev: Advice on good Application Programming Practices
Next: how to get 2 button press