Prev: FDIS
Next: calling a c function
From: Louis Krupp on 22 Apr 2010 11:06
analyst41(a)hotmail.com wrote:
<snip>
> And yes - 240 Meg of memory is an issue, I have had the program fail
> more than once for "Image size too large".

Wild guess: The problem might be just that -- the program image size --
and not strictly memory usage. Try making the array allocatable and see
if that helps.

Louis
From: Richard Maine on 21 Apr 2010 23:15
e p chandler <epc8(a)juno.com> wrote:

> "Richard Maine" <nospam(a)see.signature> wrote in message
> news:1jhb9j0.jfrt8o1unhkhsN%nospam(a)see.signature...

> > My nose for potential bugs detects two of them here. This assumes two
> > things. Both assumptions might be correct, but I tend to think it good
> > to make such assumptions explicit. Otherwise, you are likely to find
> > that the user of your code doesn't share your assumptions, which asks
> > for bugs.
....
> OK. But please distringuish the mechanics of my test program and its
> assumptions from what it was trying to demonstrate. I do think that I would
> make a production program more robust!

I understand. I might even have done something simillar myself in order
to demonstrate the idea you were after. I just thought it an opportunity
to make a separate point about carefully considering assumptions (at
least in real code). You might well not need the lesson, but enough
people do that I thought it worthwhile to take the opportunity.

--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
From: Dave Allured on 22 Apr 2010 00:21
analyst41(a)hotmail.com wrote:
>
> I need to keep in memory an array of around 60000 character variables,
> each element of which can have a max length of 4000 byres. But if you
> add up the lengths of all the actual data values, it is only 1/8 of
> 60000*4000.
>
> What would be the cleanest way to store this data to take advantage of
> this fact?

Something else to consider, an old fashioned yet simple approach if you
do not need to dynamically edit the set of strings:

integer, parameter :: n_strings = 60000
integer, parameter :: max_len = 4000
integer, parameter :: buf_size = n_strings * (max_len / 8)
character(buf_size) :: strs
character(max_len) :: in
integer i, p, infile
integer p1(n_strings), p2(n_strings)

p = 1

do i = 1, n_strings
read (infile, '(a)') in
p1(i) = p
p = p + len_trim (in)
p2(i) = p - 1
strs(p1(i):p2(i)) = in
end do

i = 1234
write (*,'(a,i0,a,a)') 'string #', i, ' = ', strs(p1(i):p2(i))

The buffer is easily accessed and searched. Inserting, removing, and
reordering are problematical. Like others said, it depends on what you
need to do with the strings.

--Dave
From: Richard Maine on 22 Apr 2010 01:30
Dave Allured <nospom(a)nospom.com> wrote:

> analyst41(a)hotmail.com wrote:
> >
> > I need to keep in memory an array of around 60000 character variables,
> > each element of which can have a max length of 4000 byres. But if you
> > add up the lengths of all the actual data values, it is only 1/8 of
> > 60000*4000.
>
> Something else to consider, an old fashioned yet simple approach if you
> do not need to dynamically edit the set of strings:
[elided]

Yes. That's the character string used as a storage pool approach that
Gordon and I mentioned, though neither of us provided sample code.

--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
From: analyst41 on 22 Apr 2010 06:18
On Apr 22, 12:21 am, Dave Allured <nos...(a)nospom.com> wrote:
> analys...(a)hotmail.com wrote:
>
> > I need to keep in memory an array of around 60000 character variables,
> > each element of which can have a max length of 4000 byres.  But if you
> > add up the lengths of all the actual data values, it is only 1/8 of
> > 60000*4000.
>
> > What would be the cleanest way to store this data to take advantage of
> > this fact?
>
> Something else to consider, an old fashioned yet simple approach if you
> do not need to dynamically edit the set of strings:
>
> integer, parameter  :: n_strings = 60000
> integer, parameter  :: max_len = 4000
> integer, parameter  :: buf_size = n_strings * (max_len / 8)
> character(buf_size) :: strs
> character(max_len)  :: in
> integer i, p, infile
> integer p1(n_strings), p2(n_strings)
>
> p = 1
>
> do i = 1, n_strings
>    read (infile, '(a)') in
>    p1(i) = p
>    p = p + len_trim (in)
>    p2(i) = p - 1
>    strs(p1(i):p2(i)) = in
> end do
>
> i = 1234
> write (*,'(a,i0,a,a)') 'string #', i, ' = ', strs(p1(i):p2(i))
>
> The buffer is easily accessed and searched.  Inserting, removing, and
> reordering are problematical.  Like others said, it depends on what you
> need to do with the strings.
>
> --Dave


At this point, all I want to do is to be able to are things like "find
the nth element", "give me all the elements that contain a particular
substring", "print out all zero length elements" etc.

Inserts etc. are handled outside the Fortran and are not an issue.
With this method, in the event I wanted to sort the elements , it
should be easy to do it through an array of pointers that would
contain the sort order.

Thank you and thanks to all the other responders (whose suggestions
are a bit too advanced for me - but would be a good tutorial when I
have the time).

And yes - 240 Meg of memory is an issue, I have had the program fail
more than once for "Image size too large".
 |  Next  |  Last
Pages: 1 2
Prev: FDIS
Next: calling a c function