Prev: Oracle BI
Next: CLI cursor fetch error
From: Sanjay J on 13 Sep 2007 18:52
Hello, I am also facing same problem. My datasets are also big ( close to
35 millions records ) and I am not sure how to tackle this issue.

Have you found solution to your problem?

Thanks
S J
From: David L Cassell on 14 Sep 2007 01:36
ssj2690(a)YAHOO.COM wrote:
>
>Hello, I am also facing same problem. My datasets are also big ( close to
>35 millions records ) and I am not sure how to tackle this issue.
>
>Have you found solution to your problem?
>
>Thanks
>S J

35M records is just not a BIG data set anymore. Your data set is what, 10
gigs? That's small compared to what a lot of SAS users have these days.

The question is: what are you doing with your data?

I see lots of people sorting when they don't have to, building multiple
copies of large data sets when they don't have to, misusing the WORK
and UTILITY areas when they shouldn't, failing to clean up after themselves,
trying to merge big data sets when they shouldn't, etc. etc. etc.

So what are you doing that causes these error messages? What SAS
version and what OS? How big are your disks, and how big are the
scratch areas you are letting SAS use?

David
--
David L. Cassell
mathematical statistician
Design Pathways
3115 NW Norwood Pl.
Corvallis OR 97330

_________________________________________________________________
A place for moms to take a break!
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
From: Michael Raithel on 14 Sep 2007 08:25
Dear SAS-L-ers,

Sanjay J posted the following:

> Hello, I am also facing same problem. My datasets are also
> big ( close to 35 millions records ) and I am not sure how to
> tackle this issue.
>
> Have you found solution to your problem?
>
Sanjay, as David points out, we could use a bit more information in
order to better understand your problem, and to better advise you.

Nonetheless, I think that you might do well to check out a paper by
hard-working, jet-setting SAS instructor and consultant Andrew Karp
titled something like "How to Handle Large Data Sets". I remember it as
being a very well-written paper full of very practical advice.

I believe that you can access that paper and more on his web site at:

http://www.sierrainformation.com/

....click on the <User Group Presentations> link on the left hand side,
fill out the questionnaire, and have at Andrew's free SAS papers;
looking for the one on Large Data Sets.

Sanjay, best of luck handling large SAS data sets with limited computing
resources!


I hope that this suggestion proves helpful now, and in the future!

Of course, all of these opinions and insights are my own, and do not
reflect those of my organization or my associates. All SAS code and/or
methodologies specified in this posting are for illustrative purposes
only and no warranty is stated or implied as to their accuracy or
applicability. People deciding to use information in this posting do so
at their own risk.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Michael A. Raithel
"The man who wrote the book on performance"
E-mail: MichaelRaithel(a)westat.com

Author: Tuning SAS Applications in the MVS Environment

Author: Tuning SAS Applications in the OS/390 and z/OS Environments,
Second Edition
http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=58172

Author: The Complete Guide to SAS Indexes

http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=60409

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Energy and persistence conquer all things. - Benjamin Franklin
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 | 
Pages: 1
Prev: Oracle BI
Next: CLI cursor fetch error