Prev: dlm= ?
Next: a very complex sas macro
From: smolkatz on 24 May 2010 07:48
Hi,
I have file of 1000000000 records with 2 variables. One of the
variables is random number between 0 to 1. I would like to sort this
file. Proc sort cannot do it (there is not enough place on the disk).
Is there another way to sort this "monster"?
Thanks a lot,
Alex
From: BruceBrad on 25 May 2010 03:26
> I have file of 1000000000 records with 2 variables. One of the
> variables is random number between 0 to 1. I would like to sort this

SAS should be able to handle that on most PCs.
10e9*2*8 = approx 16GB in size.
Sort needs space for temporary files - but I don't think these should
be more than twice the size of the original file (no doubt in the doc
somewhere). Increasing the sortsize option will make the sort run
faster and also reduce disk space needed.

Another approach would be split the file in half, sort each in turn,
then merge on the random variable. This is effectively what proc sort
does, but if you do it manually you can delete the earlier copies as
you go along.
 | 
Pages: 1
Prev: dlm= ?
Next: a very complex sas macro