From: Carlos Grohmann on
Hi all, I'm using csv to read text files, and its working fine, except
in two cases:

- when there is only one line of text (data) in the file
- when there is a blank line after the last data line

this is the kind of data:

45 67 89
23 45 06
12 34 67
....


and this is the function:

def getData(paths):
"""get data from file and create lists with values and column
names."""

filehandle = paths#[i] # filehandle = os.path.join(dirname,
filename)
csvfile = open(filehandle,'r') # Open the file and read the
contents

sample = csvfile.read( 1024 )# Grab a sample
csvfile.seek( 0 )

dialect = csv.Sniffer().sniff(sample) # Check for file format with
sniffer.
csvfile = csv.reader( csvfile, dialect )
if csv.Sniffer().has_header( sample ): #if there is a header
colnames = csvfile.next() # label columns from first line
datalist = list( csvfile ) # append data to a list
else: #if there is NO header
datalist = list( csvfile ) # append data to a list
colnames = ['col_%i' % i for i in range(len(datalist[0]))] #
label columns as col_1, col_2, etc

return datalist, colnames

TIA for any help.
From: Neil Cerutti on
On 2010-06-03, Carlos Grohmann <carlos.grohmann(a)gmail.com> wrote:
> Hi all, I'm using csv to read text files, and its working fine, except
> in two cases:
>
> - when there is only one line of text (data) in the file
> - when there is a blank line after the last data line
>
> this is the kind of data:
>
> 45 67 89
> 23 45 06
> 12 34 67
> ...

That data doesn't appear to be csv worthy. Why not use str.split
or str.partition?

> and this is the function:
>
> def getData(paths):
> """get data from file and create lists with values and column
> names."""
>
> filehandle = paths#[i] # filehandle = os.path.join(dirname,
> filename)
> csvfile = open(filehandle,'r') # Open the file and read the

In Python 2.6 and earlier, you need to open the file in binary
mode.

In Python 3.0 and later, you need to open the file with a mystic
incantation:

csvfile = open(filehandle, newline='')

> sample = csvfile.read( 1024 )# Grab a sample
> csvfile.seek( 0 )
> dialect = csv.Sniffer().sniff(sample) # Check for file format with
> sniffer.
> csvfile = csv.reader( csvfile, dialect )

Use:

csvfile = csv.reader(csvfile, dialect=dialect)

dialect is a keyword argument.

> if csv.Sniffer().has_header( sample ): #if there is a header
> colnames = csvfile.next() # label columns from first line
> datalist = list( csvfile ) # append data to a list

Do you really need to use the Sniffer? You'll probably be better
off

--
Neil Cerutti
*** You found a dead moose-rat. You sell the hide for $200. ***
From: Neil Cerutti on
On 2010-06-03, Neil Cerutti <neilc(a)norwich.edu> wrote:
> Do you really need to use the Sniffer? You'll probably be better
> off...

....defining your own dialect based on what you know to be the
file format.

--
Neil Cerutti
From: Carlos Grohmann on
Thanks for your prompt response, Neil.

> That data doesn't appear to be csv worthy. Why not use str.split
> or str.partition?

Well, I should have said that this is one kind of data. The function
is part of a larger app, and there is the possibility that someone
uses headers in the data files, or some other field separator, so I
tried to make it more universal.

> In Python 2.6 and earlier, you need to open the file in binary
> mode.
I tried that, no changes

> Use:
>    csvfile = csv.reader(csvfile, dialect=dialect)
> dialect is a keyword argument.

thanks for pointing that out.it stopped the errors when there s only
one
data line, but it still can't get the values for that line

Carlos
From: Neil Cerutti on
On 2010-06-03, Carlos Grohmann <carlos.grohmann(a)gmail.com> wrote:
>> Use:
>> ? ?csvfile = csv.reader(csvfile, dialect=dialect)
>> dialect is a keyword argument.
>
> thanks for pointing that out.it stopped the errors when there s
> only one data line, but it still can't get the values for that
> line

Is it possible your data is ill-formed in that case? If it's
lacking a line-end, I don't know what should happen.

--
Neil Cerutti