Prev: THANKS GOD! I GOT $2000 FROM PAYPAL....
Next: constructing and using large lexicon in a program
From: John Nagle on 2 Aug 2010 17:22
On 8/2/2010 12:52 PM, Thomas Jollans wrote:
> On 08/02/2010 09:41 PM, John Nagle wrote:
>> On 8/2/2010 11:02 AM, MRAB wrote:
>>> John Nagle wrote:
>>>> The regular expression "split" behaves slightly differently than
>>>> string split:
>> occurrences of pattern", which is not too helpful.
>>>>
>>> It's the plain str.split() which is unusual in that:
>>>
>>> 1. it splits on sequences of whitespace instead of one per occurrence;
>>
>> That can be emulated with the obvious regular expression:
>>
>> re.compile(r'\W+')
>>
>>> 2. it discards leading and trailing sequences of whitespace.
>>
>> But that can't, or at least I can't figure out how to do it.
>
> [ s in rexp.split(long_s) if s ]

Of course I can discard the blank strings afterward, but
is there some way to do it in the "split" operation? If
not, then the default case for "split()" is too non-standard.

(Also, "if s" won't work; if s != '' might)

John Nagle
From: Thomas Jollans on 2 Aug 2010 18:07
On 08/02/2010 11:22 PM, John Nagle wrote:
>> [ s in rexp.split(long_s) if s ]
>
> Of course I can discard the blank strings afterward, but
> is there some way to do it in the "split" operation? If
> not, then the default case for "split()" is too non-standard.
>
> (Also, "if s" won't work; if s != '' might)

Of course it will work. Empty sequences are considered false in Python.

Python 3.1.2 (release31-maint, Jul 8 2010, 09:18:08)
[GCC 4.4.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> sprexp = re.compile(r'\s+')
>>> [s for s in sprexp.split(' spaces every where ! ') if s]
['spaces', 'every', 'where', '!']
>>> list(filter(bool, sprexp.split(' more spaces \r\n\t\t ')))
['more', 'spaces']
>>>

(of course, the list comprehension I posted earlier was missing a couple
of words, which was very careless of me)
From: samwyse on 2 Aug 2010 20:53
On Aug 2, 12:34 pm, John Nagle <na...(a)animats.com> wrote:
> The regular expression "split" behaves slightly differently than string
> split:

I'm going to argue that it's the string split that's behaving oddly.
To see why, let's first look at some simple CSV values:
cat,dog
,missing,,values,

How many fields are on each line and what are they? Here's what
re.split(',') says:

>>> re.split(',', 'cat,dog')
['cat', 'dog']
>>> re.split(',', ',missing,,values,')
['', 'missing', '', 'values', '']

Note that the presence of missing values is clearly flagged via the
presence of empty strings in the results. Now let's look at string
split:

>>> 'cat,dog'.split(',')
['cat', 'dog']
>>> ',missing,,values,'.split(',')
['', 'missing', '', 'values', '']

It's the same results. Let's try it again, but replacing the commas
with spaces.

>>> re.split(' ', 'cat dog')
['cat', 'dog']
>>> re.split(' ', ' missing values ')
['', 'missing', '', 'values', '']
>>> 'cat dog'.split(' ')
['cat', 'dog']
>>> ' missing values '.split(' ')
['', 'missing', '', 'values', '']

It's the same results; however many people don't like these results
because they feel that whitespace occupies a privileged role. People
generally agree that a string of consecutive commas means missing
values, but a string of consecutive spaces just means someone held the
space-bar down too long. To accommodate this viewpoint, the string
split is special-cased to behave differently when None is passed as a
separator. First, it splits on any number of whitespace characters,
like this:

>>> re.split('\s+', ' missing values ')
['', 'missing', 'values', '']
>>> re.split('\s+', 'cat dog')
['cat', 'dog']

But it also eliminates any empty strings from the head and tail of the
list, because that's what people generally expect when splitting on
whitespace:

>>> 'cat dog'.split(None)
['cat', 'dog']
>>> ' missing values '.split(None)
['missing', 'values']
From: rantingrick on 2 Aug 2010 23:42
On Aug 2, 7:53 pm, samwyse <samw...(a)gmail.com> wrote:

> It's the same results; however many people don't like these results
> because they feel that whitespace occupies a privileged role.  People
> generally agree that a string of consecutive commas means missing
> values, but a string of consecutive spaces just means someone held the
> space-bar down too long.  To accommodate this viewpoint, the string
> split is special-cased to behave differently when None is passed as a
> separator.  First, it splits on any number of whitespace characters,
> like this:

Well we could have created another method like "splitstrip()". However
then folks would complain that they must remember two methods that are
almost identical. Uggh, you just can't win. There is always the
naysayers no matter what you do!

PS: Great post by the way. Highly informative for the pynoobs.

From: John Nagle on 3 Aug 2010 13:41
On 8/2/2010 5:53 PM, samwyse wrote:
> On Aug 2, 12:34 pm, John Nagle<na...(a)animats.com> wrote:
>> The regular expression "split" behaves slightly differently than string
>> split:
>
> I'm going to argue that it's the string split that's behaving oddly.

I tend to agree.

It doesn't seem to be possible to get the same semantics with
any regular expression split. The default "split" has a special
case for head and tail whitespace, and there's no way to express
that with a regular expression split. Applying "strip" first
will work, of course. The documentation should reflect
that.

John Nagle
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: THANKS GOD! I GOT $2000 FROM PAYPAL....
Next: constructing and using large lexicon in a program