From: Jens Müller on
Hi,

I try to decode a string,e.g.
u'M\xfcnchen, pronounced [\u02c8m\u028fn\xe7\u0259n]'.decode('cp1252',
'ignore')
but even thoug I use errors='ignore'
I get UnicodeEncodeError: 'charmap' codec can't encode character u'\u02c8'
in position 21: character maps to <undefined>

How come?

Thanks,
Jens

From: Peter Otten on
Jens Müller wrote:

> I try to decode a string,e.g.
> u'M\xfcnchen, pronounced [\u02c8m\u028fn\xe7\u0259n]'.decode('cp1252',
> 'ignore')
> but even thoug I use errors='ignore'
> I get UnicodeEncodeError: 'charmap' codec can't encode character u'\u02c8'
> in position 21: character maps to <undefined>
>
> How come?

To convert unicode into str you have to *encode()* it.

u"...".decode(...) will implicitly convert to ASCII first, i. e. is
equivalent to

u"...".encode("ascii").decode(...)

Hence the error message

....codec can't encode character u'\u02c8'...

Peter
From: Ulrich Eckhardt on
Jens Müller wrote:
> I try to decode a string,e.g.
> u'M\xfcnchen, pronounced [\u02c8m\u028fn\xe7\u0259n]'.decode('cp1252',
> 'ignore')
> but even thoug I use errors='ignore'
> I get UnicodeEncodeError: 'charmap' codec can't encode character u'\u02c8'
> in position 21: character maps to <undefined>
>
> How come?

Wrong way? Don't you want to encode the Unicode string using codepage 1252
instead?

Uli

--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

From: Jens Müller on
> To convert unicode into str you have to *encode()* it.
>
> u"...".decode(...) will implicitly convert to ASCII first, i. e. is
> equivalent to
>
> u"...".encode("ascii").decode(...)
>
> Hence the error message

Ah - yes of course.

And how can you use the system's default encoding with errors=ignore?
The default encoding is the one that is used if no parameters are given to
"encode".

Thanks again!

From: Lie Ryan on
On 01/12/10 23:50, Jens Müller wrote:
>> To convert unicode into str you have to *encode()* it.
>>
>> u"...".decode(...) will implicitly convert to ASCII first, i. e. is
>> equivalent to
>>
>> u"...".encode("ascii").decode(...)
>>
>> Hence the error message
>
> Ah - yes of course.
>
> And how can you use the system's default encoding with errors=ignore?
> The default encoding is the one that is used if no parameters are given
> to "encode".
>
> Thanks again!


>>> import sys
>>> sys.getdefaultencoding()
'ascii'
>>> u'M\xfcnchen, pronounced
[\u02c8m\u028fn\xe7\u0259n]'.encode(sys.getdefaultencoding(), 'ignore')
'Mnchen, pronounced [mnn]'


unless this is for debugging, I doubt ignoring error in this particular
case is an acceptable solution (how do you pronounce [mnn]?)