Prev: How to call application in the background with subprocess.call
Next: catch clipboard status in gnome
From: Gnarlodious on 8 Apr 2010 10:48
Attempting to read a file containing Unicode characters such as ±:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
5007: ordinal not in range(128)

I did succeed by converting all the characters to HTML entities such
as "±", but I want the characters to be the actual font in the
source file. What am I doing wrong? My understanding is that ALL
strings in Py3 are unicode so... confused.

-- Gnarlie

From: Martin v. Loewis on 8 Apr 2010 11:14
Gnarlodious wrote:
> Attempting to read a file containing Unicode characters such as �:
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
> 5007: ordinal not in range(128)
>
> I did succeed by converting all the characters to HTML entities such
> as "±", but I want the characters to be the actual font in the
> source file. What am I doing wrong? My understanding is that ALL
> strings in Py3 are unicode so... confused.

When opening the file, you need to specify the file encoding. If you
don't, it defaults to ASCII (in your situation; the specific default
depends on the environment).

Regards,
Martin
From: Gnarlodious on 8 Apr 2010 11:52
On Apr 8, 9:14 am, "Martin v. Loewis" wrote:

> When opening the file, you need to specify the file encoding.

OK, I had tried this:

open(path, 'r').read().encode('utf-8')

however I get error

TypeError: Can't convert 'bytes' object to str implicitly

I had assumed a Unicode string was a Unicode string, so why is it a
bytes string?

Sorry, doing Unicode in Py3 has really been a challenge.

-- Gnarlie
From: Martin v. Loewis on 8 Apr 2010 13:04
Gnarlodious wrote:
> On Apr 8, 9:14 am, "Martin v. Loewis" wrote:
>
>> When opening the file, you need to specify the file encoding.
>
> OK, I had tried this:
>
> open(path, 'r').read().encode('utf-8')

No, when *opening* the file, you need to specify the encoding:

open(path, 'r', encoding='utf-8').read()

> Sorry, doing Unicode in Py3 has really been a challenge.

That's because you need to re-learn some things.

Regards,
Martin
From: Gnarlodious on 8 Apr 2010 13:37
On Apr 8, 11:04 am, "Martin v. Loewis" wrote:

> That's because you need to re-learn some things.

Apparently so, every little item is a lesson. Thank you.

-- Gnarlie

 | 
Pages: 1
Prev: How to call application in the background with subprocess.call
Next: catch clipboard status in gnome