Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128) #3

Open
Maillouski opened this issue Mar 10, 2017 · 12 comments

Comments

@Maillouski
Copy link

Maillouski commented Mar 10, 2017

this is a limitation of the str() function
Solution :
in web_ui.py line 187 replace it with

--text = str(''.join(text))
++text = (''.join(text)).encode('utf-8')
@mrjbq7
Copy link
Collaborator

mrjbq7 commented Mar 10, 2017

Thanks! There's some python2 vs python3 differences:

python2:

>>> '\xa0'.encode('utf-8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 0: ordinal not in range(128)

vs. python3:

>>> '\xa0'.encode('utf-8')
b'\xc2\xa0'

@wzorroman
Copy link

wzorroman commented Oct 25, 2018

I work:

unicode("cañete").encode('utf8')

@MuerteLee
Copy link

export PYTHONIOENCODING=utf-8

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Nov 9, 2019 via email

@wzorroman
Copy link

can u help?
UnicodeEncodeError: 'ascii' codec can't encode character '\u201c' in position 17: ordinal not in range(128)

what version python do you have?, and django version??
in python 2.7 in the head you put
line 1: # coding: utf-8
line 2: # any thing you want

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Feb 1, 2021 via email

@Hasib104
Copy link

Hasib104 commented Feb 4, 2021

can u help?
UnicodeEncodeError: 'ascii' codec can't encode character '\u201c' in position 17: ordinal not in range(128)

what version python do you have?, and django version??
in python 2.7 in the head you put
line 1: # coding: utf-8
line 2: # any thing you want

tnx

@hansnewton
Copy link

this worked for me:
export LC_ALL=de_DE.UTF-8

@AnweshaSen
Copy link

I got this error (UnicodeEncodeError: 'ascii' codec can't encode character u'\xab' in position 15: ordinal not in range(128)) on python2 while writing shapefile._Record type data to a csv. Now this is for a few records only, have tried setting env variables as mentioned above but nothing worked, what should be the solution ?

Erroneous line:

for i in rd.shapefile_records():
writer.writerow(i)

@mrjbq7
Copy link
Collaborator

mrjbq7 commented Aug 19, 2021 via email

@linehammer
Copy link

Try setting the system default encoding as utf-8 at the start of the script, so that all strings are encoded using that.

Example -

import sys
reload(sys)
sys.setdefaultencoding('utf-8')

The above should set the default encoding as utf-8 .

In most cases, the issue is that when you call str(), python uses the default character encoding to try and encode the bytes you gave it, which in your case are sometimes representations of unicode characters. To fix the problem, you have to tell python how to deal with the string you give it by using .encode('whatever_unicode'). Most of the time, you should be fine using utf-8. So, stop using str() to convert from unicode to encoded text / bytes, properly use .encode() to encode the string:,

yourstring.encode('utf-8')

For python 3.x ,there is default encoding, hence there will be no issue of encoding.

@Alan-MQ
Copy link

Alan-MQ commented Oct 20, 2023

export PYTHONIOENCODING=utf-8

you saved my day bro thanks ~~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

11 participants