By Ali Nadalizadeh

2010-02-16 20:46:28 8 Comments

I have many "can't encode" and "can't decode" problems with Python when I run my applications from the console. But in the Eclipse PyDev IDE, the default character encoding is set to UTF-8, and I'm fine.

I searched around for setting the default encoding, and people say that Python deletes the sys.setdefaultencoding function on startup, and we can not use it.

So what's the best solution for it?


@Eric O Lebigot 2013-07-13 08:18:16

Here is a simpler method (hack) that gives you back the setdefaultencoding() function that was deleted from sys:

import sys
# sys.setdefaultencoding() does not exist, here!
reload(sys)  # Reload does the trick!

(Note for Python 3.4+: reload() is in the importlib library.)

This is not a safe thing to do, though: this is obviously a hack, since sys.setdefaultencoding() is purposely removed from sys when Python starts. Reenabling it and changing the default encoding can break code that relies on ASCII being the default (this code can be third-party, which would generally make fixing it impossible or dangerous).

@ibotty 2015-08-09 19:33:45

I downvoted, because that answer doesn't help for running existing applications (which is one way to interpret the question), is wrong when you are writing/maintaining an application and dangerous when writing a library. The right way is to set LC_CTYPE (or in an application, check whether it is set right and abort with a meaningful error message).

@Eric O Lebigot 2015-08-10 00:16:39

@ibotty I do agree that this answer is a hack and that it is dangerous to use it. It does answer the question, though ("Changing default encoding of Python?"). Do you have a reference about the effect of the environment variable LC_CTYPE on the Python interpreter?

@ibotty 2015-08-10 11:46:16

well, it did not mention, it's a hack at first. other than that, dangerous answers that lack any mention that they are, are not helpful.

@ibotty 2015-08-10 11:49:52

@Eric O Lebigot 2015-08-10 17:55:41

@ibotty LC_CTYPE is independent from the "default encoding of Python" that the question refers to (sys.getdefaultencoding() returns ascii for me, with LC_CTYPE=en_US.UTF-8). So, what (different) problem that you have in mind does LC_CTYPE solve?

@ibotty 2015-08-11 08:05:38

@EOL you are right. It does effect the preferredencoding though (in python 2 and 3): LC_CTYPE=C python -c 'import locale; print( locale.getpreferredencoding())'

@Marlon Abeykoon 2016-06-07 09:31:30

@user2394901 The use of sys.setdefaultencoding() has always been discouraged!! And the encoding of py3k is hard-wired to "utf-8" and changing it raises an error.

@kiril 2016-09-16 13:11:13

I'm using ipython notebooks, and in my case, as soon as I execute the hack the print function no longer prints to stdout.

@Eric O Lebigot 2016-09-21 17:15:11

… and print still does not work when setting back the default encoding to 'ascii'… This smells like some magic done by the notebook… In any case, this solution is only a hack, and can thus break.

@Superdooperhero 2020-02-13 17:27:53

Gives me NameError: name 'reload' is not defined

@Eric O Lebigot 2020-03-01 09:08:19

In Python 3.4+, you need to use the reload() from the importlib library.

@Boris 2020-02-17 18:41:38

You could change the encoding of your entire operating system. On Ubuntu you can do this with

sudo apt install locales 
sudo locale-gen en_US en_US.UTF-8    
sudo dpkg-reconfigure locales

@twasbrillig 2018-04-12 21:38:57

This fixed the issue for me.

import os
os.environ["PYTHONIOENCODING"] = "utf-8"

@Dalton Bentley 2017-06-05 23:41:02

This is a quick hack for anyone who is (1) On a Windows platform (2) running Python 2.7 and (3) annoyed because a nice piece of software (i.e., not written by you so not immediately a candidate for encode/decode printing maneuvers) won't display the "pretty unicode characters" in the IDLE environment (Pythonwin prints unicode fine), For example, the neat First Order Logic symbols that Stephan Boyer uses in the output from his pedagogic prover at First Order Logic Prover.

I didn't like the idea of forcing a sys reload and I couldn't get the system to cooperate with setting environment variables like PYTHONIOENCODING (tried direct Windows environment variable and also dropping that in a in site-packages as a one liner ='utf-8').

So, if you are willing to hack your way to success, go to your IDLE directory, typically: "C:\Python27\Lib\idlelib" Locate the file Make a copy of that file and store it somewhere else so you can revert to original behavior when you choose. Open the file in the idlelib with an editor (e.g., IDLE). Go to this code area:

# Encoding for file names
filesystemencoding = sys.getfilesystemencoding()

encoding = "ascii"
if sys.platform == 'win32':
    # On Windows, we could use "mbcs". However, to give the user
    # a portable encoding name, we need to find the code page 
        # --> 6/5/17 hack to force IDLE to display utf-8 rather than cp1252
        # --> encoding = locale.getdefaultlocale()[1]
        encoding = 'utf-8'
    except LookupError:

In other words, comment out the original code line following the 'try' that was making the encoding variable equal to locale.getdefaultlocale (because that will give you cp1252 which you don't want) and instead brute force it to 'utf-8' (by adding the line 'encoding = 'utf-8' as shown).

I believe this only affects IDLE display to stdout and not the encoding used for file names etc. (that is obtained in the filesystemencoding prior). If you have a problem with any other code you run in IDLE later, just replace the file with the original unmodified file.

@Att Righ 2017-05-25 20:50:07

Here is the approach I used to produce code that was compatible with both python2 and python3 and always produced utf8 output. I found this answer elsewhere, but I can't remember the source.

This approach works by replacing sys.stdout with something that isn't quite file-like (but still only using things in the standard library). This may well cause problems for your underlying libraries, but in the simple case where you have good control over how sys.stdout out is used through your framework this can be a reasonable approach.

sys.stdout =, 'w', encoding='utf8')

@kxr 2017-02-09 20:18:00

First: reload(sys) and setting some random default encoding just regarding the need of an output terminal stream is bad practice. reload often changes things in sys which have been put in place depending on the environment - e.g. sys.stdin/stdout streams, sys.excepthook, etc.

Solving the encode problem on stdout

The best solution I know for solving the encode problem of print'ing unicode strings and beyond-ascii str's (e.g. from literals) on sys.stdout is: to take care of a sys.stdout (file-like object) which is capable and optionally tolerant regarding the needs:

  • When sys.stdout.encoding is None for some reason, or non-existing, or erroneously false or "less" than what the stdout terminal or stream really is capable of, then try to provide a correct .encoding attribute. At last by replacing sys.stdout & sys.stderr by a translating file-like object.

  • When the terminal / stream still cannot encode all occurring unicode chars, and when you don't want to break print's just because of that, you can introduce an encode-with-replace behavior in the translating file-like object.

Here an example:

#!/usr/bin/env python
# encoding: utf-8
import sys

class SmartStdout:
    def __init__(self, encoding=None, org_stdout=None):
        if org_stdout is None:
            org_stdout = getattr(sys.stdout, 'org_stdout', sys.stdout)
        self.org_stdout = org_stdout
        self.encoding = encoding or \
                        getattr(org_stdout, 'encoding', None) or 'utf-8'
    def write(self, s):
        self.org_stdout.write(s.encode(self.encoding, 'backslashreplace'))
    def __getattr__(self, name):
        return getattr(self.org_stdout, name)

if __name__ == '__main__':
    if sys.stdout.isatty():
        sys.stdout = sys.stderr = SmartStdout()

    us = u'aouäöüфżß²'
    print us

Using beyond-ascii plain string literals in Python 2 / 2 + 3 code

The only good reason to change the global default encoding (to UTF-8 only) I think is regarding an application source code decision - and not because of I/O stream encodings issues: For writing beyond-ascii string literals into code without being forced to always use u'string' style unicode escaping. This can be done rather consistently (despite what anonbadger's article says) by taking care of a Python 2 or Python 2 + 3 source code basis which uses ascii or UTF-8 plain string literals consistently - as far as those strings potentially undergo silent unicode conversion and move between modules or potentially go to stdout. For that, prefer "# encoding: utf-8" or ascii (no declaration). Change or drop libraries which still rely in a very dumb way fatally on ascii default encoding errors beyond chr #127 (which is rare today).

And do like this at application start (and/or via in addition to the SmartStdout scheme above - without using reload(sys):

def set_defaultencoding_globally(encoding='utf-8'):
    assert sys.getdefaultencoding() in ('ascii', 'mbcs', encoding)
    import imp
    _sys_org = imp.load_dynamic('_sys_org', 'sys')

if __name__ == '__main__':
    sys.stdout = sys.stderr = SmartStdout()
    s = 'aouäöüфżß²'
    print s

This way string literals and most operations (except character iteration) work comfortable without thinking about unicode conversion as if there would be Python3 only. File I/O of course always need special care regarding encodings - as it is in Python3.

Note: plains strings then are implicitely converted from utf-8 to unicode in SmartStdout before being converted to the output stream enconding.

@kiril 2016-09-16 13:25:47

Regarding python2 (and python2 only), some of the former answers rely on using the following hack:

import sys
reload(sys)  # Reload is a hack

It is discouraged to use it (check this or this)

In my case, it come with a side-effect: I'm using ipython notebooks, and once I run the code the ´print´ function no longer works. I guess there would be solution to it, but still I think using the hack should not be the correct option.

After trying many options, the one that worked for me was using the same code in the, where that piece of code is meant to be. After evaluating that module, the setdefaultencoding function is removed from sys.

So the solution is to append to file /usr/lib/python2.7/ the code:

import sys

When I use virtualenvwrapper the file I edit is ~/.virtualenvs/venv-name/lib/python2.7/

And when I use with python notebooks and conda, it is ~/anaconda2/lib/python2.7/

@ibotty 2015-06-17 07:30:07

There is an insightful blog post about it.


I paraphrase its content below.

In python 2 which was not as strongly typed regarding the encoding of strings you could perform operations on differently encoded strings, and succeed. E.g. the following would return True.

u'Toshio' == 'Toshio'

That would hold for every (normal, unprefixed) string that was encoded in sys.getdefaultencoding(), which defaulted to ascii, but not others.

The default encoding was meant to be changed system-wide in, but not somewhere else. The hacks (also presented here) to set it in user modules were just that: hacks, not the solution.

Python 3 did changed the system encoding to default to utf-8 (when LC_CTYPE is unicode-aware), but the fundamental problem was solved with the requirement to explicitly encode "byte"strings whenever they are used with unicode strings.

@iman 2014-11-21 16:33:51

If you get this error when you try to pipe/redirect output of your script

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)

Just export PYTHONIOENCODING in console and then run your code.


@Pryo 2015-02-09 10:47:50

This is the only solution that made any difference for me. - I'm on Debian 7, with broken locale settings. Thanks.

@ibotty 2015-06-17 09:40:04

Set LC_CTYPE to something sensible instead. It makes all the other programs happy as well.

@Tino 2015-09-27 23:28:06

A bigger bug in Python3 is, that PYTHONIOENCODING=utf8 is not the default. This makes scripts break just because LC_ALL=C

@Att Righ 2017-05-25 20:43:14

Set LC_CTYPE to something sensible instead This is a reasonable suggestion. This doesn't work so well when you are trying to distribute code that just works on another person's system.

@Arthur2e5 2018-03-13 19:05:22

Debian and Redhat OSes use a C.utf8 locale to provide more sensible C. glibc upstream is working on adding it, so perhaps we should not be blaming Python for respecting locale settings\…?

@bisherbas 2019-06-29 03:08:16

Thank you so much! Spent hours online and eventually came across with your response. Adding that line into .bashrc of Cygwin addresses the encoding problem with Python.

@ChristopheD 2010-02-16 20:52:49

Starting with PyDev 3.4.1, the default encoding is not being changed anymore. See this ticket for details.

For earlier versions a solution is to make sure PyDev does not run with UTF-8 as the default encoding. Under Eclipse, run dialog settings ("run configurations", if I remember correctly); you can choose the default encoding on the common tab. Change it to US-ASCII if you want to have these errors 'early' (in other words: in your PyDev environment). Also see an original blog post for this workaround.

@Sean 2011-04-30 00:40:13

Thanks Chris. Especially considering Mark T's comment above, your answer seems to be the most appropriate to me. And for somebody who's not primarily an Eclipse/PyDev user, I never would have figured that out on my own.

@Tim Diggins 2012-02-22 11:58:11

I'd like to change this globally (rather than once per run configuration), but haven't figured out how - have asked a separate q:…

@lukmdo 2011-10-25 16:55:23

A) To control sys.getdefaultencoding() output:

python -c 'import sys; print(sys.getdefaultencoding())'



echo "import sys; sys.setdefaultencoding('utf-16-be')" >


PYTHONPATH=".:$PYTHONPATH" python -c 'import sys; print(sys.getdefaultencoding())'


You could put your higher in your PYTHONPATH.

Also you might like to try reload(sys).setdefaultencoding by @EOL

B) To control stdin.encoding and stdout.encoding you want to set PYTHONIOENCODING:

python -c 'import sys; print(sys.stdin.encoding, sys.stdout.encoding)'

ascii ascii


PYTHONIOENCODING="utf-16-be" python -c 'import sys; 
print(sys.stdin.encoding, sys.stdout.encoding)'

utf-16-be utf-16-be

Finally: you can use A) or B) or both!

@lukmdo 2015-02-04 00:34:16

(python2 only) separate but interesting is extending above with from __future__ import unicode_literals see discussion

Related Questions

Sponsored Content

26 Answered Questions

[SOLVED] Does Python have a ternary conditional operator?

44 Answered Questions

22 Answered Questions

[SOLVED] What are metaclasses in Python?

42 Answered Questions

[SOLVED] How to get the current time in Python

  • 2009-01-06 04:54:23
  • user46646
  • 3367437 View
  • 2967 Score
  • 42 Answer
  • Tags:   python datetime time

7 Answered Questions

[SOLVED] Why does modern Perl avoid UTF-8 by default?

  • 2011-05-28 15:12:36
  • w.k
  • 101220 View
  • 562 Score
  • 7 Answer
  • Tags:   perl unicode utf-8

61 Answered Questions

[SOLVED] Calling an external command from Python

10 Answered Questions

[SOLVED] Does Python have a string 'contains' substring method?

31 Answered Questions

[SOLVED] "Least Astonishment" and the Mutable Default Argument

10 Answered Questions

[SOLVED] Setting the correct encoding when piping stdout in Python

Sponsored Content