[Reportbug-maint] Bug#739507: UnicodeEncodeError: 'ascii' codec can't encode character

Sat Apr 18 13:33:52 UTC 2015

Package: reportbug
Version: 6.6.3
Followup-For: Bug #739507

Dear Maintainer,
this issue manifested on a fresh Jessie installation and
therefore I tried to find out something about it.

To me it looks like on affected systems no one sets the
default encoding in this python process.
At least my system where it does not appear it is set.

Therefore I collected now 2 workarounds:
- install package python-gtk2 (which sets default encoding on initialization,
    initpango () from /usr/lib/python2.7/dist-packages/gtk-2.0/pango.so)
- add to /usr/bin/reportbug after the "import sys" these 2 lines:
    reload(sys)
    sys.setdefaultencoding("utf-8")

Kind regards,
Bernhard

(Attached some lines about the debugging.)

-- Package-specific info:
** Environment settings:
INTERFACE="text"

** /home/benutzer/.reportbugrc:
reportbug_version "6.6.3"
mode standard
ui text
realname "Bernhard Übelacker"
email "bernhardu at vr-web.de"
no-cc
header "X-Debbugs-CC: bernhardu at vr-web.de"
smtphost reportbug.debian.org

-- System Information:
Debian Release: 8.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/1 CPU core)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages reportbug depends on:
ii  apt               1.0.9.7
ii  python            2.7.9-1
ii  python-reportbug  6.6.3
pn  python:any        <none>

reportbug recommends no packages.

Versions of packages reportbug suggests:
pn  claws-mail                                 <none>
pn  debconf-utils                              <none>
pn  debsums                                    <none>
pn  dlocate                                    <none>
pn  emacs23-bin-common | emacs24-bin-common    <none>
ii  exim4                                      4.84-8
ii  exim4-daemon-light [mail-transport-agent]  4.84-8
ii  file                                       1:5.22+15-2
ii  gnupg                                      1.4.18-7
pn  python-gtk2                                <none>
pn  python-gtkspell                            <none>
pn  python-urwid                               <none>
pn  python-vte                                 <none>
pn  xdg-utils                                  <none>

Versions of packages python-reportbug depends on:
ii  apt               1.0.9.7
ii  python-debian     0.1.25
ii  python-debianbts  1.12
pn  python:any        <none>

python-reportbug suggests no packages.

-- no debconf information
-------------- next part --------------

apt-get source python2.7
cd python2.7-2.7.9/Objects

gdb --args /usr/bin/python pdb /usr/bin/reportbug
(gdb) run
(Pdb) b /usr/bin/reportbug:578
(Pdb) cont
    # answer the questions, enter a realname with non-ascii character
> /usr/bin/reportbug(578)offer_configuration()
-> realname = realname.decode(charset, 'replace')
(Pdb) 
    # CTRL+c
(gdb) b PyUnicodeEncodeError_Create
(gdb) cont
cont
Breakpoint 1, PyUnicodeEncodeError_Create (encoding=0x60a268 "ascii", object=0x7ffff3e5bad0, length=18, start=9, end=10, reason=0x625c9c "ordinal not in range(128)") at ../Objects/exceptions.c:1713
1713    {
(gdb) bt
#0  PyUnicodeEncodeError_Create (encoding=0x60a268 "ascii", object=0x7ffff3e5bad0, length=18, start=9, end=10, reason=0x625c9c "ordinal not in range(128)") at ../Objects/exceptions.c:1713
#1  0x0000000000554709 in make_encode_exception () at ../Objects/unicodeobject.c:3522
#2  0x00000000005554fe in raise_encode_exception.lto_priv () at ../Objects/unicodeobject.c:3545
#3  0x00000000005523cc in unicode_encode_ucs1.lto_priv () at ../Objects/unicodeobject.c:3677
#4  0x000000000058da8d in _PyUnicodeUCS4_AsDefaultEncodedString (unicode=unicode at entry=u'Bernhard \xdcbelacker', errors=errors at entry=0x0) at ../Objects/unicodeobject.c:1391
#5  0x00000000004b5fd0 in convertsimple.lto_priv () at ../Python/getargs.c:843
#6  0x00000000004c03de in convertitem (freelist=<optimized out>, bufsize=<optimized out>, msgbuf=<optimized out>, levels=<optimized out>, flags=<optimized out>, p_va=<optimized out>, p_format=<optimized out>, arg=<optimized out>) at ../Python/getargs.c:514
#7  vgetargs1.lto_priv () at ../Python/getargs.c:345
#8  0x00000000004e31e9 in _PyArg_ParseTuple_SizeT (args=<optimized out>, format=format at entry=0x62c91b "s*|zi:utf_8_decode") at ../Python/getargs.c:97
#9  0x0000000000589f29 in utf_8_decode.lto_priv () at ../Modules/_codecsmodule.c:265
#10 0x00000000004cd9ca in call_function (oparg=<optimized out>, pp_stack=<optimized out>) at ../Python/ceval.c:4033
#11 PyEval_EvalFrameEx () at ../Python/ceval.c:2679
#12 0x00000000004e5fe8 in PyEval_EvalCodeEx (closure=<optimized out>, defcount=<optimized out>, defs=<optimized out>, kwcount=<optimized out>, kws=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=<optimized out>, g
....
#59 0x0000000000497ca0 in _start ()

####################

The first difference between working and not working was in ../Objects/unicodeobject.c:1355
1355            if (strcmp(encoding, "utf-8") == 0)

After looking where default encoding is set on a working machine, one could
find out it is done via pango via gtk. So I tried to install the package containing pango.so.

gdb --args /usr/bin/python /usr/bin/reportbug
(gdb) b ../Objects/unicodeobject.c:1428
(gdb) run
Breakpoint 1, PyUnicodeUCS4_SetDefaultEncoding (encoding=0x7fffecb648c5 "utf-8") at ../Objects/unicodeobject.c:1432
1432        v = _PyCodec_Lookup(encoding);
(gdb) bt
#0  PyUnicodeUCS4_SetDefaultEncoding (encoding=0x7fffecb648c5 "utf-8") at ../Objects/unicodeobject.c:1432
#1  0x00007fffecb577f1 in initpango () from /usr/lib/python2.7/dist-packages/gtk-2.0/pango.so
#2  0x00000000005411f9 in _PyImport_LoadDynamicModule () at ../Python/importdl.c:53
....
#13 0x00007ffff2ab237b in ?? () from /usr/lib/python2.7/dist-packages/gtk-2.0/gtk/_gtk.so
#14 0x00007ffff29c8b26 in init_gtk () from /usr/lib/python2.7/dist-packages/gtk-2.0/gtk/_gtk.so
#15 0x00000000005411f9 in _PyImport_LoadDynamicModule () at ../Python/importdl.c:53
....
#81 0x0000000000497ca0 in _start ()