[Python-modules-team] Bug#951696: certbot: Incompatibility with python3-urllib3=1.25.8-1

[Daniele Tricoli]

Hello Harlan,
thanks for forwarding this!

On Fri, Mar 27, 2020 at 11:43:36AM -0400, Harlan Lieberman-Berg wrote:
> reassign 951696 requests
> retitle 951696 requests: incompatibility between requests == 2.22.0-2
> and urllib3 == 1.25.8-1
> tag 951696 +unreproducible
> affects 951696 certbot python3-urllib3
> thanks
> 
> I've looked over this several times over the last few months, and I
> can't reproduce the problem on any of my test systems.  However,
> looking at the code, if this bug exists, it's definitely between
> requests and urllib3, not in certbot itself.
> 
> Sending it over to the requests folks, in the hopes that they've seen
> this somewhere else and can reproduce it.

Unfortunately I never seen something like this but I will investigate. 
I would exclude requests involvement for now, it seems more on urllib3 side,
but let's see (anyway no need to move this now, since I maintain also urllib3).
I strictly follow upstream for compatibility versions, so I'm more
inclined to ithink to an urllib3 bug.

I looked at the differences of urllib3.util.url.parse_url in 1.25.8-1 and
1.24.1-1: upstream moved to a more "ask forgiveness" approach, and we can see
a try/except block from line 360 to line 391.
The except is:

    except (ValueError, AttributeError):
        return six.raise_from(LocationParseError(source_url), None)

So although Python 3 can chain exceptions since None is passed to
six.raise_from, every exceptions of kind ValueError or AttributeError are
masked as the LocationParseError we see in the traceback that Alexandre posted.
This can hide a possible root cause and I would look at this first.

@Alexandre since it seems tricky to reproduce, meanwhile I setup a test
environment, please can you try to unmask the exception above?
You need only to edit the lines above (lines 391 and 392 of 
/usr/lib/python3/dist-packages/urllib3/util/url.py) into:

    except (ValueError, AttributeError) as exp:
        return six.raise_from(LocationParseError(source_url), exp)

This way you should get also the traceback of another exception that is
eventually raised.
Or you can just rethrow `exp`: choose the way you prefer.

From the exception we get now it's not possible to understand what is
happening. The source_url mentioned in the first report is parsed fine in my
system with urllib3 1.25.8-1:

❯ python3 -c "from urllib3.util.url import parse_url; print(repr(parse_url('https://acme-v02.api.letsencrypt.org/directory')))"
Url(scheme='https', auth=None, host='acme-v02.api.letsencrypt.org', port=None, path='/directory', query=None, fragment=None)

So maybe there is something hidden somewhere.

Regards,

-- 
  Daniele Tricoli 'eriol'
  https://mornie.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/python-modules-team/attachments/20200328/8e39d806/attachment-0001.sig>