urllib.parse.urlunsplit makes relative path to absolute (http:g -> http:///g)

[openandclose]

BPO	40938
Nosy	@orsenthil, @openandclose, @jaswdr

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-06-10.11:18:48.328>
labels = ['3.7', '3.8', 'type-bug', 'library', '3.9']
title = 'urllib.parse.urlunsplit makes relative path to absolute (http:g -> http:///g)'
updated_at = <Date 2021-05-13.14:11:56.189>
user = 'https://github.com/openandclose'

bugs.python.org fields:

activity = <Date 2021-05-13.14:11:56.189>
actor = 'op368'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-06-10.11:18:48.328>
creator = 'op368'
dependencies = []
files = []
hgrepos = []
issue_num = 40938
keywords = []
message_count = 7.0
messages = ['371179', '393544', '393574', '393576', '393577', '393578', '393583']
nosy_count = 3.0
nosy_names = ['orsenthil', 'op368', 'jaswdr']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40938'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

Linked PRs

• gh-85110: Preserve relative path in URL without netloc in urllib.parse.urlunsplit() #123179
• [3.13] gh-85110: Preserve relative path in URL without netloc in urllib.parse.urlunsplit() (GH-123179) #123187
• [3.12] gh-85110: Preserve relative path in URL without netloc in urllib.parse.urlunsplit() (GH-123179) #123188

[openandclose]

path 'g' in 'http:g' becomes '/g'.

    >>> urlsplit('http:g')
    SplitResult(scheme='http', netloc='', path='g', query='', fragment='')
    >>> urlunsplit(urlsplit('http:g'))
    'http:///g'
    >>> urlsplit('http:///g')
    SplitResult(scheme='http', netloc='', path='/g', query='', fragment='')

    >>> urljoin('http://a/b/c/d', 'http:g')
    'http://a/b/c/g'
    >>> urljoin('http://a/b/c/d', 'http:///g')
    'http://a/g'

The problematic part of the code is:

    def urlunsplit(components):
        [...]
        if netloc or (scheme and scheme in uses_netloc and url[:2] != '//'):
--->        if url and url[:1] != '/': url = '/' + url
            url = '//' + (netloc or '') + url

Note also that urllib has decided on the interpretation of 'http:g' (in test).

    def test_RFC3986(self):
        [...]
        #self.checkJoin(RFC3986_BASE, 'http:g','http:g') # strict parser
        self.checkJoin(RFC3986_BASE, 'http:g','http://a/b/c/g') #relaxed parser

[jaswdr]

@op368 I don't think that this is a bug, [1] literally uses this exact example and shows the expected behaviour.

[1] https://datatracker.ietf.org/doc/html/rfc3986#section-5.4.2

[openandclose]

hello, @jaswdr, but I can't understand what's wrong with my point.
What is 'the expected behaviour'?

[jaswdr]

@op368 as far as I can see, regarding of any miss interpretation, yes, the RFC has this section:

  "http:g"        =  "http:g"         ; for strict parsers
                  /  "http://a/b/c/g" ; for backward compatibility

What I can understand is that for "http:g" it will be translated to "http:///g" because of backward compatibility, this seems to be an edge case for the parser, since the RFC text also mention that this should be avoided.

[openandclose]

'http:///g' has absolute path '/g',
and as urljoin shows:

    >>> urljoin('http://a/b/c/d', 'http:///g')
    'http://a/g'  # 'a' is netloc

So you are proposing third interpretation.

  "http:g"        =  "http:g"         ; for strict parsers
                  /  "http://a/b/c/g" ; for backward compatibility
                  /  "http://a/g"     ; (yours)

[jaswdr]

Not exactly, in the RFC example they use a/b/c for the path, but when using http:g there is no nested path, so it should be http:///g, no?

[openandclose]

I tried hard (even read RFC1630),
but I think no.

[serhiy-storchaka]

The problem is that urljoin(base, urlunsplit(urlsplit(relurl))) is not equivalent to urljoin(base, relurl). I think that the problem cannot be solved without large breaking changes (like #67041), but this particular case can be mitigated. urlunsplit) can be made preserving non-empty relative path with non-empty scheme and empty netloc.

An exception for empty path was added to not break an existing test for #104139. All other existing tests are passed with this change.