Python 3.13: symlink loop handling broken? (tests/test_web_urldispatcher.py::test_access_symlink_loop failure)

[mgorny]

Describe the bug

1. In 3.10.0, the tests/test_web_urldispatcher.py::test_access_symlink_loop test is failing on Python 3.13.0b4. The backtrace suggests it's not a problem with the test itself but with handling symlink loops.

To Reproduce

# using pypi sdist is easier
pip install aiohttp pytest-cov yarl
python -m pytest tests/test_web_urldispatcher.py::test_access_symlink_loop

Expected behavior

Tests passing :-).

Logs/tracebacks

========================================================= test session starts =========================================================
platform linux -- Python 3.13.0b4, pytest-8.3.2, pluggy-1.5.0 -- /tmp/aiohttp/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/aiohttp
configfile: setup.cfg
plugins: cov-5.0.0
collected 1 item                                                                                                                      

tests/test_web_urldispatcher.py::test_access_symlink_loop[pyloop] FAILED                                                        [100%]

============================================================== FAILURES ===============================================================
__________________________________________________ test_access_symlink_loop[pyloop] ___________________________________________________

tmp_path = PosixPath('/tmp/pytest-of-mgorny/pytest-0/test_access_symlink_loop_pyloo0')
aiohttp_client = <function aiohttp_client.<locals>.go at 0x7f711a30d760>

    async def test_access_symlink_loop(
        tmp_path: pathlib.Path, aiohttp_client: AiohttpClient
    ) -> None:
        # Tests the access to a looped symlink, which could not be resolved.
        my_dir_path = tmp_path / "my_symlink"
        pathlib.Path(str(my_dir_path)).symlink_to(str(my_dir_path), True)
    
        app = web.Application()
    
        # Register global static route:
        app.router.add_static("/", str(tmp_path), show_index=True)
        client = await aiohttp_client(app)
    
        # Request the root of the static directory.
>       r = await client.get("/" + my_dir_path.name)

aiohttp_client = <function aiohttp_client.<locals>.go at 0x7f711a30d760>
app        = <Application 0x7f711a6baae0>
client     = <aiohttp.test_utils.TestClient object at 0x7f711a6d1fd0>
my_dir_path = PosixPath('/tmp/pytest-of-mgorny/pytest-0/test_access_symlink_loop_pyloo0/my_symlink')
tmp_path   = PosixPath('/tmp/pytest-of-mgorny/pytest-0/test_access_symlink_loop_pyloo0')

tests/test_web_urldispatcher.py:513: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
aiohttp/test_utils.py:309: in _request
    resp = await self._session.request(method, self.make_url(path), **kwargs)
        kwargs     = {}
        method     = 'GET'
        path       = '/my_symlink'
        self       = <aiohttp.test_utils.TestClient object at 0x7f711a6d1fd0>
aiohttp/client.py:616: in _request
    await resp.start(conn)
        all_cookies = <SimpleCookie: >
        allow_redirects = True
        auth       = None
        auth_from_url = None
        auto_decompress = True
        chunked    = None
        compress   = None
        conn       = Connection<ConnectionKey(host='127.0.0.1', port=36269, is_ssl=False, ssl=True, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
        cookies    = None
        data       = None
        expect100  = False
        handle     = None
        headers    = <CIMultiDict()>
        history    = []
        json       = None
        max_field_size = 8190
        max_line_size = 8190
        max_redirects = 10
        method     = 'GET'
        params     = {}
        proxy      = None
        proxy_auth = None
        proxy_headers = <CIMultiDict()>
        raise_for_status = None
        read_bufsize = 65536
        read_until_eof = True
        real_timeout = ClientTimeout(total=300, connect=None, sock_read=None, sock_connect=None, ceil_threshold=5)
        redirects  = 0
        req        = <aiohttp.client_reqrep.ClientRequest object at 0x7f711a6c2c10>
        resp       = <ClientResponse(http://127.0.0.1:36269/my_symlink) [None None]>
None

        retry_persistent_connection = False
        self       = <aiohttp.client.ClientSession object at 0x7f711a3f7140>
        server_hostname = None
        skip_auto_headers = None
        skip_headers = set()
        ssl        = True
        str_or_url = URL('http://127.0.0.1:36269/my_symlink')
        timeout    = <_SENTINEL.sentinel: 1>
        timer      = <aiohttp.helpers.TimerContext object at 0x7f711a6d2e40>
        tm         = <aiohttp.helpers.TimeoutHandle object at 0x7f711a6d27b0>
        trace_request_ctx = None
        traces     = []
        url        = URL('http://127.0.0.1:36269/my_symlink')
        version    = HttpVersion(major=1, minor=1)
aiohttp/client_reqrep.py:926: in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
        connection = Connection<ConnectionKey(host='127.0.0.1', port=36269, is_ssl=False, ssl=True, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
        protocol   = <aiohttp.client_proto.ResponseHandler object at 0x7f711a340ef0>
        self       = <ClientResponse(http://127.0.0.1:36269/my_symlink) [None None]>
None

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <aiohttp.client_proto.ResponseHandler object at 0x7f711a340ef0>

    async def read(self) -> _SizedT:
        if not self._buffer and not self._eof:
            assert not self._waiter
            self._waiter = self._loop.create_future()
            try:
>               await self._waiter
E               aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected

self       = <aiohttp.client_proto.ResponseHandler object at 0x7f711a340ef0>

aiohttp/streams.py:626: ServerDisconnectedError
---------------------------------------------------------- Captured log call ----------------------------------------------------------
ERROR    aiohttp.server:web_protocol.py:442 Unhandled exception
Traceback (most recent call last):
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 545, in start
    resp, reset = await task
                  ^^^^^^^^^^
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 491, in _handle_request
    reset = await self.finish_response(request, resp, start_time)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 647, in finish_response
    await prepare_meth(request)
  File "/tmp/aiohttp/aiohttp/web_fileresponse.py", line 188, in prepare
    file_path, st, file_encoding = await loop.run_in_executor(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, self._get_file_path_stat_encoding, accept_encoding
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/tmp/aiohttp/aiohttp/web_fileresponse.py", line 180, in _get_file_path_stat_encoding
    return file_path, file_path.stat(), None
                      ~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/pathlib/_local.py", line 515, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 40] Too many levels of symbolic links: '/tmp/pytest-of-mgorny/pytest-0/test_access_symlink_loop_pyloo0/my_symlink'
ERROR    aiohttp.server:web_protocol.py:442 Unhandled exception
Traceback (most recent call last):
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 545, in start
    resp, reset = await task
                  ^^^^^^^^^^
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 491, in _handle_request
    reset = await self.finish_response(request, resp, start_time)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiohttp/aiohttp/web_protocol.py", line 647, in finish_response
    await prepare_meth(request)
  File "/tmp/aiohttp/aiohttp/web_fileresponse.py", line 188, in prepare
    file_path, st, file_encoding = await loop.run_in_executor(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, self._get_file_path_stat_encoding, accept_encoding
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib/python3.13/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/tmp/aiohttp/aiohttp/web_fileresponse.py", line 180, in _get_file_path_stat_encoding
    return file_path, file_path.stat(), None
                      ~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/pathlib/_local.py", line 515, in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
           ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 40] Too many levels of symbolic links: '/tmp/pytest-of-mgorny/pytest-0/test_access_symlink_loop_pyloo0/my_symlink'

----------- coverage: platform linux, python 3.13.0-beta-4 -----------
Name                                     Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------------------
aiohttp/__init__.py                         26     10      2      0    57%
aiohttp/abc.py                              97      2     66      0    99%
aiohttp/base_protocol.py                    67     36     24      3    40%
aiohttp/client.py                          530    251    234     43    47%
aiohttp/client_exceptions.py               138     55     36      1    61%
aiohttp/client_proto.py                    170     80     68     11    46%
aiohttp/client_reqrep.py                   657    299    328     59    48%
aiohttp/client_ws.py                       220    172     84      1    20%
aiohttp/compression_utils.py                70     37     22      0    42%
aiohttp/connector.py                       700    407    329     45    36%
aiohttp/cookiejar.py                       254    187    128      3    19%
aiohttp/formdata.py                         86     69     40      0    15%
aiohttp/hdrs.py                             90      0      0      0   100%
aiohttp/helpers.py                         558    299    231     21    39%
aiohttp/http.py                              8      0      0      0   100%
aiohttp/http_exceptions.py                  50     20      4      0    56%
aiohttp/http_parser.py                     490    273    208     33    37%
aiohttp/http_websocket.py                  381    288    152      0    18%
aiohttp/http_writer.py                     118     40     54     14    57%
aiohttp/locks.py                            24     16      4      0    29%
aiohttp/log.py                               7      0      0      0   100%
aiohttp/multipart.py                       608    501    272      0    14%
aiohttp/payload.py                         220    108     74      1    47%
aiohttp/pytest_plugin.py                   159     65     70      8    59%
aiohttp/resolver.py                         63     42     18      0    26%
aiohttp/streams.py                         395    292    140      2    20%
aiohttp/tcp_helpers.py                      19      2      8      3    81%
aiohttp/test_utils.py                      301    117     78     15    58%
aiohttp/tracing.py                         191     69     64      0    73%
aiohttp/typedefs.py                         23      0      0      0   100%
aiohttp/web.py                             121     84     52      0    23%
aiohttp/web_app.py                         247     84     93     19    63%
aiohttp/web_exceptions.py                  214     37     42      6    80%
aiohttp/web_fileresponse.py                161    109     52      2    29%
aiohttp/web_log.py                         102     39     40      0    64%
aiohttp/web_middlewares.py                  54     38     22      0    21%
aiohttp/web_protocol.py                    355    166    157     26    43%
aiohttp/web_request.py                     450    267    210      7    41%
aiohttp/web_response.py                    439    309    240      2    27%
aiohttp/web_routedef.py                    104     44     18      2    57%
aiohttp/web_runner.py                      221     71     70     12    67%
aiohttp/web_server.py                       45      8     14      4    80%
aiohttp/web_urldispatcher.py               723    354    235     20    50%
aiohttp/web_ws.py                          341    273    130      1    17%
aiohttp/worker.py                          123    123     32      0     0%
tests/conftest.py                          130     74     59      0    48%
tests/test_base_protocol.py                198    198      8      0     0%
tests/test_circular_imports.py              29     29     13      0     0%
tests/test_classbasedview.py                39     39      4      0     0%
tests/test_client_connection.py             93     93     12      0     0%
tests/test_client_exceptions.py            180    180     14      0     0%
tests/test_client_fingerprint.py            24     24      4      0     0%
tests/test_client_functional.py           2399   2399    456      0     0%
tests/test_client_proto.py                 100    100      0      0     0%
tests/test_client_request.py               759    759    108      0     0%
tests/test_client_response.py              461    461     36      0     0%
tests/test_client_session.py               492    492     97      0     0%
tests/test_client_ws.py                    468    468    154      0     0%
tests/test_client_ws_functional.py         673    673     54      0     0%
tests/test_connector.py                   1653   1653    218      0     0%
tests/test_cookiejar.py                    358    358     52      0     0%
tests/test_flowcontrol_streams.py          101    101      6      0     0%
tests/test_formdata.py                      77     77     24      0     0%
tests/test_helpers.py                      541    541    162      0     0%
tests/test_http_exceptions.py              109    109     10      0     0%
tests/test_http_parser.py                 1053   1053    208      0     0%
tests/test_http_writer.py                  183    183     14      0     0%
tests/test_imports.py                       34     34     12      0     0%
tests/test_locks.py                         40     40      4      0     0%
tests/test_loop.py                          36     36      4      0     0%
tests/test_multipart.py                    759    759    220      0     0%
tests/test_multipart_helpers.py            446    446     82      0     0%
tests/test_payload.py                       77     77      8      0     0%
tests/test_proxy.py                        299    299     92      0     0%
tests/test_proxy_functional.py             456    456    106      0     0%
tests/test_pytest_plugin.py                 47     47      2      0     0%
tests/test_resolver.py                     179    179     52      0     0%
tests/test_route_def.py                    211    211     26      0     0%
tests/test_run_app.py                      554    554    100      0     0%
tests/test_streams.py                     1058   1058    128      0     0%
tests/test_tcp_helpers.py                   52     52     14      0     0%
tests/test_test_utils.py                   230    230     54      0     0%
tests/test_tracing.py                       49     49      2      0     0%
tests/test_urldispatch.py                  859    859     98      0     0%
tests/test_web_app.py                      381    381     30      0     0%
tests/test_web_cli.py                       76     76     20      0     0%
tests/test_web_exceptions.py               274    274     42      0     0%
tests/test_web_functional.py              1487   1487    152      0     0%
tests/test_web_log.py                      143    143     12      0     0%
tests/test_web_middleware.py               230    230     26      0     0%
tests/test_web_request.py                  543    543     40      0     0%
tests/test_web_request_handler.py           42     42      2      0     0%
tests/test_web_response.py                 813    813     78      0     0%
tests/test_web_runner.py                   175    175     36      0     0%
tests/test_web_sendfile.py                  85     85      0      0     0%
tests/test_web_sendfile_functional.py      656    656     70      0     0%
tests/test_web_server.py                   178    178     18      0     0%
tests/test_web_urldispatcher.py            468    406     78      0    17%
tests/test_web_websocket.py                383    383     68      0     0%
tests/test_web_websocket_functional.py     682    682     34      0     0%
tests/test_websocket_handshake.py          153    153     26      0     0%
tests/test_websocket_parser.py             293    293     44      0     0%
tests/test_websocket_writer.py              95     95     16      0     0%
tests/test_worker.py                       190    190     20      0     0%
--------------------------------------------------------------------------
TOTAL                                    33273  28478   7674    364    15%

======================================================== slowest 10 durations =========================================================
0.02s call     tests/test_web_urldispatcher.py::test_access_symlink_loop[pyloop]
0.01s teardown tests/test_web_urldispatcher.py::test_access_symlink_loop[pyloop]

(1 durations < 0.005s hidden.  Use -vv to show these durations.)
======================================================= short test summary info =======================================================
FAILED tests/test_web_urldispatcher.py::test_access_symlink_loop[pyloop] - aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
========================================================== 1 failed in 3.48s ==========================================================

Python Version

$ python --version
Python 3.13.0b4

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.10.0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: 
Author-email: 
License: Apache 2
Location: /tmp/aiohttp/.venv/lib/python3.13/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, yarl
Required-by:

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.0.5
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /tmp/aiohttp/.venv/lib/python3.13/site-packages
Requires: 
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.9.4
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /tmp/aiohttp/.venv/lib/python3.13/site-packages
Requires: idna, multidict
Required-by: aiohttp

OS

Gentoo Linux amd64

Related component

Server

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[bdraco]

Thanks

[steverep]

Looks like they changed the exception for circular symlinks from RuntimeError to OSError in 3.13 for Path.resolve():

Changed in version 3.13: Symlink loops are treated like other errors: OSError is raised in strict mode, and no exception is raised in non-strict mode. In previous versions, RuntimeError is raised no matter the value of strict.

The fix is easy, but I'm inclined to let this be for now since it's still beta and we cannot verify in CI yet that the exception will be consistent across OS.

[steverep]

Actually I take that back, since we catch it now without strict mode, this would have to now be caught another way which is not straightforward.

This is a really annoying change in pathlib to work around.

[bdraco]

Moving to 3.10.2 since 3.13 isn't released yet and we don't have the ability to test it in the CI yet

[steverep]

@mgorny would you be able to test #8642?