[Python][CI] AppVeyor CI is failing on test_fs.py::test_filesystem_is_functional_after_pickling

[AlenkaF]

Describe the bug, including details regarding any error messages, version, and platform.

As seen on a PR:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48347810

Currently our Appveyor CI seems to be failing on python/pyarrow/tests/test_fs.py::test_filesystem_is_functional_after_pickling with:

_ test_filesystem_is_functional_after_pickling[builtin_pickle-PyFileSystem(FSSpecHandler(s3fs.S3FileSystem()))] _
fs = <pyarrow._fs.PyFileSystem object at 0x000001753AB82860>
pathfn = <method-wrapper '__add__' of str object at 0x0000017539D21FC0>
pickle_module = <module 'pickle' from 'C:\\Miniconda38-x64\\envs\\arrow\\lib\\pickle.py'>
    def test_filesystem_is_functional_after_pickling(fs, pathfn, pickle_module):
        if fs.type_name.split('::')[-1] == 'mock':
            pytest.xfail(reason='MockFileSystem is not serializable')
        skip_fsspec_s3fs(fs)
    
        aaa = pathfn('a/aa/aaa/')
        bb = pathfn('a/bb')
        c = pathfn('c.txt')
    
        fs.create_dir(aaa)
        with fs.open_output_stream(bb):
            pass  # touch
        with fs.open_output_stream(c) as fp:
            fp.write(b'test')
    
        restored = pickle_module.loads(pickle_module.dumps(fs))
        aaa_info, bb_info, c_info = restored.get_file_info([aaa, bb, c])
>       assert aaa_info.type == FileType.Directory
E       AssertionError: assert <FileType.NotFound: 0> == <FileType.Directory: 3>
E        +  where <FileType.NotFound: 0> = <FileInfo for 'pyarrow-filesystem/a/aa/aaa/': type=FileType.NotFound>.type
E        +  and   <FileType.Directory: 3> = FileType.Directory
pyarrow\tests\test_fs.py:588: AssertionError
.
.
.

Component(s)

Continuous Integration

[AlenkaF]

The version of s3fs in the failing builds is 2023.10.0, in the last passing AppVeyor build the version has been 2023.9.2. Will look at the release notes and open issues in s3fs.

[AlenkaF]

I think the failures are due to a type_name change connected to https://github.com/fsspec/s3fs/pull/812/files. Because the type_name changed from "py::fsspec+s3" to "py::fsspec+('s3', 's3a')" the skip doesn't work:

arrow/python/pyarrow/tests/test_fs.py

Lines 465 to 467 in 25e5d19

	def skip_fsspec_s3fs(fs):
	if fs.type_name == "py::fsspec+s3":
	pytest.xfail(reason="Not working with fsspec's s3fs")

Will add a PR and see if this is it.

[bdice]

Thanks for reporting @AlenkaF. I just ran into the same issue here while backporting a patch to PyArrow 13.0.0 on conda-forge: conda-forge/arrow-cpp-feedstock#1211 (comment)