Flakey Test in CI - Unclosed File Handle

[alimcmaster1]

Seen this in a few pipelines today:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=20521
https://github.com/pandas-dev/pandas/runs/295247988

______________ test_chunks_have_consistent_numerical_type[python] ______________
[gw0] linux -- Python 3.6.1 /home/vsts/miniconda3/envs/pandas-dev/bin/python

all_parsers = <pandas.tests.io.parser.conftest.PythonParser object at 0x7f635d65e470>

    def test_chunks_have_consistent_numerical_type(all_parsers):
        parser = all_parsers
        integers = [str(i) for i in range(499999)]
        data = "a\n" + "\n".join(integers + ["1.0", "2.0"] + integers)
    
        # Coercions should work without warnings.
        with tm.assert_produces_warning(None):
>           result = parser.read_csv(StringIO(data))

pandas/tests/io/parser/test_common.py:1204: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <contextlib._GeneratorContextManager object at 0x7f631f91c8d0>
type = None, value = None, traceback = None

    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed file <_io.BufferedReader name='test1.xlsx'>",), 

[WillAyd]

Pretty interesting the warnings are only for .xlsx and .xlsm files, so maybe something with the openpyxl reader; would think .xls would show as well if it were xlrd

[dongho-jung]

I'm suffering from this too #28681 . what can I do to help? I want to help... but I don't know where to start. It would be nice if I can help something.

[alimcmaster1]

https://travis-ci.org/pandas-dev/pandas/jobs/631970657?utm_medium=notification&utm_source=github_status Looks like this test is still an issue

[alimcmaster1]

Also see test_legacy_sparse_warning causing issues here

[jorisvandenbossche]

Is this a blocker for 1.0 ?

[TomAugspurger]

I don't think so.

[mroeschke]

Don't think we've seen this in a while. Closable @alimcmaster1?

[alimcmaster1]

Yeah can't see this test in any recent build failures - do see a different done though...

_________________ TestNestedToRecord.test_deprecated_import ___________________
[gw0] linux -- Python 3.6.10 /home/travis/miniconda3/envs/pandas-dev/bin/python
self = <pandas.tests.io.json.test_normalize.TestNestedToRecord object at 0x7f329239d668>
    def test_deprecated_import(self):
        with tm.assert_produces_warning(FutureWarning):
            from pandas.io.json import json_normalize
    
            recs = [{"a": 1, "b": 2, "c": 3}, {"a": 4, "b": 5, "c": 6}]
>           json_normalize(recs)
pandas/tests/io/json/test_normalize.py:742: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <contextlib._GeneratorContextManager object at 0x7f329239dac8>
type = None, value = None, traceback = None
    def __exit__(self, type, value, traceback):
        if type is None:
            try:
>               next(self.gen)
E               AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed <socket.socket fd=33, family=AddressFamily.AF_INET, 

https://travis-ci.org/github/pandas-dev/pandas/jobs/677027259

Will try take a look

[TomAugspurger]

Here's the smallest reproducer I could get for the unhanlded ResourceWarning

pytest pandas/tests/io/json/test_pandas.py::TestPandasContainer::test_read_s3_jsonl pandas/tests/io/json/test_pandas.py::TestPandasContainer::test_deprecate_numpy_argument_read_json

No idea why those two are interacting like that.

[smithto1]

Just reporting that I've had this error three times today on Travis.

https://travis-ci.org/github/pandas-dev/pandas/jobs/708801493

FAILED pandas/tests/io/parser/test_common.py::test_chunks_have_consistent_numerical_type[python