flake8 fails if sys.stdout is an io.StringIO object

[eerovaher]

Please describe how you installed Flake8

$ pip install flake8

The exact, unmodified output of `flake8 --bug-report` (click this line to show)

{
"dependencies": [],
"platform": {
"python_implementation": "CPython",
"python_version": "3.8.10",
"system": "Linux"
},
"plugins": [
{
"is_local": false,
"plugin": "mccabe",
"version": "0.6.1"
},
{
"is_local": false,
"plugin": "pycodestyle",
"version": "2.8.0"
},
{
"is_local": false,
"plugin": "pyflakes",
"version": "2.4.0"
}
],
"version": "4.0.1"
}

Please describe the problem or feature

There seems to be a problem with this line:

flake8/src/flake8/formatting/base.py

Line 187 in ecee230

sys.stdout.buffer.write(output.encode() + self.newline.encode())

The Python documentation (https://docs.python.org/3.8/library/sys.html#sys.stdout) warns that sys.stdout might end up being an io.StringIO object, which does not have a buffer attribute, and I have found a case where this happens in practice.

If this is a bug report, please explain with examples (and example code) what you expected to happen and what actually happened.

I wanted to use pycodestyle_magic together with flake8 in a Jupyter notebook. So the first code cell is:

%load_ext pycodestyle_magic
%flake8_on

and the second:

print( 0)

The result is:

AttributeError: '_io.StringIO' object has no attribute 'buffer'

I replaced the previously mentioned line 187 with the following four lines:

            try:
                sys.stdout.buffer.write(output.encode() + self.newline.encode())
            except AttributeError:
                print(output, end=self.newline)

and that fixed the problem for me.

[asottile]

if you're monkeypatching sys.stdout that's on you

I'd suggest using an io.TextIOWrapper to better match what sys.stdout actually is

[jaraco]

This error is encountered when using flake8 under pytest:

draft $ cat module.py
import foo
draft $ pip-run -q pytest pytest-flake8 -- -m pytest --flake8
======================================================================== test session starts ========================================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/jaraco/draft
plugins: flake8-1.0.7
collected 1 item                                                                                                                                                    

module.py F                                                                                                                                                   [100%]

============================================================================= FAILURES ==============================================================================
___________________________________________________________________________ FLAKE8-check ____________________________________________________________________________
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/_pytest/runner.py:170: in pytest_runtest_call
    raise e
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/_pytest/runner.py:162: in pytest_runtest_call
    item.runtest()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/pytest_flake8.py:120: in runtest
    found_errors, out, err = call(
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/py/_io/capture.py:150: in call
    res = func(*args, **kwargs)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/pytest_flake8.py:222: in check_file
    app.report_errors()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/main/application.py:309: in report_errors
    results = self.file_checker_manager.report()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/checker.py:249: in report
    results_reported += self._handle_results(filename, results)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/checker.py:155: in _handle_results
    reported_results_count += style_guide.handle_error(
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/style_guide.py:429: in handle_error
    return guide.handle_error(
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/style_guide.py:581: in handle_error
    self.formatter.handle(error)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/formatting/base.py:100: in handle
    self.write(line, source)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/formatting/base.py:203: in write
    self._write(line)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-tv9773ay/flake8/formatting/base.py:187: in _write
    sys.stdout.buffer.write(output.encode() + self.newline.encode())
E   AttributeError: '_io.StringIO' object has no attribute 'buffer'
------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------
WARNING  flake8.options.manager:manager.py:217 option --max-complexity: please update from optparse string `type=` to argparse callable `type=` -- this will be an error in the future
WARNING  flake8.checker:checker.py:115 The multiprocessing module is not available. Ignoring --jobs arguments.
====================================================================== short test summary info ======================================================================
FAILED module.py::FLAKE8 - AttributeError: '_io.StringIO' object has no attribute 'buffer'
========================================================================= 1 failed in 0.11s =========================================================================

Pinning flake8 to <4 restores the working behavior:

draft $ pip-run -q pytest pytest-flake8 'flake8<4' -- -m pytest --flake8
======================================================================== test session starts ========================================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/jaraco/draft
plugins: flake8-1.0.7
collected 1 item                                                                                                                                                    

module.py F                                                                                                                                                   [100%]

============================================================================= FAILURES ==============================================================================
___________________________________________________________________________ FLAKE8-check ____________________________________________________________________________
/Users/jaraco/draft/module.py:1:1: F401 'foo' imported but unused

------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------
WARNING  flake8.options.manager:manager.py:186 option --indent-size: please update `help=` text to use %(default)s instead of %default -- this will be an error in the future
WARNING  flake8.options.manager:manager.py:207 option --max-complexity: please update from optparse string `type=` to argparse callable `type=` -- this will be an error in the future
WARNING  flake8.checker:checker.py:119 The multiprocessing module is not available. Ignoring --jobs arguments.
====================================================================== short test summary info ======================================================================
FAILED module.py::FLAKE8
========================================================================= 1 failed in 0.03s =========================================================================

[brettcannon]

We also ran into this when trying to put flake8 behind a language server for editor integration.

And I think Anthony meant to suggest using io.TextIOWrapper and not io.StringIOWrapper.

[asottile]

thanks -- fixed! I'm currently working on a revamped public api which will return (file, position, code, message) tuples directly which should hopefully make these unsupported hacks unnecessary

[HebaruSan]

@asottile, where does this say what should be done by users of pytest --flake8?

[asottile]

check the linked issues