pre-commit error

[wangkuiyi]

Not sure if this is related with the fact that we switched to https://github.com/PaddlePaddle/mirrors-yapf.

I tried pre-commit install and pre-commit autoupdate, and they don't fix the following error:

yi@WangYis-iMac:~/work/paddle/paddle/majel (remove_unnecessary_test_output)*$ git commit -a -m 'Remove unnecesssary test outputs'
[INFO] Installing environment for https://github.com/PaddlePaddle/mirrors-yapf.git.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: Command: ('/Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/python2.7', u'/Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/pip', 'install', '.')
Return code: 1
Expected return code: 0
Output: 
    Processing /Users/yi/.pre-commit/repo6Y20ij
    Collecting yapf==0.16.2 (from pre-commit-dummy-package==0.0.0)
      Downloading yapf-0.16.2-py2.py3-none-any.whl (149kB)
    Installing collected packages: yapf, pre-commit-dummy-package
      Running setup.py install for pre-commit-dummy-package: started
        Running setup.py install for pre-commit-dummy-package: finished with status 'error'
        Complete output from command /Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-Ok2_Hv-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-bqpi8n-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/../include/site/python2.7/pre-commit-dummy-package:
        running install
        running build
        running install_egg_info
        running egg_info
        creating pre_commit_dummy_package.egg-info
        writing requirements to pre_commit_dummy_package.egg-info/requires.txt
        writing pre_commit_dummy_package.egg-info/PKG-INFO
        writing top-level names to pre_commit_dummy_package.egg-info/top_level.txt
        writing dependency_links to pre_commit_dummy_package.egg-info/dependency_links.txt
        writing manifest file 'pre_commit_dummy_package.egg-info/SOURCES.txt'
        reading manifest file 'pre_commit_dummy_package.egg-info/SOURCES.txt'
        writing manifest file 'pre_commit_dummy_package.egg-info/SOURCES.txt'
        Copying pre_commit_dummy_package.egg-info to /Users/yi/.pre-commit/repo6Y20ij/py_env-default/lib/python2.7/site-packages/pre_commit_dummy_package-0.0.0-py2.7.egg-info
        running install_scripts
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/private/var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-Ok2_Hv-build/setup.py", line 7, in <module>
            install_requires=['yapf==0.16.2'],
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 151, in setup
            dist.run_commands()
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 953, in run_commands
            self.run_command(cmd)
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
            cmd_obj.run()
          File "/Users/yi/.pre-commit/repo6Y20ij/py_env-default/lib/python2.7/site-packages/setuptools/command/install.py", line 61, in run
            return orig.install.run(self)
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 575, in run
            self.run_command(cmd_name)
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/cmd.py", line 326, in run_command
            self.distribution.run_command(command)
          File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 972, in run_command
            cmd_obj.run()
          File "/Users/yi/.pre-commit/repo6Y20ij/py_env-default/lib/python2.7/site-packages/setuptools/command/install_scripts.py", line 17, in run
            import setuptools.command.easy_install as ei
          File "/Users/yi/.pre-commit/repo6Y20ij/py_env-default/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 49, in <module>
            from setuptools.py27compat import rmtree_safe
          File "/Users/yi/.pre-commit/repo6Y20ij/py_env-default/lib/python2.7/site-packages/setuptools/py27compat.py", line 7, in <module>
            import six
        ImportError: No module named six
        
        ----------------------------------------
    
Errors: 
    Command "/Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-Ok2_Hv-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-bqpi8n-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/yi/.pre-commit/repo6Y20ij/py_env-default/bin/../include/site/python2.7/pre-commit-dummy-package" failed with error code 1 in /private/var/folders/f9/ymd5d3xs62gc7j6db4_8jdp40000gn/T/pip-Ok2_Hv-build/
    

Check the log at ~/.pre-commit/pre-commit.log

[wangkuiyi]

It complains that

ImportError: No module named six

but

yi@WangYis-iMac:~/work/paddle/paddle/majel (remove_unnecessary_test_output)*$ pip install six
Requirement already satisfied: six in /usr/local/lib/python2.7/site-packages

[reyoung]

It seems that in this computer, Python cannot install any package inside a virtualenv.

virtualenv tmp
source tmp/bin/activate
pip install protobuf  # cause same error.

So it seems something going wrong in virtualenv.

[reyoung]

Related issue is here

[alexanderad]

Just leaving a comment here that we started seeing this error this morning on one of our projects completely unrelated to Paddle (I'm coming from the Google Search here). So might be related to some change in setuptools?

[reyoung]

@alexanderad I think it because the setuptools just release the 36.0.0 version.

And it seems that this version adds six as a dependency, but I did not dig into setuptools code.

[reyoung]

@alexanderad Is there any way we can bypass this issue now?

It seems that virtualenv will install the newest version of setuptools that we cannot change?

[alexanderad]

I've seen on setuptools issue tracker, that setuptools 36 has been removed from pypi index recently. We are back to old 35.0.2. Which effectively means that you can just re-try the build and it should work now.

[alexanderad]

You can keep track here: pypa/setuptools#1042

[wangkuiyi]

I confirm that I can just run pre-commit run -a to reformat the source code. Thanks to @alexanderad and @reyoung