Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

String install_requires fails in parsing #511

Closed
offbyone opened this issue May 20, 2017 · 6 comments
Closed

String install_requires fails in parsing #511

offbyone opened this issue May 20, 2017 · 6 comments
Labels
bug Something is not working setuptools Related to compiling requirements with `setuptools` build backend

Comments

@offbyone
Copy link

My setup.py defines install_requires thusly:

install_requires="""\
markovify>=0.6,<1.0
tweepy
"""

This is valid from setuptools' POV; it installs fine. However, when I run pip-compile in this directory I get a massive stack trace blowup:

Using indexes:
  http://localhost:3141/root/pypi/+simple/
Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 92, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1617, in parseString
    raise exc
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1607, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3376, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3698, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3359, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 2670, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_install.py", line 82, in __init__
    req = Requirement(req)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 96, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'>'"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.6.0/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/piptools/scripts/compile.py", line 173, in cli
    tmpfile.name, finder=repository.finder, session=repository.session, options=pip_options))
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_file.py", line 93, in parse_requirements
    for req in req_iter:
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_file.py", line 158, in process_line
    isolated=isolated, options=req_options, wheel_cache=wheel_cache
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_install.py", line 235, in from_line
    wheel_cache=wheel_cache, constraint=constraint)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_install.py", line 91, in __init__
    "Invalid requirement: '%s'\n%s" % (req, add_msg))
pip.exceptions.InstallationError: Invalid requirement: '>'
Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 92, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1617, in parseString
    raise exc
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1607, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3376, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3698, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3359, in parseImpl
    loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 2670, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/req/req_install.py", line 82, in __init__
    req = Requirement(req)
  File "/usr/local/opt/pyenv/versions/3.6.0/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 96, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'>'"
@rpkilby
Copy link

rpkilby commented May 22, 2017 •
edited
Loading

Not sure if it's relevant, but the trailing \ should be unnecessary in install_requires="""\. nvm, I'm a dummy. Forgot that the backslash removes the newline

@rpkilby
Copy link

rpkilby commented May 23, 2017 •
edited
Loading

Out of curiosity, is there a reason why you're using a multiline string here? Generally, the expected usage is a list of strings.

My guess is that the code is doing a for-each over the string (as it might expect to loop over a list), and failing once it tries to evaluate '>'.

@offbyone
Copy link
Author

offbyone commented May 23, 2017 via email

@vphilippon
Copy link
Member

Tried to reproduce, here's a minimal reproduction case:

setup.py:

from setuptools import setup

setup(
    name='test_piptools',
    version='0.0.1',
    install_requires="""\
markovify>=0.6,<1.0
tweepy
"""
)

Then, run pip-compile, which will parse the setup.py:

pip-compile

Note: This only happens if parsing the setup.py directly.
If we pipe "-e ." to pip-compile's stdin, which is equivalent to having a requirements.in with "-e .", like

echo "-e ." | pip-compile -o requirements.txt -

then it works.
So that's most likely in the parsing of the setup.py, as mentionned by @rpkilby.

@rpkilby
Copy link

rpkilby commented May 24, 2017

I'd assume this is related to the changes in #418/#481

@atugushev atugushev mentioned this issue Sep 24, 2019
Closed
@atugushev atugushev added the bug Something is not working label Sep 26, 2019
@atugushev atugushev added the setuptools Related to compiling requirements with `setuptools` build backend label Nov 22, 2019
@atugushev
Copy link
Member

I'll close it because the bug seems fixed in setuptools and couldn't reproduce since setuptools>=37.0.0. Thanks for the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something is not working setuptools Related to compiling requirements with `setuptools` build backend
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@offbyone @rpkilby @vphilippon @atugushev