Requirements parsing error on some non-normalized spellings in version selection

[tserg]

This issue was migrated from gh-action-pip-audit, since the bug being reported isn't in the action itself.

Upstream tracking: pypa/packaging#653

---

Running the following .yml file results in the following error. I am currently trying to troubleshoot an error flagged by pip-audit for setuptools in the CI environment, and it is likely that setuptools is getting flagged by pip-audit still, before running into this error.

jobs:
  pip-audit:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Run pip-audit
        uses: pypa/gh-action-pip-audit@v1.0.4
        with:
          inputs: requirements-dev.txt
          ignore-vulns: PYSEC-2021-427

Traceback (most recent call last):
  File "/home/runner/work/_actions/pypa/gh-action-pip-audit/v1.0.4/action.py", line 138, in <module>
    with open("/tmp/pip-audit-output.txt", "r") as io:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-audit-output.txt'

[woodruffw]

Thanks for the report!

That error (which isn't great -- sorry about that) probably indicates that the underlying pip-audit invocation failed, and thus failed to produce our expected output.

Could you re-run your workflow with debugging enabled, i.e.:

      - name: Run pip-audit
        uses: pypa/gh-action-pip-audit@v1.0.4
        with:
          inputs: requirements-dev.txt
          ignore-vulns: PYSEC-2021-427
          internal-be-careful-debug: true

and share the output from that?

[tserg]

This is the full traceback with debugging enabled:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/packaging/requirements.py", line 35, in __init__
    parsed = parse_requirement(requirement_string)
  File "/usr/local/lib/python3.10/dist-packages/packaging/_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "/usr/local/lib/python3.10/dist-packages/packaging/_parser.py", line 82, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
  File "/usr/local/lib/python3.10/dist-packages/packaging/_parser.py", line 116, in _parse_requirement_details
    specifier = _parse_specifier(tokenizer)
  File "/usr/local/lib/python3.10/dist-packages/packaging/_parser.py", line 203, in _parse_specifier
    with tokenizer.enclosing_tokens("LEFT_PARENTHESIS", "RIGHT_PARENTHESIS"):
  File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/usr/local/lib/python3.10/dist-packages/packaging/_tokenizer.py", line 181, in enclosing_tokens
    self.raise_syntax_error(
  File "/usr/local/lib/python3.10/dist-packages/packaging/_tokenizer.py", line 161, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected closing RIGHT_PARENTHESIS
    eth-utils (<2.0.0,>=1.0.0-beta.1)
              ~~~~~~~~~~~~~~~~~^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line [86](https://github.com/lindy-labs/aura_contracts/actions/runs/3843296440/jobs/6545421863#step:3:89), in _run_code
    exec(code, run_globals)
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/__main__.py", line 8, in <module>
    audit()
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_cli.py", line 455, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_audit.py", line 67, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 155, in query_all
    for spec in specs:
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 116, in collect
    for _, dep in self._collect_cached_deps(filename, reqs):
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 328, in _collect_cached_deps
    for req, resolved_deps in self._resolver.resolve_all(iter(req_values)):
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line [88](https://github.com/lindy-labs/aura_contracts/actions/runs/3843296440/jobs/6545421863#step:3:91), in resolve_all
    yield (req, self.resolve(req))
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 77, in resolve
    result = self.resolver.resolve([req])
  File "/home/runner/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 521, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/runner/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 402, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/home/runner/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 238, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/home/runner/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 228, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 415, in get_dependencies
    return candidate.dependencies
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 126, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/home/runner/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 112, in _get_dependencies
    r = Requirement(d)
  File "/usr/local/lib/python3.10/dist-packages/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected closing RIGHT_PARENTHESIS
    eth-utils (<2.0.0,>=1.0.0-beta.1)
              ~~~~~~~~~~~~~~~~~^

Traceback (most recent call last):
  File "/home/runner/work/_actions/pypa/gh-action-pip-audit/v1.0.4/action.py", line [138](https://github.com/lindy-labs/aura_contracts/actions/runs/3843296440/jobs/6545421863#step:3:141), in <module>
    with open("/tmp/pip-audit-output.txt", "r") as io:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-audit-output.txt'

[woodruffw]

Hmm, that's a strange one -- it looks like packaging's own requirements parser failed, taking pip-audit down with it.

Two things:

1. Could you confirm that your requirements file actually functions when used with a recent pip? I.e., could you try python -m pip install --upgrade pip && python -m pip install -r dev-requirements.txt in a virtual environment?
2. Could you share the requirements-dev.txt you're using, or at least the part that's highlighted in that error?

[di]

Looks like a bug in packaging, I filed pypa/packaging#653.

In the interim @tserg, you can change that requirement to eth-utils (<2.0.0,>=1.0.0b1) if it's in your control.

[di]

(Since this isn't necessarily an issue with the action, this should probably get moved to pypa/pip-audit instead.)

[woodruffw]

Agreed, moving it.

[woodruffw]

Looks like we'll be able to resolve once packaging releases version 23: pypa/packaging#635

[tserg]

Thanks both! Unfortunately, it's not within our control so we will wait for the fix.

[woodruffw]

Looks like 23.0 was cut yesterday, so I'll create a bump PR in a moment.

[woodruffw]

#471 has the bump.

[woodruffw]

This has been cut with 2.4.13. I'll update the action once that goes live.