Bug: crash where no classifiers field exists #14

takeda · 2021-09-13T19:43:52Z

Bug

I have read the comment above and have completed each step
I have filled out the system info
I have described the bug, filled in the expected outcome and the actual
outcome including screenshots where appropriate

System info

OS: N/A
Browser: N/A
Version: 2021.4.1

Describe the bug

When classifiers is not defined in pyproject.toml licensecheck crashes with an error that can confuse new users.

Expected outcome

Program working

Actual outcome

Exception like this:

Traceback (most recent call last):
  File "<snip>/.venv/bin/licensecheck", line 8, in <module>
    sys.exit(cli())
  File "<snip>/.venv/lib/python3.8/site-packages/licensecheck/__init__.py", line 84, in cli
    dependenciesWLicenses = get_deps.getDepsWLicenses(
  File "<snip>/.venv/lib/python3.8/site-packages/licensecheck/get_deps.py", line 107, in getDepsWLicenses
    myLiceTxt = packageinfo.getMyPackageLicense()
  File "<snip>/.venv/lib/python3.8/site-packages/licensecheck/packageinfo.py", line 201, in getMyPackageLicense
    licenseClassifier = licenseFromClassifierlist(metaData["classifiers"])  # type:ignore
  File "/<snip>/.venv/lib/python3.8/site-packages/tomlkit/items.py", line 1007, in __getitem__
    return self._value[key]
  File "<snip>/.venv/lib/python3.8/site-packages/tomlkit/container.py", line 553, in __getitem__
    raise NonExistentKey(key)
tomlkit.exceptions.NonExistentKey: 'Key "classifiers" does not exist.'

The text was updated successfully, but these errors were encountered:

FredHappyface · 2021-09-13T21:26:21Z

Have you specified the license at all in the pyproject.toml file?

For example

[tool.poetry]
license= "mit"

takeda · 2021-09-13T21:52:23Z

Yes I did, although my value is:

license = "proprietary"

I thought licensecheck completly ignored that.

FredHappyface · 2021-09-13T22:08:36Z

If you change that to mit for the purposes of testing then it should work fine

What I'll need to do is provide some more useful warnings/errors in the case of an 'invalid' license and in the event license and classifiers are blank

FredHappyface · 2021-09-13T22:55:47Z

Fixed in 746eace

takeda · 2021-09-14T00:31:32Z

@FredHappyface the #15 was fixed, but #14 is still there

746eace#diff-f57c096bd6398fbed56e24e43f5e856dbe357704addd15869f01809a20e9627eR204

licenseClassifier = licenseFromClassifierlist(metaData["classifiers"])

I believe should be

licenseClassifier = licenseFromClassifierlist(metaData.get("classifiers", []))

Note that I don't know if metaData is a dict() so this might still be wrong.

bruceadams · 2022-02-01T19:45:17Z

I am very confused. Even on pretty much the simplest poetry project I can imagine running licensecheck fails giving me the error:

tomlkit.exceptions.NonExistentKey: 'Key "classifiers" does not exist.'

Steps to reproduce:

$ git clone https://github.com/bruceadams/discovery-files.git
Cloning into 'discovery-files'...
remote: Enumerating objects: 239, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 239 (delta 0), reused 3 (delta 0), pack-reused 235
Receiving objects: 100% (239/239), 144.36 KiB | 4.25 MiB/s, done.
Resolving deltas: 100% (140/140), done.
$ cd discovery-files
$ poetry install
Creating virtualenv discovery-files-bb0M-Gbu-py3.9 in /Users/bruce/Library/Caches/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 38 installs, 0 updates, 0 removals

  • Installing certifi (2021.10.8)
  • Installing charset-normalizer (2.0.11)
  • Installing idna (3.3)
  • Installing six (1.16.0)
  • Installing urllib3 (1.26.8)
  • Installing attrs (21.4.0)
  • Installing lazy-object-proxy (1.7.1)
  • Installing pyjwt (2.3.0)
  • Installing pyparsing (3.0.7)
  • Installing python-dateutil (2.8.2)
  • Installing requests (2.27.1)
  • Installing tomli (2.0.0)
  • Installing types-setuptools (57.4.8)
  • Installing typing-extensions (4.0.1)
  • Installing wrapt (1.13.3)
  • Installing astroid (2.9.3)
  • Installing click (8.0.3)
  • Installing fhconfparser (2022)
  • Installing ibm-cloud-sdk-core (0.5.2)
  • Installing iniconfig (1.1.1)
  • Installing isort (5.10.1)
  • Installing mccabe (0.6.1)
  • Installing mypy-extensions (0.4.3)
  • Installing packaging (21.3)
  • Installing pathspec (0.9.0)
  • Installing platformdirs (2.4.1)
  • Installing pluggy (1.0.0)
  • Installing py (1.11.0)
  • Installing requirements-parser (0.5.0)
  • Installing toml (0.10.2)
  • Installing tomlkit (0.9.0)
  • Installing websocket-client (0.48.0)
  • Installing black (22.1.0)
  • Installing ibm-watson (3.4.2)
  • Installing licensecheck (2022)
  • Installing mypy (0.931)
  • Installing pylint (2.12.2)
  • Installing pytest (6.2.5)
$ poetry run licensecheck
Traceback (most recent call last):
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/bin/licensecheck", line 8, in <module>
    sys.exit(cli())
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/lib/python3.9/site-packages/licensecheck/__init__.py", line 90, in cli
    dependenciesWLicenses = get_deps.getDepsWLicenses(
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/lib/python3.9/site-packages/licensecheck/get_deps.py", line 107, in getDepsWLicenses
    myLiceTxt = packageinfo.getMyPackageLicense()
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/lib/python3.9/site-packages/licensecheck/packageinfo.py", line 143, in getMyPackageLicense
    licenseClassifier = licenseFromClassifierlist(metaData["classifiers"])
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/lib/python3.9/site-packages/tomlkit/items.py", line 1229, in __getitem__
    return cast(Item, self._value[key])
  File "/Users/bruce/Library/Caches/pypoetry/virtualenvs/discovery-files-bb0M-Gbu-py3.9/lib/python3.9/site-packages/tomlkit/container.py", line 606, in __getitem__
    raise NonExistentKey(key)
tomlkit.exceptions.NonExistentKey: 'Key "classifiers" does not exist.'

takeda · 2022-02-01T20:02:43Z

@bruceadams This is what I was talking about. I think you should open a brand new issue, and could reference this one. I believe this issue no longer notifies the author about responses.

FredHappyface · 2022-02-01T20:25:19Z

You're correct, just seen this now - apologies for this. Reopened

bruceadams · 2022-02-01T20:43:04Z

Oh! If I just add classifiers = [] in my pyproject.toml, the tool runs fine! (I'm not publishing my packages, so classifiers is not relevant to my packages.)

FredHappyface · 2022-02-01T21:05:21Z

Yeah absolutely, I've updated with (hopefully) a fix and removed a couple packages - the update (2022.0.1) does require py >= 3.8 though

FredHappyface · 2022-02-01T21:06:01Z

Keeping this open awaiting confirmation this time. Sorry again for missing the comments before

bruceadams · 2022-02-01T21:28:16Z

Updating to licensecheck 2022.0.1 and all is good! 🎉🎊 Thanks!

$ poetry run licensecheck
┌──────────┬────────────────────┬────────────────────┐
│Compatible│Package             │License             │
├──────────┼────────────────────┼────────────────────┤
│True      │certifi             │Mozilla Public Licen│
│True      │charset-normalizer  │MIT License         │
│True      │ibm-cloud-sdk-core  │Apache Software Lice│
│True      │ibm-watson          │Apache Software Lice│
│True      │idna                │BSD License         │
│True      │pyjwt               │MIT License         │
│True      │python-dateutil     │Apache Software Lice│
│True      │requests            │Apache Software Lice│
│True      │six                 │MIT License         │
│True      │urllib3             │MIT License         │
│True      │websocket-client    │Apache Software Lice│
└──────────┴────────────────────┴────────────────────┘

I'm mostly on Python 3.9; requiring 3.8 is no problem at all for me.

takeda · 2022-02-04T02:04:04Z

@FredHappyface after the update I got:

Traceback (most recent call last):
  File "~/.venv/bin/licensecheck", line 8, in <module>
    sys.exit(cli())
  File "~/.venv/lib/python3.9/site-packages/licensecheck/__init__.py", line 90, in cli
    dependenciesWLicenses = get_deps.getDepsWLicenses(
  File "~/.venv/lib/python3.9/site-packages/licensecheck/get_deps.py", line 112, in getDepsWLicenses
    for package in packageinfo.getPackages(reqs):
  File "~/.venv/lib/python3.9/site-packages/licensecheck/packageinfo.py", line 115, in getPackages
    localReqs = getPackagesFromLocal(reqs)
  File "~/.venv/lib/python3.9/site-packages/licensecheck/packageinfo.py", line 49, in getPackagesFromLocal
    "size": getModuleSize(cast(Path, resources.files(requirement)), name),
  File "/nix/store/qx995zl14gqgx0ly0hwx3cy0fcz27ich-python3-3.9.6/lib/python3.9/importlib/resources.py", line 147, in files
    return _common.from_package(_get_package(package))
  File "/nix/store/qx995zl14gqgx0ly0hwx3cy0fcz27ich-python3-3.9.6/lib/python3.9/importlib/resources.py", line 51, in _get_package
    raise TypeError('{!r} is not a package'.format(package))
TypeError: 'tabulate' is not a package

Also I got this error when trying to update licensecheck:

 poetry add -D licensecheck@latest
Using version ^2022.0.1 for licensecheck

Updating dependencies
Resolving dependencies... (1.4s)

  SolverProblemError

  Because no versions of black match >21.9b0,<21.10b0 || >21.10b0,<21.11b0 || >21.11b0,<21.11b1 || >21.11b1,<21.12b0 || >21.12b0,<22.0
   and black (21.9b0) depends on tomli (>=0.2.6,<2.0.0), black (>=21.9b0,<21.10b0 || >21.10b0,<21.11b0 || >21.11b0,<21.11b1 || >21.11b1,<21.12b0 || >21.12b0,<22.0) requires tomli (>=0.2.6,<2.0.0).
  And because black (21.10b0) depends on tomli (>=0.2.6,<2.0.0)
   and black (21.11b0) depends on tomli (>=0.2.6,<2.0.0), black (>=21.9b0,<21.11b1 || >21.11b1,<21.12b0 || >21.12b0,<22.0) requires tomli (>=0.2.6,<2.0.0).
  And because black (21.11b1) depends on tomli (>=0.2.6,<2.0.0)
   and black (21.12b0) depends on tomli (>=0.2.6,<2.0.0), black (>=21.9b0,<22.0) requires tomli (>=0.2.6,<2.0.0).
  Because no versions of licensecheck match >2022.0.1,<2023.0.0
   and licensecheck (2022.0.1) depends on tomli (>=2.0.0,<3), licensecheck (>=2022.0.1,<2023.0.0) requires tomli (>=2.0.0,<3).
  Thus, licensecheck (>=2022.0.1,<2023.0.0) is incompatible with black (>=21.9b0,<22.0).
  So, because amiman depends on both black (^21.9b0) and licensecheck (^2022.0.1), version solving failed.

But, after I removed black then updated licensecheck then I was able to install black again.

takeda · 2022-02-04T02:06:47Z

I think the conflict could be that previously black was ^21.9b0 but when I removed and added it upgraded to ^22.1.0.

Niccolum · 2022-04-04T07:11:19Z

Maybe close ticket? Or some problem still exists?
@takeda

takeda · 2022-04-04T07:27:11Z

Good point, after that I did not see it happening anymore, so I'm closing it.

takeda added the bug Something isn't working label Sep 13, 2021

takeda assigned FredHappyface Sep 13, 2021

FredHappyface added this to the 2021.5 milestone Sep 13, 2021

FredHappyface closed this as completed Sep 13, 2021

FredHappyface reopened this Feb 1, 2022

takeda closed this as completed Apr 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: crash where no classifiers field exists #14

Bug: crash where no classifiers field exists #14

takeda commented Sep 13, 2021 •

edited

Loading

FredHappyface commented Sep 13, 2021

takeda commented Sep 13, 2021

FredHappyface commented Sep 13, 2021

FredHappyface commented Sep 13, 2021

takeda commented Sep 14, 2021

bruceadams commented Feb 1, 2022

takeda commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

bruceadams commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

bruceadams commented Feb 1, 2022

takeda commented Feb 4, 2022

takeda commented Feb 4, 2022

Niccolum commented Apr 4, 2022

takeda commented Apr 4, 2022

Bug: crash where no classifiers field exists #14

Bug: crash where no classifiers field exists #14

Comments

takeda commented Sep 13, 2021 • edited Loading

Bug

System info

Describe the bug

Expected outcome

Actual outcome

FredHappyface commented Sep 13, 2021

takeda commented Sep 13, 2021

FredHappyface commented Sep 13, 2021

FredHappyface commented Sep 13, 2021

takeda commented Sep 14, 2021

bruceadams commented Feb 1, 2022

takeda commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

bruceadams commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

FredHappyface commented Feb 1, 2022

bruceadams commented Feb 1, 2022

takeda commented Feb 4, 2022

takeda commented Feb 4, 2022

Niccolum commented Apr 4, 2022

takeda commented Apr 4, 2022

takeda commented Sep 13, 2021 •

edited

Loading