-
-
Notifications
You must be signed in to change notification settings - Fork 698
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DeprecationWarning: pkg_resources is deprecated as an API #2057
Comments
I ran this prompt against ChatGPT with the Browsing alpha:
It looked up the fix for me: And suggested: import importlib.resources
# Replace pkg_resources.resource_isdir with importlib.resources.files().is_file()
if importlib.resources.files(plugin.__name__).joinpath("static").is_file():
static_path = importlib.resources.as_file(
importlib.resources.files(plugin.__name__).joinpath("static")
)
if importlib.resources.files(plugin.__name__).joinpath("templates").is_file():
templates_path = importlib.resources.as_file(
importlib.resources.files(plugin.__name__).joinpath("templates")
) This looks wrong to me - I would expect something like |
Another prompt:
Response: import importlib.metadata
# Get the version number of the specified package
package_version = importlib.metadata.version(package) That seems to work: >>> import importlib.metadata
>>> importlib.metadata.version("datasette")
'0.64.2'
>>> importlib.metadata.version("pluggy")
'1.0.0'
>>> importlib.metadata.version("not-a-package")
...
importlib.metadata.PackageNotFoundError: No package metadata was found for not-a-package |
I was right about that: >>> importlib.resources.files('datasette_graphql')
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql')
>>> importlib.resources.files('datasette_graphql').joinpath("static")
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')
>>> p = importlib.resources.files('datasette_graphql').joinpath("static")
>>> p
PosixPath('/Users/simon/.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette_graphql/static')
>>> p.is_
p.is_absolute() p.is_char_device() p.is_fifo() p.is_mount() p.is_reserved() p.is_symlink()
p.is_block_device() p.is_dir() p.is_file() p.is_relative_to( p.is_socket()
>>> p.is_dir()
True
>>> p.is_file()
False |
Frustrating that this warning doesn't show up in the Datasette test suite itself. It shows up in plugin test suites that run this test: @pytest.mark.asyncio
async def test_plugin_is_installed():
datasette = Datasette(memory=True)
response = await datasette.client.get("/-/plugins.json")
assert response.status_code == 200
installed_plugins = {p["name"] for p in response.json()}
assert "datasette-chronicle" in installed_plugins If you run that test inside Datasette core |
Weird, I still can't get the warning to show even with this: @pytest.mark.asyncio
async def test_plugin_is_installed():
datasette = Datasette(memory=True)
class DummyPlugin:
__name__ = "DummyPlugin"
@hookimpl
def actors_from_ids(self, datasette, actor_ids):
return {}
try:
pm.register(DummyPlugin(), name="DummyPlugin")
response = await datasette.client.get("/-/plugins.json")
assert response.status_code == 200
installed_plugins = {p["name"] for p in response.json()}
assert "DummyPlugin" in installed_plugins
finally:
pm.unregister(name="ReturnNothingPlugin") |
Here's the exception it uses: >>> importlib.metadata.version("datasette")
'1.0a6'
>>> importlib.metadata.version("datasette2")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 996, in version
return distribution(distribution_name).version
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 969, in distribution
return Distribution.from_name(distribution_name)
File "/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for datasette2 |
Now I need to switch out datasette/datasette/plugins.py Lines 33 to 74 in 852f501
|
The >>> import importlib.metadata
>>> from pprint import pprint
>>> pprint(importlib.metadata.entry_points())
{'babel.checkers': [EntryPoint(name='num_plurals', value='babel.messages.checkers:num_plurals', group='babel.checkers'),
EntryPoint(name='python_format', value='babel.messages.checkers:python_format', group='babel.checkers')],
'babel.extractors': [EntryPoint(name='jinja2', value='jinja2.ext:babel_extract[i18n]', group='babel.extractors'),
EntryPoint(name='ignore', value='babel.messages.extract:extract_nothing', group='babel.extractors'),
EntryPoint(name='javascript', value='babel.messages.extract:extract_javascript', group='babel.extractors'),
EntryPoint(name='python', value='babel.messages.extract:extract_python', group='babel.extractors')],
'console_scripts': [EntryPoint(name='datasette', value='datasette.cli:cli', group='console_scripts'),
EntryPoint(name='normalizer', value='charset_normalizer.cli.normalizer:cli_detect', group='console_scripts'),
EntryPoint(name='pypprint', value='pprintpp:console', group='console_scripts'),
EntryPoint(name='cog', value='cogapp:main', group='console_scripts'),
EntryPoint(name='icdiff', value='icdiff:start', group='console_scripts'),
EntryPoint(name='pycodestyle', value='pycodestyle:_main', group='console_scripts'),
EntryPoint(name='sphinx-autobuild', value='sphinx_autobuild.__main__:main', group='console_scripts'),
EntryPoint(name='sphinx-apidoc', value='sphinx.ext.apidoc:main', group='console_scripts'),
EntryPoint(name='sphinx-autogen', value='sphinx.ext.autosummary.generate:main', group='console_scripts'),
EntryPoint(name='sphinx-build', value='sphinx.cmd.build:main', group='console_scripts'),
EntryPoint(name='sphinx-quickstart', value='sphinx.cmd.quickstart:main', group='console_scripts'),
EntryPoint(name='sphinx-to-sqlite', value='sphinx_to_sqlite.cli:cli', group='console_scripts'),
EntryPoint(name='pybabel', value='babel.messages.frontend:main', group='console_scripts'),
EntryPoint(name='docutils', value='docutils.__main__:main', group='console_scripts'),
EntryPoint(name='isort', value='isort.main:main', group='console_scripts'),
EntryPoint(name='isort-identify-imports', value='isort.main:identify_imports_main', group='console_scripts'),
EntryPoint(name='hupper', value='hupper.cli:main', group='console_scripts'),
EntryPoint(name='sqlite-utils', value='sqlite_utils.cli:cli', group='console_scripts'),
EntryPoint(name='py.test', value='pytest:console_main', group='console_scripts'),
EntryPoint(name='pytest', value='pytest:console_main', group='console_scripts'),
EntryPoint(name='pyflakes', value='pyflakes.api:main', group='console_scripts'),
EntryPoint(name='livereload', value='livereload.cli:main', group='console_scripts'),
EntryPoint(name='uvicorn', value='uvicorn.main:main', group='console_scripts'),
EntryPoint(name='httpx', value='httpx:main', group='console_scripts'),
EntryPoint(name='flake8', value='flake8.main.cli:main', group='console_scripts'),
EntryPoint(name='blacken-docs', value='blacken_docs:main', group='console_scripts'),
EntryPoint(name='pip', value='pip._internal.cli.main:main', group='console_scripts'),
EntryPoint(name='pip3', value='pip._internal.cli.main:main', group='console_scripts'),
EntryPoint(name='pip3.10', value='pip._internal.cli.main:main', group='console_scripts'),
EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts'),
EntryPoint(name='pygmentize', value='pygments.cmdline:main', group='console_scripts'),
EntryPoint(name='black', value='black:patched_main', group='console_scripts'),
EntryPoint(name='blackd', value='blackd:patched_main [d]', group='console_scripts'),
EntryPoint(name='codespell', value='codespell_lib:_script_main', group='console_scripts'),
EntryPoint(name='tabulate', value='tabulate:_main', group='console_scripts')],
'datasette': [EntryPoint(name='debug_permissions', value='datasette_debug_permissions', group='datasette'),
EntryPoint(name='codespaces', value='datasette_codespaces', group='datasette'),
EntryPoint(name='vega', value='datasette_vega', group='datasette'),
EntryPoint(name='x_forwarded_host', value='datasette_x_forwarded_host', group='datasette'),
EntryPoint(name='json_html', value='datasette_json_html', group='datasette'),
EntryPoint(name='datasette_write_ui', value='datasette_write_ui', group='datasette'),
EntryPoint(name='pretty_json', value='datasette_pretty_json', group='datasette'),
EntryPoint(name='graphql', value='datasette_graphql', group='datasette')],
'distutils.commands': [EntryPoint(name='compile_catalog', value='babel.messages.frontend:compile_catalog', group='distutils.commands'),
EntryPoint(name='extract_messages', value='babel.messages.frontend:extract_messages', group='distutils.commands'),
EntryPoint(name='init_catalog', value='babel.messages.frontend:init_catalog', group='distutils.commands'),
EntryPoint(name='update_catalog', value='babel.messages.frontend:update_catalog', group='distutils.commands'),
EntryPoint(name='isort', value='isort.setuptools_commands:ISortCommand', group='distutils.commands'),
EntryPoint(name='alias', value='setuptools.command.alias:alias', group='distutils.commands'),
EntryPoint(name='bdist_egg', value='setuptools.command.bdist_egg:bdist_egg', group='distutils.commands'),
EntryPoint(name='bdist_rpm', value='setuptools.command.bdist_rpm:bdist_rpm', group='distutils.commands'),
EntryPoint(name='build', value='setuptools.command.build:build', group='distutils.commands'),
EntryPoint(name='build_clib', value='setuptools.command.build_clib:build_clib', group='distutils.commands'),
EntryPoint(name='build_ext', value='setuptools.command.build_ext:build_ext', group='distutils.commands'),
EntryPoint(name='build_py', value='setuptools.command.build_py:build_py', group='distutils.commands'),
EntryPoint(name='develop', value='setuptools.command.develop:develop', group='distutils.commands'),
EntryPoint(name='dist_info', value='setuptools.command.dist_info:dist_info', group='distutils.commands'),
EntryPoint(name='easy_install', value='setuptools.command.easy_install:easy_install', group='distutils.commands'),
EntryPoint(name='editable_wheel', value='setuptools.command.editable_wheel:editable_wheel', group='distutils.commands'),
EntryPoint(name='egg_info', value='setuptools.command.egg_info:egg_info', group='distutils.commands'),
EntryPoint(name='install', value='setuptools.command.install:install', group='distutils.commands'),
EntryPoint(name='install_egg_info', value='setuptools.command.install_egg_info:install_egg_info', group='distutils.commands'),
EntryPoint(name='install_lib', value='setuptools.command.install_lib:install_lib', group='distutils.commands'),
EntryPoint(name='install_scripts', value='setuptools.command.install_scripts:install_scripts', group='distutils.commands'),
EntryPoint(name='rotate', value='setuptools.command.rotate:rotate', group='distutils.commands'),
EntryPoint(name='saveopts', value='setuptools.command.saveopts:saveopts', group='distutils.commands'),
EntryPoint(name='sdist', value='setuptools.command.sdist:sdist', group='distutils.commands'),
EntryPoint(name='setopt', value='setuptools.command.setopt:setopt', group='distutils.commands'),
EntryPoint(name='test', value='setuptools.command.test:test', group='distutils.commands'),
EntryPoint(name='upload_docs', value='setuptools.command.upload_docs:upload_docs', group='distutils.commands'),
EntryPoint(name='bdist_wheel', value='wheel.bdist_wheel:bdist_wheel', group='distutils.commands')],
'distutils.setup_keywords': [EntryPoint(name='message_extractors', value='babel.messages.frontend:check_message_extractors', group='distutils.setup_keywords'),
EntryPoint(name='cffi_modules', value='cffi.setuptools_ext:cffi_modules', group='distutils.setup_keywords'),
EntryPoint(name='dependency_links', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
EntryPoint(name='eager_resources', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
EntryPoint(name='entry_points', value='setuptools.dist:check_entry_points', group='distutils.setup_keywords'),
EntryPoint(name='exclude_package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
EntryPoint(name='extras_require', value='setuptools.dist:check_extras', group='distutils.setup_keywords'),
EntryPoint(name='include_package_data', value='setuptools.dist:assert_bool', group='distutils.setup_keywords'),
EntryPoint(name='install_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
EntryPoint(name='namespace_packages', value='setuptools.dist:check_nsp', group='distutils.setup_keywords'),
EntryPoint(name='package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
EntryPoint(name='packages', value='setuptools.dist:check_packages', group='distutils.setup_keywords'),
EntryPoint(name='python_requires', value='setuptools.dist:check_specifier', group='distutils.setup_keywords'),
EntryPoint(name='setup_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
EntryPoint(name='test_loader', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
EntryPoint(name='test_runner', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
EntryPoint(name='test_suite', value='setuptools.dist:check_test_suite', group='distutils.setup_keywords'),
EntryPoint(name='tests_require', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
EntryPoint(name='use_2to3', value='setuptools.dist:invalid_unless_false', group='distutils.setup_keywords'),
EntryPoint(name='zip_safe', value='setuptools.dist:assert_bool', group='distutils.setup_keywords')],
'egg_info.writers': [EntryPoint(name='PKG-INFO', value='setuptools.command.egg_info:write_pkg_info', group='egg_info.writers'),
EntryPoint(name='dependency_links.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
EntryPoint(name='depends.txt', value='setuptools.command.egg_info:warn_depends_obsolete', group='egg_info.writers'),
EntryPoint(name='eager_resources.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
EntryPoint(name='entry_points.txt', value='setuptools.command.egg_info:write_entries', group='egg_info.writers'),
EntryPoint(name='namespace_packages.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
EntryPoint(name='requires.txt', value='setuptools.command.egg_info:write_requirements', group='egg_info.writers'),
EntryPoint(name='top_level.txt', value='setuptools.command.egg_info:write_toplevel_names', group='egg_info.writers')],
'flake8.extension': [EntryPoint(name='C90', value='mccabe:McCabeChecker', group='flake8.extension'),
EntryPoint(name='E', value='flake8.plugins.pycodestyle:pycodestyle_logical', group='flake8.extension'),
EntryPoint(name='F', value='flake8.plugins.pyflakes:FlakesChecker', group='flake8.extension'),
EntryPoint(name='W', value='flake8.plugins.pycodestyle:pycodestyle_physical', group='flake8.extension')],
'flake8.report': [EntryPoint(name='default', value='flake8.formatting.default:Default', group='flake8.report'),
EntryPoint(name='pylint', value='flake8.formatting.default:Pylint', group='flake8.report'),
EntryPoint(name='quiet-filename', value='flake8.formatting.default:FilenameOnly', group='flake8.report'),
EntryPoint(name='quiet-nothing', value='flake8.formatting.default:Nothing', group='flake8.report')],
'pylama.linter': [EntryPoint(name='isort', value='isort.pylama_isort:Linter', group='pylama.linter')],
'pytest11': [EntryPoint(name='icdiff', value='pytest_icdiff', group='pytest11'),
EntryPoint(name='asyncio', value='pytest_asyncio.plugin', group='pytest11'),
EntryPoint(name='xdist', value='xdist.plugin', group='pytest11'),
EntryPoint(name='xdist.looponfail', value='xdist.looponfail', group='pytest11'),
EntryPoint(name='timeout', value='pytest_timeout', group='pytest11'),
EntryPoint(name='anyio', value='anyio.pytest_plugin', group='pytest11')],
'setuptools.finalize_distribution_options': [EntryPoint(name='keywords', value='setuptools.dist:Distribution._finalize_setup_keywords', group='setuptools.finalize_distribution_options'),
EntryPoint(name='parent_finalize', value='setuptools.dist:_Distribution.finalize_options', group='setuptools.finalize_distribution_options')],
'sphinx.html_themes': [EntryPoint(name='alabaster', value='alabaster', group='sphinx.html_themes'),
EntryPoint(name='basic-ng', value='sphinx_basic_ng', group='sphinx.html_themes'),
EntryPoint(name='furo', value='furo', group='sphinx.html_themes')],
'sqlite_utils': [EntryPoint(name='hello_world', value='sqlite_utils_hello_world', group='sqlite_utils')]} |
Just found this migration guide: https://importlib-metadata.readthedocs.io/en/latest/migration.html |
This broke in Python 3.8:
|
Confirmed: https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files
|
I think I can fix this using https://importlib-resources.readthedocs.io/en/latest/using.html - maybe as a dependency only installed if the Python version is less than 3.9. |
I'll imitate 'importlib_resources>=1.3.1; python_version < "3.9"', Looks like if sys.version_info >= (3, 9): # pragma: no cover
import importlib.resources as importlib_resources
else: # pragma: no cover
import importlib_resources |
That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the "Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run. |
In my Python 3.8 environment I ran: datasette install datasette-init datasette-json-html And now
|
In the debugger:
|
Relevant code: Lines 1127 to 1146 in 80a9cd9
|
So the problem is the datasette/datasette/plugins.py Lines 61 to 91 in 80a9cd9
|
The problem is here:
That |
No that's not it actually, it's something else. Got to this point: DATASETTE_LOAD_PLUGINS=datasette-init python -i $(which datasette) plugins That fails and drops me into a debugger:
|
Hunch: https://pypi.org/project/importlib-metadata/ may help here. |
That does seem to fix the problem! |
Still fails in Python 3.9: https://github.com/simonw/datasette/actions/runs/6266752548/job/17018363302
|
Tested that locally with Python 3.9 from |
Tests all pass now. |
Got this running tests against Python 3.11.
I ran with
pytest -Werror --pdb -x
to get the debugger for that warning, but it turned out searching the code worked better. It's used in these two places:datasette/datasette/plugins.py
Lines 43 to 50 in 5890a20
datasette/datasette/app.py
Line 1037 in 5890a20
The text was updated successfully, but these errors were encountered: