Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] ubuntu plugins not valid after installation #3289

Open
1 task done
monchin opened this issue Nov 14, 2024 · 4 comments · May be fixed by #3293
Open
1 task done

[bug] ubuntu plugins not valid after installation #3289

monchin opened this issue Nov 14, 2024 · 4 comments · May be fixed by #3293
Labels
🐛 bug Something isn't working

Comments

@monchin
Copy link
Contributor

monchin commented Nov 14, 2024

Describe the bug

plugins is invalid after installation on Ubuntu

To reproduce

In one pdm project, create plugin/my_plugin.py

# plugin/my_plugin.py`
def plugin(core):
    print("test")

and plugin/pyproject.toml

# plugin/pyproject.toml
[project]
name = "my_plugin"
version = "0.0.0"

[project.entry-points.pdm]
test_plug = "my_plugin:plugin"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

In pyproject.toml, add

[tool.pdm]
plugins = [
    "-e file:///${PROJECT_ROOT}/plugin#egg=my-plugin"
]

run pdm install --plugins, make sure the plugins are installed successfully by checking .pdm-plugins which should be

.pdm-plugins
└── local
    └── lib
        └── python3.10
            └── dist-packages
                ├── my_plugin-0.0.0.dist-info
                │   ├── direct_url.json
                │   ├── entry_points.txt
                │   ├── METADATA
                │   ├── RECORD
                │   └── WHEEL
                └── my_plugin.pth

Then, try to type something starts with pdm like pdm lock

Expected Behavior

If the plugin is valid, whatever you input, "test" would be output as long as you are running a pdm command

Environment Information

pdm 2.20.1
ubuntu 22.04

pdm -v output

No response

Additional Context

Reason Analysis

On windows system, plugins' installation route and loading route would be the same, both ${PROJECT}/.pdm-plugins/Lib/site-packages, but on ubuntu they are different.
On ubuntu, as site.py said

For Debian and derivatives, this sys.path is augmented with directories
for packages distributed within the distribution. Local addons go
into /usr/local/lib/python<version>/dist-packages, Debian addons
install into /usr/lib/python3/dist-packages.
/usr/lib/python<version>/site-packages is not used.

So the installation route would be /${PROJECT}/.pdm-plugins/local/lib/python{ver}/dist-packages, while the loading route got by

# pdm/core.py
    def _add_project_plugins_library(self) -> None:
        ...
        purelib = sysconfig.get_path("purelib", scheme, replace_vars)

would be ${PROJECT}/.pdm-plugins/lib/python{ver}/site-pacakges, so pdm would not load any plugins on ubuntu.

In fact, when I manually change the local/lib/python{ver}/dist-packages to lib/python{ver}/site-pacakges, plugins are enabled.

So, the two routes on ubuntu must somehow be consistent.

Are you willing to submit a PR to fix this bug?

  • Yes, I would like to submit a PR.
@monchin monchin added the 🐛 bug Something isn't working label Nov 14, 2024
@monchin
Copy link
Contributor Author

monchin commented Nov 15, 2024

Noticed that:

# models/in_process/sysconfig_get_paths.py # L31
def get_paths(kind="default", vars=None):
    ...
    else:
        if sys.platform == "darwin" and "osx_framework_library" in scheme_names and kind == "prefix":
            return sysconfig.get_paths("posix_prefix", vars=vars)
        return sysconfig.get_paths(vars=vars)

ubuntu's sys.platform is 'linux', kind is 'prefix', so it'll run sysconfig.get_paths(vars=vars), which would use default scheme "posix_local", but when loding plugins,

# core.py # L314
    def _add_project_plugins_library(self) -> None:
        ...
        scheme = "nt" if os.name == "nt" else "posix_prefix"
        purelib = sysconfig.get_path("purelib", scheme, replace_vars)

the scheme is "posix_prefix", that's the reason that installation and loding routes are different. I tried to change scheme to "posix_local", and the plugins could be loaded successfully.

Since I only have windows and ubuntu system computers, I'm not sure how to fix it correctly in all systems. On my ubuntu computer, sysconfig.get_scheme_names() got ('deb_system', 'nt', 'nt_user', 'osx_framework_user', 'posix_home', 'posix_local', 'posix_prefix', 'posix_user'), no osx_framework_library. So maybe it should be changed to

        if kind == "prefix":
            if (sys.platform == "darwin" and "osx_framework_library" in scheme_names) or (sys.platform == "linux"):
                return sysconfig.get_paths("posix_prefix", vars=vars)

? But I'm afraid it may fail on other systems such as centos

@monchin
Copy link
Contributor Author

monchin commented Nov 15, 2024

Perhaps

if kind == "prefix":
    if (sys.platform == "darwin" and "osx_framework_library" in scheme_names) or (sys.platform == "linux" and "deb_system" in scheme names):

would be better, but I have no way to test it on other systems

@o-moe
Copy link
Contributor

o-moe commented Nov 15, 2024

Here's the sysconfig from a macOS 14.7 (Apple silicon and Python 3.11.10):

('nt', 'nt_user', 'nt_venv', 'osx_framework_library', 'osx_framework_user', 'posix_home', 'posix_prefix', 'posix_user', 'posix_venv', 'venv')

@monchin
Copy link
Contributor Author

monchin commented Nov 15, 2024

Here's the sysconfig from a macOS 14.7

I believe pdm is adapting macOS well because it checks if the platform is "darwin". But it dosen't check for "linux". What I'm afraid is linux systems other than debian and derivatives, I don't know their result of syscomfig.get_default_scheme(). But as load_plugins use "posix_prefix" for all systems besides os.name == nt, I doubt if

if kind == "prefix" and os.name != "nt":
    return sysconfig.get_paths("posix_prefix", vars=vars)

is enough.

@monchin monchin linked a pull request Nov 15, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants