poetry-core 2.0.0 can no longer build when the sourcetree contains symlinks outside itself

[DrewDevereux]

Description

Since the release of poetry-core v2.0.0, twine can no longer build a poetry package whose source tree contains symlinks outside itself. Previously it could. Now it fails with errors like

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    main()
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/usr/local/lib/python3.10/dist-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
    return _build_backend().build_wheel(
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 58, in build_wheel
    return WheelBuilder.make_in(
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 91, in make_in
    wb.build(target_dir=directory)
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 131, in build
    self._copy_module(zip_file)
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 308, in _copy_module
    self._add_file(wheel, file.path, file.relative_to_target_root())
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/builder.py", line 374, in relative_to_target_root
    path = self.relative_to_source_root()
  File "/tmp/build-env-cf76kflm/lib/python3.10/site-packages/poetry/core/masonry/builders/builder.py", line 371, in relative_to_source_root
    return self.path.relative_to(self.source_root)
  File "/usr/lib/python3.10/pathlib.py", line 818, in relative_to
    raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '/true/location/of/myschema.json' is not in the subpath of '/mypackage/src' OR one path is relative and the other is absolute.

Is this deliberate? I realise it is an edge case, and probably there are arguments for disallowing these symlinks.

(It doesn't seem to be possible for me to raise an issue against poetry-core itself.)

Workarounds

Poetry Installation Method

other

Operating System

Ubuntu 22.04

Poetry Version

poetry-core v2.0.0

Poetry Configuration

-

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

-

[dimbleby]

Are you certain that this has changed? there are several existing issues reporting the same behaviour

either way it looks intended and correct

[DrewDevereux]

Well, we've had these symlinks in place for about 6 months, and an unchanged CI pipeline building our package. The failures coincide with the release of the new version.

If it is confirmed that this is intended and correct, we'll just have to find a way to get rid of these symlinks. As I said, I can certainly see why you wouldn't want to support this kind of thing.

[dimbleby]

nevertheless I tried for myself with older versions of poetry / poetry-core - and indeed the same error is raised.

So either there is something more subtle about what you are doing which we have not identified - or you have misunderstood what has changed

[abn]

It does looks like tools like setuptools allow for symlinks in the source tree. Whether this is good or bad, I am not sure. To me, it feels like you could potentially end up with completely different files when you attempt to rebuild a package and or different failures.

Maybe it is worth considering the adding the ability to allow for includes to explicitly allow symlinks if a project really wants to do it.

[dimbleby]

Supporting symlinks feels like quite the can of worms: and I continue to claim that this hasn't actually changed recently at all. (Though I don't know how to explain OP's report that something has changed!)

If I am right and this has never been supported then I suppose this could become a feature request. But it sounds as though even the reporter doesn't think this is a particularly great idea, my gut would be to turn that down.

If I am wrong and something used to work but now doesn't then it probably depends what that investigation shows: in that case maybe something should be reverted, or maybe it's better now anyway.