Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

locker: move export logic into locker for reuse #3119

Merged
merged 8 commits into from
Oct 14, 2020

Conversation

abn
Copy link
Member

@abn abn commented Oct 7, 2020

This change, moves common functionality from exporter into the locker
to allow for sharing of logic that can generate DependencyPackage
instances given a a list of requirements using the lock data.

This change set also resolves #3115.

Resolves: #3112
Resolves: #3115
Resolves: #3160
Closes: #3125

@abn abn requested a review from a team October 7, 2020 13:52
@abn abn added this to the 1.1 milestone Oct 7, 2020
@abn abn marked this pull request as draft October 7, 2020 19:42
@abn
Copy link
Member Author

abn commented Oct 7, 2020

Moving to draft as this requires new tests and also existing tests need to be validate.

@abn abn force-pushed the cleanup/exports branch 2 times, most recently from 01da8ad to 689a949 Compare October 7, 2020 20:34
@abn abn marked this pull request as ready for review October 7, 2020 20:35
@marns93
Copy link

marns93 commented Oct 8, 2020

@abn Thanks for the fix. I've tested it locally in our repositories and it works as expected.
Looking forward to the next Poetry release. :)

@zyv
Copy link
Contributor

zyv commented Oct 12, 2020

Another test case from our users @ moneymeets/python-poetry-buildpack#16 - I didn't check, but that should work properly with this PR because you check whether the locked package is None, but doesn't work with my hack @ #3125 :

[[package]]
category = "main"
description = "Fast and simple WSGI-framework for small web-applications."
name = "bottle"
optional = false
python-versions = "*"
version = "0.12.18"

[[package]]
category = "main"
description = "WSGI HTTP Server for UNIX"
name = "gunicorn"
optional = false
python-versions = ">=3.4"
version = "20.0.4"

[package.dependencies]
setuptools = ">=3.0"

[package.extras]
eventlet = ["eventlet (>=0.9.7)"]
gevent = ["gevent (>=0.13)"]
setproctitle = ["setproctitle"]
tornado = ["tornado (>=0.2)"]

[[package]]
category = "main"
description = "Python client for Redis key-value store"
name = "redis"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
version = "3.5.3"

[package.extras]
hiredis = ["hiredis (>=0.1.3)"]

[metadata]
content-hash = "184994068e190b2720ed44423cc8673e8300ceae582e90301adbe7502e598b42"
lock-version = "1.0"
python-versions = "3.8.6"

[metadata.files]
bottle = [
    {file = "bottle-0.12.18-py3-none-any.whl", hash = "sha256:43157254e88f32c6be16f8d9eb1f1d1472396a4e174ebd2bf62544854ecf37e7"},
    {file = "bottle-0.12.18.tar.gz", hash = "sha256:0819b74b145a7def225c0e83b16a4d5711fde751cd92bae467a69efce720f69e"},
]
gunicorn = [
    {file = "gunicorn-20.0.4-py2.py3-none-any.whl", hash = "sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c"},
    {file = "gunicorn-20.0.4.tar.gz", hash = "sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626"},
]
redis = [
    {file = "redis-3.5.3-py2.py3-none-any.whl", hash = "sha256:432b788c4530cfe16d8d943a09d40ca6c16149727e4afe8c2c9d5580c59d9f24"},
    {file = "redis-3.5.3.tar.gz", hash = "sha256:0e7e0cfca8660dea8b7d5cd8c4f6c5e29e11f31158c0b0ae91a397f00e5a05a2"},
]

@abn
Copy link
Member Author

abn commented Oct 12, 2020

@zyv is the pyproject.toml available for that?

@zyv
Copy link
Contributor

zyv commented Oct 12, 2020

@abn

[tool.poetry]
name = "bruh"
version = "0.1.0"
description = "Service that implements REST-ful API service"
authors = ["Ilya Breitburg <me@breitburg.com>"]

[tool.poetry.dependencies]
python = "3.8.6"
redis = "^3.5.3"
bottle = "^0.12.18"
gunicorn = "^20.0.4"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

abn and others added 8 commits October 13, 2020 20:36
This change, moves common functionality from exporter into the locker
to allow for sharing of logic that can generate `DependencyPackage`
instances given a a list of requirements using the lock data.
This change ensures that markers are propagated from top level
dependencies to the deepest level by walking top to bottom instead of
iterating over all available packages.

In addition, we also compress any dependencies with the same name and
constraint to provide a more concise representation.

Resolves: python-poetry#3112 #3160
Previously, when determining nested dependencies, the check for
activated extras/features of top level dependencies were done after the
nested dependencies were processed. This lead to exports containing
in active extras. This change resolves this by pre-selecting top level
packages prior to identifying nested dependencies.
@abn abn merged commit ed43d94 into python-poetry:1.1 Oct 14, 2020
@abn abn deleted the cleanup/exports branch October 14, 2020 15:33
@zyv
Copy link
Contributor

zyv commented Oct 14, 2020

❤️

@marcinzaremba
Copy link

marcinzaremba commented Jan 14, 2021

This change seems to introduce a serious incompatibility in regards to how an export file looks for local file dependencies. Namely instead of using a path exactly how it is specified in pyproject.toml it is always transformed into an absolute one which makes a resulting requirements.txt file less portable as it in corporates an absolute path that is dependant of a running environment.

For example for Poetry 1.1.2:
Having a local file dependency declaration pyproject.toml:

telemetry = { file = "vendor/telemetry-1.1.1-py3-none-any.whl" }

would yield:

telemetry @ vendor/telemetry-1.1.1-py3-none-any.whl \
    --hash=sha256:777828bea2a9b9b4125cedcb044afdb74b1dc8f3cc35c04b86ab072caf09d9d2

On the other hand after these changes (for Poetry 1.1.3) resulting requirements.txt looks like:

telemetry @ file:///Users/<my-local-path>.../vendor/telemetry-1.1.1-py3-none-any.whl; python_version >= "3.7" and python_version < "4.0" \
    --hash=sha256:777828bea2a9b9b4125cedcb044afdb74b1dc8f3cc35c04b86ab072caf09d9d2

Is this an expected change/behavior that we need to get used to for the following versions or is it a regression in comparison with earlier versions?

Copy link

github-actions bot commented Mar 1, 2024

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected kind/refactor Pulls that refactor, or clean-up code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants