Poetry export does not respect groups defined in lockfile

[jkgenser]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 22.04
• Poetry version: Poetry (version 1.2.0b2.dev0), Master@865aa94dd3393076f790d3351f6ae18483673e38
• Link of a Gist with the contents of your pyproject.toml file:

Issue

When running poetry export from the following pyproject.toml and poetry.lock, specifying various combinations of requirements do not result in an optional group excluded. The difference between the main group and the test group is that one of them specifies an extra from the dependency and one does not.

As you can see from the screenshot below, poetry install does what is expected given the poetry.lock and pyproject.toml. However, exporting to requirements.txt does not. Given the difference in behavior between install and export to requirements, seems like there's something broken unless the difference in behavior is intended. From what I can tell based on documentation, it should be possible to completely eject a poetry.lock file to requirements.txt since the export includes similar options to poetry install

Project installing

• pyproject.toml
• poetry.lock

git project being installed:

https://github.com/IndicoDataSolutions/poetry-install-from/blob/main/pyproject.toml, commit d6fb91081828c1644235b247f9757f1fb0f161a0

Poetry lock contents:

The poetry lock contents indicate that the extra was picked up correctly.

[[package]]
name = "mock-git-dep"
version = "0.1.0"
description = "Test exporting poetry with dependencies and not"
category = "main"
optional = false
python-versions = "~3.10"
develop = false

[package.dependencies]
pytest = {version = "^6.2.4", optional = true}

[package.extras]
test = ["pytest (>=6.2.4,<7.0.0)"]

[package.source]
type = "git"
url = "https://github.com/IndicoDataSolutions/poetry-install-from.git"
reference = "main"
resolved_reference = "d6fb91081828c1644235b247f9757f1fb0f161a0"

Result of running poetry export against the poetry.lock

I've experimented with various attempts like,

• not supplying any extra group args,
• --only main --> should only get the main group. but since my test group is defined as optional, it should not be picked up
• --without test --> If test wasn't optional, this should exclude test group.

(poetry)  ~/indico/poetry-export main poetry export -f requirements.txt --without-hashes
atomicwrites==1.4.0 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
attrs==21.4.0 ; python_version >= "3.10" and python_version < "3.11"
colorama==0.4.4 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
iniconfig==1.1.1 ; python_version >= "3.10" and python_version < "3.11"
mock-git-dep @ git+https://github.com/IndicoDataSolutions/poetry-install-from.git@main ; python_version >= "3.10" and python_version < "3.11"
packaging==21.3 ; python_version >= "3.10" and python_version < "3.11"
pluggy==1.0.0 ; python_version >= "3.10" and python_version < "3.11"
py==1.11.0 ; python_version >= "3.10" and python_version < "3.11"
pyparsing==3.0.9 ; python_version >= "3.10" and python_version < "3.11"
pytest==6.2.5 ; python_version >= "3.10" and python_version < "3.11"
toml==0.10.2 ; python_version >= "3.10" and python_version < "3.11"

(poetry)  ~/indico/poetry-export main poetry export -f requirements.txt --without-hashes --only main
atomicwrites==1.4.0 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
attrs==21.4.0 ; python_version >= "3.10" and python_version < "3.11"
colorama==0.4.4 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
iniconfig==1.1.1 ; python_version >= "3.10" and python_version < "3.11"
mock-git-dep @ git+https://github.com/IndicoDataSolutions/poetry-install-from.git@main ; python_version >= "3.10" and python_version < "3.11"
packaging==21.3 ; python_version >= "3.10" and python_version < "3.11"
pluggy==1.0.0 ; python_version >= "3.10" and python_version < "3.11"
py==1.11.0 ; python_version >= "3.10" and python_version < "3.11"
pyparsing==3.0.9 ; python_version >= "3.10" and python_version < "3.11"
pytest==6.2.5 ; python_version >= "3.10" and python_version < "3.11"
toml==0.10.2 ; python_version >= "3.10" and python_version < "3.11"

(poetry)  ~/indico/poetry-export main poetry export -f requirements.txt --without-hashes --without test
atomicwrites==1.4.0 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
attrs==21.4.0 ; python_version >= "3.10" and python_version < "3.11"
colorama==0.4.4 ; python_version >= "3.10" and python_version < "3.11" and sys_platform == "win32"
iniconfig==1.1.1 ; python_version >= "3.10" and python_version < "3.11"
mock-git-dep @ git+https://github.com/IndicoDataSolutions/poetry-install-from.git@main ; python_version >= "3.10" and python_version < "3.11"
packaging==21.3 ; python_version >= "3.10" and python_version < "3.11"
pluggy==1.0.0 ; python_version >= "3.10" and python_version < "3.11"
py==1.11.0 ; python_version >= "3.10" and python_version < "3.11"
pyparsing==3.0.9 ; python_version >= "3.10" and python_version < "3.11"
pytest==6.2.5 ; python_version >= "3.10" and python_version < "3.11"
toml==0.10.2 ; python_version >= "3.10" and python_version < "3.11"

[dimbleby]

It looks as though parsing the lockfile does not set .in_extras for the pytest dependency, which I think is what I would have expected to happen. That not-in-extras package is created here.

The export can be fixed by an uglier check that pytest is an extra, like this:

diff --git a/src/poetry/packages/locker.py b/src/poetry/packages/locker.py
index c2a6c35c..d2b3db63 100644
--- a/src/poetry/packages/locker.py
+++ b/src/poetry/packages/locker.py
@@ -319,8 +319,9 @@ class Locker:
             requirement.set_constraint(constraint)

             for require in locked_package.requires:
-                if require.in_extras and locked_package.features.isdisjoint(
-                    require.in_extras
+                if require.is_optional() and not any(
+                    require in locked_package.extras[feature]
+                    for feature in locked_package.features
                 ):
                     continue

But I'm not sure whether that's just putting a sticky plaster over the in_extras problem, and we'd be better arranging that pytest has .in_extras set all along.

Or perhaps I've misunderstood in_extras and it was only ever supposed to refer to the top-level project extras, not the extras of dependencies?

Hard to tell.

[jkgenser]

Would you be able to give me some guidance on where export ends up calling locked_repository fn that the line you reference is from in?(

poetry/src/poetry/packages/locker.py

Line 98 in 374cff5

def locked_repository(self) -> Repository:

)

I'd be happy to look into it with, and come up with a regression test case.

Edit: I found this in the poetry-export package, will keep tracing through the code: https://github.com/python-poetry/poetry-plugin-export/blob/main/src/poetry_plugin_export/command.py#L15

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.