poetry install --extras doesn't work

[anentropic]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 10.14.6
• Poetry version: 1.0.3
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/anentropic/661458664b20923cf07f7e377972870b

Issue

The docs show a pyproject.toml:

[tool.poetry]
name = "awesome"

[tool.poetry.dependencies]
# These packages are mandatory and form the core of this package’s distribution.
mandatory = "^1.0"

# A list of all of the optional dependencies, some of which are included in the
# below `extras`. They can be opted into by apps.
psycopg2 = { version = "^2.7", optional = true }
mysqlclient = { version = "^1.3", optional = true }

[tool.poetry.extras]
mysql = ["mysqlclient"]
pgsql = ["psycopg2"]

And then an example of installing the "extras":

poetry install --extras "mysql pgsql"

But pasting the relevant part of this example into my own pyproject.toml (see gist) it doesn't work:

$ poetry install --extras "mysql pgsql"
Installing dependencies from lock file

[ValueError]
Extra [mysql] is not specified.

[anentropic]

is there some caching of the pyproject.toml?

I had this problem with the extras I originally wanted to install, so I tried with the exact example from the docs before posting this issue

Neither worked (as shown above)...

Now I have gone back and replaced the docs example with my intended extras again:

[tool.poetry.dependencies]
...
ipython = {version=">=7.10.1", optional=true}
ipdb = {version="*", optional=true}

[tool.poetry.extras]
devtools = ["ipython", "ipdb"]

and it works now 🤔

$ poetry install --extras devtools
Installing dependencies from lock file


Package operations: 2 installs, 0 updates, 0 removals

  - Installing ipython (7.12.0)
  - Installing ipdb (0.12.3)
  - Installing waterloo (0.1.0)

After all that I did a poetry build and I realise what I am trying to do is not what extras is for anyway 🤦‍♂ ... extras is for the square-bracketed optional extra deps when installing my lib from pypi

I see now what I really wanted was #1644 ... I commonly have three groups of deps: "install", "test" (what's needed to run tests, what pipenv and poetry call dev dependencies) and "devtools" which are stuff like ipython and ipdb and other utils that are only useful to me working on the project and don't need to be installed on the CI

[anentropic]

what I am loving is no waiting minutes for pipenv to 'lock' though 👍

[JBKahn]

I'm having a similar issue when I go from 1.0.0 to 1.0.1-1.0.3, extras are removed from my lockfile. I tried clearing the cache but that doesn't seem to help.

[JBKahn]

It still has a [package.extras] section like this, which is correct:

[package.extras]
first = ["A (>=3.0.0)"]
second = ["B (>=10.0.0)", "C (<3.0.0)", "D (>=6.0.0)"]

and

[package.dependencies.D]
optional = true
version = ">=6.0.0"

etc for each but D is otherwise omitted from the lockfile.

[JBKahn]

D is also in the dependancy tree, so it's just whatever is exporting the lockfile is the issue

[JBKahn]

poetry lock -vvv also says:
1: selecting D (10.0.0)

So it's just an issue with lockfile generation for me

[JBKahn]

I've had to manually resort to copying D into my pyproject.toml for now. Without it, it omits from the lockfile and then refuses to install claiming it doesn't know how to resolve the dependancy that it omitted. Very strange.

[dekoza]

Any progress on this bug? It's a blocker.

[JBKahn]

Sadly I no longer have access to the old projects I was working with now that I'm with a new company so I'd need to find a way to reproduce it from scratch.

[dekoza]

My django-getpaid project has a test extra dependency group declared among others.

poetry install does not install pytest

poetry install -E test does not install pytest either.

[matthijskooijman]

This might be a duplicate of #1145? Though that shows the extras being omitted, while this issue shows an error message about the extras being unknown...

[wshayes]

Looks like you can install multiple dependencies in a single set using the following approach:

poetry add -E api fastapi
poetry add -E api gunicorn
poetry add -E api uvicorn

resulting in the following changes to the pyproject.toml file:

fastapi = {version = "^0.61.0", extras = ["api"]}
gunicorn = {version = "^20.0.4", extras = ["api"]}
uvicorn = {version = "^0.11.8", extras = ["api"]}

But running poetry update; poetry install -E API doesn't work yet.

You need to add the following entry to the pyproject.toml file (NOTE you have to add the requirements specifier otherwise what gets passed to pip install complains and errors out):

[tool.poetry.extras]
api = ["fastapi^0.61.0", "gunicorn^20.0.4", "uvicorn^0.11.8"]

and then poetry update; poetry install -E API works fine.

[ndevenish]

Looks like you can install multiple dependencies in a single set using the following approach:

Thanks - it's listed on poetry add which caused me to try and use it. But this (still!) doesn't appear to actually work.

[aprilahijriyan]

I highly recommend writing extras syntax like this:

[tool.poetry.extras.full]

django-extensions = {version = "^3.1.3" }
drf-yasg = {version = "^1.20.0"}

[KotlinIsland]

I highly recommend writing extras syntax like this:

[tool.poetry.extras.full]

django-extensions = {version = "^3.1.3" }
drf-yasg = {version = "^1.20.0"}

@aprilahijriyan Is that a feature suggestion? because I think that's invalid:

>poetry install -E full

  RuntimeError

  The Poetry configuration is invalid:
    - [extras.full] {'django-extensions': {'version': '^3.1.3'}, 'drf-yasg': {'version': '^1.20.0'}} is not of type 'array'

[aprilahijriyan]

@KotlinIsland
Yes it's a feature suggestion, for more details on defining extras see here https://python-poetry.org/docs/pyproject/#extras

[mkniewallner]

Is this issue still relevant? With the pyproject.toml given in the description (and the addition of some keys required by newer versions of Poetry) it seems to work just fine.

[mkniewallner]

Since it seems to work just fine on recent Poetry versions, and we didn't hear back from the author, closing the issue.

If you still reproduce on recent versions, feel free to reopen it.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.