Unable to install packages from Azure DevOps private repo, 401 unauthorized

[FilBot3]

I am trying to install from an Azure DevOps Artifacts Python feed. I can publish to the feed if I use the following:

poetry publish \
  --repository 'azure' \
  --username 'orgname' \
  --password "$(cat ~/azdo_pat.txt)" \
  --build

Which I set up by doing:

poetry config \
  --local \
  repostiory.azure \
  https://pkgs.dev.azure.com/${ORG_NAME}/${PROJECT_NAME}/_packaging/python-azure-artifacts/pypi/simple/

So then my poetry.toml has:

[repositories]
[repositories.azure]
url = "https://pkgs.dev.azure.com/${ORG_NAME}/${PROJECT_NAME}/_packaging/python-azure-artifacts/pypi/simple/"

My pyproject.toml has this section added to it.

[[tool.poetry.source]]
name = "azure"
url = "https://pkgs.dev.azure.com/${ORG_NAME}/${ROFJECT_NAME}/_packaging/python-azure-artifacts/pypi/simple/"
secondary = true

Now, when ever I attempt to do anything with a package, I get 401 unauthorized errors. I had issues before with publishing to the feed when configuring the http-basic.azure before.

poetry add azdo                                                                                           2 ↵

RepositoryError

401 Client Error:  for url: https://pkgs.dev.azure.com/${ORG_NAME}/${PROJECT_NAME}/_packaging/python-azure-artifacts/pypi/simple/azdo/

at ~/.poetry/lib/poetry/repositories/legacy_repository.py:393 in _get
389│             if response.status_code == 404:
390│                 return
391│             response.raise_for_status()
392│         except requests.HTTPError as e:
→ 393│             raise RepositoryError(e)
394│
395│         if response.status_code in (401, 403):
396│             self._log(
397│                 "Authorization error accessing {url}".format(url=url), level="warn"

What do I need to do to make the add or install portion work now?

My pyproject.toml

[tool.poetry]
name = "developer.dudley.terraform_stuff"
version = "0.1.0"
description = "Python Tasks for the project"
authors = ["Phillip Dudley <Predatorian3@gmail.com>"]
license = "MIT"

[[tool.poetry.source]]
name = "azure"
url = "https://pkgs.dev.azure.com/${ORG_NAME}/${ROFJECT_NAME}/_packaging/python-azure-artifacts/pypi/simple/"
secondary = true

[tool.poetry.dependencies]
python = "^3.8"
invoke = "^1.4.1"

[tool.poetry.dev-dependencies]
keyring = "^21.5.0"
artifacts-keyring = "^0.2.10"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Poetry Version

poetry -V
Poetry version 1.1.4

[ariebovenberg]

@FilBot3 looks similar to #3303. I'm having this problem as well. It looks like there already is a PR in the works: #3337

[martinVyoo]

I am having the same issue
I don't think this is related, the issue in #3303 explains a 403 for a non-existing package, but in this case, the package exists, but poetry is not able to authenticate

[martinVyoo]

using pip with the --index-url option works, along with artifacts-keyring, as suggested in the Azure Artifacts documentation
For now, I suggest exporting your poetry lock to a requirements.txt and installing with pip from that
If you are installing a new package, install directly with pip and add to your pyproject.toml

[martinVyoo]

It looks related to #3216 to me, but all the workarounds mentioned in the thread do not work for me

[martinVyoo]

It worked by configuring poetry with poetry config repositories.<repository name> <repository url> and poetry config http-basic.<repository name> user pass,
In this case, the user is my username before the @ (for example, if your username on Azure artifacts is john@doe.com, then the username to put is john), and the password is the generated PAT token

Looks like poetry is not using keyring correctly for authentication, or artifacts-keyring is not compatible in some way

[bneijt]

Opened up a PR. Another work-around is calling poetry config with the creds from the keyring. Make sure you have artifacts-keyring installed and run:

#!/usr/bin/env python3
import toml
from keyring import get_credential
import subprocess

with open("pyproject.toml", "r") as pyproject_file:
    project = toml.load(pyproject_file)
    for source in project["tool"]["poetry"]["source"]:
        name, url = source.get("name"), source.get("url")
        if name and url:
            creds = get_credential(url, None)
            if creds:
                print(f"Setting poetry auth config '{name}'")
                subprocess.run(
                    [
                        "poetry",
                        "config",
                        "http-basic." + name,
                        creds.username,
                        creds.password,
                    ]
                )
            else:
                print(f"Could not find creds for '{name}'")

[Darsstar]

I created a new PR that could solve this issue, it doesn't any new caching since the Authenticator does that already.

Plus I want to mention a different potential workaround: creating a keyring poetry bridge keyring implementation. A new keyring backend that detects poetry lookups and then retrieves the repo url and do a recursive pip style keyring lookup based on the full url and netloc. I have not created that, so I can't say with complete confidence it will work because I don't know recursive keyring lookups are supported or not.

I should probably mention the azure-devops-artifacts-helpers package as well for people finding this issue while trying to figure out how to use artifacts-keyring with poetry or pipenv since it provides a virtualenv seeder that installs artifacts-keyring in new virtualenv since I came across it very late during my own googling. For pipenv that package would help a lot, for poetry not as much.

[mpicard]

Is there a proper workaround or fix for this? This is closed but I still have this issue, how did people fix this? I have configured using poetry config -- http-basic.<repository name> user pass correctly.

[Darsstar]

I have colleages run the script on https://pypi.org/project/keyring-subprocess/, which is my library. Which installs Pipx and then then installs keyring-subprocess in a way that it is available for all Pipx venvs. As if you ran pipx inject <tool> keyring-subprocess for every tool you installed with Pipx, like Poetry. Then it installs keyring and injects artifacts-keyring, keyring-subprocess and keyring-subprocess-landmark into it. Just run pipx install poetry after that.

Installing artifacts-keyring should also do the trick: poetry plugin add artifacts-keyring.

PS. you might want to read the new section of the Pip documentation on authentication, curtosy of your's truely. (It applies to Pip 23.1 and above.) The username portion of the URL should be VssSessionToken, as that is the only value artifacts-keyring accepts when using keyring via the CLI.
PPS. there is a missing semi-colon on the line that starts with $venv = that might be required, but I believe it not to be.

[mpicard]

I tried this artifacts-keyring script above, but it ask for interactive session i.e. open a browser, does anyone know how to configure it in automation/azure devops pipelines?

[Darsstar]

I tried this artifacts-keyring script above, but it ask for interactive session i.e. open a browser, does anyone know how to configure it in automation/azure devops pipelines?

artifacts-keyring is a wrapper of https://github.com/microsoft/artifacts-credprovider. So you use the NuGetAuthenticate task to set the right environment variables. Or you study the documentation and set them yourself.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.