Please utilize COURSIER_CREDENTIALS environment variable for all dependency retrievals

[johnsonjii]

When running Mill in an environment that can access Maven Central artifacts without authentication, build.sc scripts that use import $ivy statements and ivyDeps work correctly.

When running Mill in an environment that has to authenticate with the artifact repository, build.sc scripts fail to retrieve the requested artifacts. This is true for import $ivy statements, ivyDeps, and other artifacts that Mill is trying to fetch for itself.

In both kinds of environment, plain Ammonite scripts work correctly.

Given the following:

• Ammonite REPL & Script-Runner, 3.0.0-M2
• The Mill bootstrapping wrapper scrpit

.mill-version

0.11.12

build.sc

import mill._, scalalib._
import $ivy.`commons-io:commons-io:2.16.1`

object main extends JavaModule {
  def ivyDeps = Agg(ivy"commons-io:commons-io:2.16.1")
}

plain.sc

import $ivy.`commons-io:commons-io:2.16.1`
println("Done")

Note: Between each of the various experiments below, I throw away the Mill out directory and the Coursier cache directory to simulate an initial environment.

In an unrestricted environment:

$ export COURSIER_REPOSITORIES="ivy2Local|central"

Mill behaves correctly...

$ ./mill main.prepareOffline
[13/13] main.prepareOffline

and Ammonite behaves correctly...

$ amm plain.sc
https://repo1.maven.org/maven2/commons-io/commons-io/2.16.1/commons-io-2.16.1.pom
  100.0% [##########] 19.2 KiB (16.8 KiB / s)
https://repo1.maven.org/maven2/org/apache/commons/commons-parent/69/commons-parent-69.pom
  100.0% [##########] 75.2 KiB (238.0 KiB / s)
https://repo1.maven.org/maven2/org/apache/apache/31/apache-31.pom
  100.0% [##########] 23.0 KiB (117.2 KiB / s)
https://repo1.maven.org/maven2/org/junit/junit-bom/5.10.2/junit-bom-5.10.2.pom
  100.0% [##########] 5.5 KiB (28.6 KiB / s)
https://repo1.maven.org/maven2/commons-io/commons-io/2.16.1/commons-io-2.16.1-sources.jar
  100.0% [##########] 515.3 KiB (429.1 KiB / s)
https://repo1.maven.org/maven2/commons-io/commons-io/2.16.1/commons-io-2.16.1.jar
  100.0% [##########] 496.9 KiB (393.7 KiB / s)
Compiling plain.sc
Done

In a restricted environment:

Note: Host artifactory.example.com is used in the examples below in place of the real Artifactory server that I connected to. It mirrors Maven Central, but requires authentication.

$ export COURSIER_REPOSITORIES="ivy2Local|https://artifactory.example.com/artifactory/public"

Note: No credentials have been provided to Coursier.

Mill fails to get the JARs it needs. This is expected...

$ ./mill main.prepareOffline
[build.sc] [15/59] resolvedRunIvyDeps | Downloading [6/6] artifacts (~0/0 bytes)
1 targets failed
resolvedRunIvyDeps
Resolution failed for 3 modules:
--------------------------------------------
  com.lihaoyi:mill-moduledefs_2.13:0.10.9
	not found: ${HOME}/.ivy2/local/com.lihaoyi/mill-moduledefs_2.13/0.10.9/ivys/ivy.xml
	unauthorized: https://artifactory.example.com/artifactory/public/com/lihaoyi/mill-moduledefs_2.13/0.10.9/mill-moduledefs_2.13-0.10.9.pom (Artifactory Realm)
  commons-io:commons-io:2.16.1
	not found: ${HOME}/.ivy2/local/commons-io/commons-io/2.16.1/ivys/ivy.xml
	unauthorized: https://artifactory.example.com/artifactory/public/commons-io/commons-io/2.16.1/commons-io-2.16.1.pom (Artifactory Realm)
  org.scala-lang:scala-library:2.13.14
	not found: ${HOME}/.ivy2/local/org.scala-lang/scala-library/2.13.14/ivys/ivy.xml
	unauthorized: https://artifactory.example.com/artifactory/public/org/scala-lang/scala-library/2.13.14/scala-library-2.13.14.pom (Artifactory Realm)

--------------------------------------------

For additional information on library dependencies, see the docs at
https://mill-build.org/mill/Library_Dependencies.html

Ammonite also fails. This is also expected...

$ amm plain.sc
Failed to resolve ivy dependencies:Error downloading commons-io:commons-io:2.16.1
  not found: ${HOME}/.ivy2/local/commons-io/commons-io/2.16.1/ivys/ivy.xml
  unauthorized: https://artifactory.example.com/artifactory/public/commons-io/commons-io/2.16.1/commons-io-2.16.1.pom (Artifactory Realm)

Now with the correct credentials provided to Coursier. (In my experiment I set this variable with the correct values, the text below has fake values.)

$ export COURSIER_CREDENTIALS="artifactory.example.com(Artifactory Realm) fake_username:fake_password"

Ammonite successfully gets the JARs it needs...

$ amm plain.sc
https://artifactory.example.com/artifactory/public/commons-io/commons-io/2.16.1/commons-io-2.16.1.jar
  100.0% [##########] 496.9 KiB (256.8 KiB / s)
https://artifactory.example.com/artifactory/public/commons-io/commons-io/2.16.1/commons-io-2.16.1-sources.jar
  100.0% [##########] 515.3 KiB (220.6 KiB / s)
Compiling plain.sc
Done

... but Mill does not. This is a scenario that I believe should work correctly.

$ ./mill main.prepareOffline
[build.sc] [15/59] resolvedRunIvyDeps | Downloading [6/6] artifacts (~129393/129393 bytes)
1 targets failed
resolvedRunIvyDeps Failed to load dependencies
  unauthorized: https://artifactory.example.com/artifactory/public/commons-io/commons-io/2.16.1/commons-io-2.16.1.jar (Artifactory Realm)
  unauthorized: https://artifactory.example.com/artifactory/public/com/lihaoyi/mill-moduledefs_2.13/0.10.9/mill-moduledefs_2.13-0.10.9.jar (Artifactory Realm)
  unauthorized: https://artifactory.example.com/artifactory/public/org/scala-lang/scala-library/2.13.14/scala-library-2.13.14.jar (Artifactory Realm)

[alexarchambault]

Looking into this (respecting COURSIER_REPOSITORIES and COURSIER_CREDENTIALS )