Mysterious ECONNREFUSED within docker build

[youmustfight]

Hey folks, loving this project but running into a mystery as I take it from my local monorepo dev environment into CI. The challenge I'm having is doing package installs when within a docker build command.

At the start of CI I'm running Verdaccio as its own container (built via docker-compose if that matters).

CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                    NAMES
b87da3d1bacb        verdaccio/verdaccio:5   "uid_entrypoint /bin…"   About an hour ago   Up About an hour    0.0.0.0:4873->4873/tcp   project_verdaccio_1

Then I try to build a backend API via docker build. The port linking and registry const are passed in like so:

docker build -t api \
    --build-arg "NPM_CONFIG_REGISTRY=http://verdaccio:4873" \
    --network project_default \
    ./backends

Then in my Dockerfile, there is a step that looks like this (just doing a single package install without lock file to minimize variables):

RUN echo "\"registry\" \"${NPM_CONFIG_REGISTRY}\"" > .yarnrc && yarn install --update-checksums

When I do this command, I can see that the running Verdaccio container is getting the request, and seemingly is fine with it and responding (I've also deleted /storage/lodash to do fresh 200 streaming):

 http --- 172.18.0.3 requested 'GET /lodash'
 info --- auth/allow_action: access granted to: undefined
 info --- allowed access for lodash
 info --- making request: 'GET https://registry.npmjs.org/lodash'
 http --- 304, req: 'GET https://registry.npmjs.org/lodash' (streaming)
 http --- 304, req: 'GET https://registry.npmjs.org/lodash', bytes: 0/0
 http --- 200, user: null(172.18.0.3), req: 'GET /lodash', bytes: 0/22363

BUT in docker build, the command totally bails :(

warning upsolve_backends@0.1.0: No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "http://0.0.0.0:4873/lodash/-/lodash-4.17.21.tgz: connect ECONNREFUSED 0.0.0.0:4873".
info If you think this is a bug, please open a bug report with the information provided in "/usr/src/backends/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
The command '/bin/sh -c echo '"registry" "NPM_CONFIG_REGISTRY"' > .yarnrc && yarn add lodash && yarn install lodash' returned a non-zero code: 1

What could be causing the ECONNREFUSED? Verdaccio looks like its sending a 200 request when its trying to do an install and I'm seeing the containers when I do a docker network inpsect. Are there other logs or something I should look for?

Update: Just realized how to exec into Verdaccio lol

🙏🙏🙏🙏

Versions:
Verdaccio: 5.0.1
Node base image: 12.18.0
NPM: 6.14.4
Yarn: 1.22.10
CircleCI machine executor: ubuntu-1604:202004-01

config.yaml

storage: /verdaccio/storage
plugins: /verdaccio/plugins
auth:
  htpasswd:
    file: ./htpasswd
uplinks:
  npmjs:
    url: https://registry.npmjs.org/
listen:
  - 0.0.0.0:4873
packages:
  '@upsolve/*':
    access: $all
    publish: $all
  "@*/*":
    access: $all
    publish: $authenticated
    proxy: npmjs
  "**":
    access: $all
    proxy: npmjs
logs:
  - { type: stdout, format: pretty, level: http }

[youmustfight]

Alright, so I added the listen property to my config after seeing this issue/discussion #356 (comment) and now it seems to work!

listen:
  - 0.0.0.0:4873

[juanpicado]

Regarding the listen, maybe worth improve docs here https://verdaccio.org/docs/en/docker#docker-and-custom-port-configuration what do you think?

To improve the performance, you also might want to disable the pretty on the logs, is pretty expensive and not recommendable for CI, also reduce the level to warn.

logs:
  - { type: stdout, format: json, level: warn }

Sometimes package managers struggle to perform request since the registry is busy, I do this https://github.com/verdaccio/verdaccio/blob/5.x/.github/workflows/e2e-angular-cli-workflow.yml#L70 to avoid failures

   npm config set fetch-retries="5"
   npm config set fetch-retry-factor="50"
   npm config set fetch-retry-mintimeout="20000"
   npm config set fetch-retry-maxtimeout="80000"

Try to reduce the work that registry needs to do on CI, maybe even disable cache tarballs, since I guess is an env you want to discard when the builds end.

[youmustfight]

Oh ya, broader examples in that section would be great. I came into experience a little rusty around docker networking. I remember reading that but didn't really know how it would play out. I guess what confused me about the situation above was that Verdaccio was seeing the request come in, so in a sense it was listening already but it was dropping the request in a way that didn't result in an obvious error being thrown. It was just a sort of stalemate.

UPDATE: OH. It just occurred to me that those http request statuses are to the uplink NPM registry, not responses to the requesting local service!!!!!

Awesome points on performance, will definitely implenent. Thanks! I don't think we're at a scale or volume where that's too heavy an issue but great to know.