Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error: connect ECONNREFUSED 127.0.0.1:443 for downloading a binary #615

Closed
jchiare opened this issue Mar 10, 2022 · 9 comments
Closed

Error: connect ECONNREFUSED 127.0.0.1:443 for downloading a binary #615

jchiare opened this issue Mar 10, 2022 · 9 comments
Labels
bug Needs More Information

Comments

@jchiare
Copy link

jchiare commented Mar 10, 2022 •
edited
Loading

Versions

  • NodeJS: v16.13.1
  • mongodb-memory-server-*: ^8.4.0
  • mongodb(the binary version): 5.0.6
  • system: MacOS version 12.1 M1

package: mongo-memory-server

What is the Problem?

It's not possible to start a mongoServer as an error on the https.get call to the mongodb binary happens 100% of the time, if you don't have the binary installed locally.

I've tried different MongoMemoryServer versions (4.2.8, 5.0.3) etc. Accessing the link directly (via browser) works and installs the binary (As expected, see why I think it happens)

Error: Couldnt download "https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-5.0.3.tgz"! connect ECONNREFUSED 127.0.0.1:443

Code Example

  const mongoServer = await MongoMemoryServer.create({
      binary: {
          version: '5.0.3'
      }
  });

  const mongoUri = mongoServer.getUri();

Debug Output

Debug Output 
  MongoMS:ResolveConfig Debug Mode Enabled, through Environment Variable +0ms
  MongoMS:ResolveConfig findPackageJson: Found package.json at "/Users/jay/Code/work/integration_middleware/package.json" +1ms
  MongoMS:ResolveConfig Debug Mode Enabled, through package.json +1ms


  MongoMS:MongoMemoryServer create: Called .create() method +0ms
  MongoMS:MongoMemoryServer Mongo[unknown]: start: Called .start() method +0ms
  MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Called MongoMemoryServer._startUpInstance() method +0ms
  MongoMS:MongoMemoryServer Mongo[unknown]: getStartOptions: forceSamePort: false +0ms
  MongoMS:MongoMemoryServer Mongo[unknown]: _startUpInstance: Creating new MongoDB instance with options: {
  instance: {
    port: 51442,
    dbName: '',
    ip: '127.0.0.1',
    storageEngine: 'ephemeralForTest',
    replSet: undefined,
    dbPath: '/var/folders/qd/m2hpd2x91vzffsqx9jlfjm6h0000gn/T/mongo-mem--19030-qmCgrgmz2k4O',
    tmpDir: {
      name: '/var/folders/qd/m2hpd2x91vzffsqx9jlfjm6h0000gn/T/mongo-mem--19030-qmCgrgmz2k4O',
      removeCallback: [Function: _cleanupCallback]
    },
    args: undefined,
    auth: undefined
  },
  binary: { version: '5.0.3' },
  spawn: undefined
} +2ms
  MongoMS:MongoInstance create: Called .create() method +0ms
  MongoMS:MongoInstance Mongo[51442]: start +0ms
  MongoMS:MongoBinary getPath +0ms
  MongoMS:DryMongoBinary generateOptions +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +1ms
  MongoMS:DryMongoBinary generatePaths {
  version: '5.0.3',
  downloadDir: '',
  os: { os: 'darwin' },
  arch: 'arm64',
  systemBinary: ''
} +0ms
  MongoMS:DryMongoBinary getBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Paths: {
  legacyHomeCache: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  modulesCache: '/Users/jay/Code/work/integration_middleware/node_modules/.cache/mongodb-memory-server/mongod-arm64-darwin-5.0.3',
  relative: '/Users/jay/Code/work/integration_middleware/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  resolveConfig: ''
}  +0ms
  MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "5.0.3" was found +1ms
  MongoMS:DryMongoBinary generateDownloadPath: using global (preferGlobal) "/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3" +0ms
  MongoMS:MongoBinary getPath: MongoBinary options: {
  "version": "5.0.3",
  "downloadDir": "/Users/jay/.cache/mongodb-binaries",
  "os": {
    "os": "darwin"
  },
  "arch": "arm64",
  "systemBinary": "",
  "platform": "darwin",
  "checkMD5": false
} +2ms
  MongoMS:DryMongoBinary locateBinary: Trying to locate Binary for version "5.0.3" +0ms
  MongoMS:DryMongoBinary generateOptions +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +0ms
  MongoMS:DryMongoBinary generatePaths {
  version: '5.0.3',
  downloadDir: '/Users/jay/.cache/mongodb-binaries',
  os: { os: 'darwin' },
  arch: 'arm64',
  systemBinary: ''
} +0ms
  MongoMS:DryMongoBinary getBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Paths: {
  legacyHomeCache: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  modulesCache: '/Users/jay/Code/work/integration_middleware/node_modules/.cache/mongodb-memory-server/mongod-arm64-darwin-5.0.3',
  relative: '/Users/jay/Code/work/integration_middleware/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  resolveConfig: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3'
}  +0ms
  MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "5.0.3" was found +1ms
  MongoMS:DryMongoBinary generateDownloadPath: using resolveConfig (DOWNLOAD_DIR) "/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3" +0ms
  MongoMS:DryMongoBinary locateBinary: running generateDownloadPath +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +0ms
  MongoMS:DryMongoBinary generatePaths {
  version: '5.0.3',
  downloadDir: '/Users/jay/.cache/mongodb-binaries',
  os: { os: 'darwin' },
  arch: 'arm64',
  systemBinary: ''
} +0ms
  MongoMS:DryMongoBinary getBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Paths: {
  legacyHomeCache: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  modulesCache: '/Users/jay/Code/work/integration_middleware/node_modules/.cache/mongodb-memory-server/mongod-arm64-darwin-5.0.3',
  relative: '/Users/jay/Code/work/integration_middleware/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  resolveConfig: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3'
}  +0ms
  MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "5.0.3" was found +0ms
  MongoMS:DryMongoBinary generateDownloadPath: using resolveConfig (DOWNLOAD_DIR) "/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3" +0ms
  MongoMS:DryMongoBinary locateBinary: could not find a existing binary +0ms
  MongoMS:MongoBinary getPath: "RUNTIME_DOWNLOAD" is "true", trying to download +1ms
  MongoMS:MongoBinary download +0ms
  MongoMS:MongoBinary download: Waiting to acquire Download lock for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +1ms
  MongoMS:LockFile lock: Locking file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +0ms
  MongoMS:LockFile checkLock: for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" with uuid: "undefined" +0ms
  MongoMS:LockFile createLock: trying to create a lock file for "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +0ms
  MongoMS:LockFile createLock: Lock File Created for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +8ms
  MongoMS:MongoBinary download: Download lock acquired +8ms
  MongoMS:MongoBinary download: Adding version 5.0.3 to cache +0ms
  MongoMS:MongoBinaryDownload getMongodPath +0ms
  MongoMS:DryMongoBinary generateOptions +9ms
  MongoMS:DryMongoBinary generateDownloadPath: Generating Download Path, preferGlobal: "true" +0ms
  MongoMS:DryMongoBinary generatePaths {
  version: '5.0.3',
  downloadDir: '/Users/jay/.cache/mongodb-binaries',
  os: { os: 'darwin' },
  arch: 'arm64',
  systemBinary: ''
} +0ms
  MongoMS:DryMongoBinary getBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generatePaths: resolveConfigValue is not empty +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:DryMongoBinary generateDownloadPath: Paths: {
  legacyHomeCache: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  modulesCache: '/Users/jay/Code/work/integration_middleware/node_modules/.cache/mongodb-memory-server/mongod-arm64-darwin-5.0.3',
  relative: '/Users/jay/Code/work/integration_middleware/mongodb-binaries/mongod-arm64-darwin-5.0.3',
  resolveConfig: '/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3'
}  +0ms
  MongoMS:DryMongoBinary generateDownloadPath: no existing binary for version "5.0.3" was found +1ms
  MongoMS:DryMongoBinary generateDownloadPath: using resolveConfig (DOWNLOAD_DIR) "/Users/jay/.cache/mongodb-binaries/mongod-arm64-darwin-5.0.3" +0ms
  MongoMS:DryMongoBinary getBinaryName +0ms
  MongoMS:DryMongoBinary combineBinaryName +0ms
  MongoMS:MongoBinaryDownload startDownload +2ms
  MongoMS:MongoBinaryDownloadUrl getArchiveNameOsx: Arch is "arm64", using x64 binary +0ms
  MongoMS:MongoBinaryDownloadUrl Using "mongodb-macos-x86_64-5.0.3.tgz" as the Archive String +0ms
  MongoMS:MongoBinaryDownloadUrl Using "https://fastdl.mongodb.org" as the mirror +0ms
  MongoMS:MongoBinaryDownload download +3ms
  MongoMS:MongoBinaryDownload download: Downloading: "https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-5.0.3.tgz" +0ms
  MongoMS:MongoBinaryDownload httpDownload +0ms
  MongoMS:MongoBinaryDownload httpDownload: trying to download "https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-5.0.3.tgz" +0ms
Couldnt download "https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-5.0.3.tgz"! connect ECONNREFUSED 127.0.0.1:443
  MongoMS:MongoBinary download: Removing Download lock +15ms
  MongoMS:LockFile unlock: Unlocking file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +15ms
  MongoMS:LockFile checkLock: for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" with uuid: "fc0b7120-1aa4-49b3-82bb-3b25591ad928" +0ms
  MongoMS:LockFile checkLock: Lock File Already exists, and is for *this* process, with uuid: "fc0b7120-1aa4-49b3-82bb-3b25591ad928" +2ms
  MongoMS:LockFile unlock: Lock Status was "availableInstance" for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +0ms
  MongoMS:LockFile unlockCleanup: for file "/Users/jay/.cache/mongodb-binaries/5.0.3.lock" +0ms
  MongoMS:MongoBinary download: Download lock removed +2ms
  MongoMS:MongoMemoryServer Mongo[unknown]: stop: Called .stop() method +31ms
  MongoMS:MongoMemoryServer Mongo[unknown]: stop: "instanceInfo" is not defined (never ran?) +0ms

Do you know why it happens?

The correct URL is created (const downloadUrl = this.assignDownloadingURL(url);) but is not used later by the actual HTTPS request here.
@jchiare jchiare added the bug label Mar 10, 2022
@jchiare
Copy link
Author

jchiare commented Mar 10, 2022

I don't have permission to push so I can't create a PR:
ERROR: Permission to nodkz/mongodb-memory-server.git denied to jchiare.

@hasezoey
Copy link
Member

hasezoey commented Mar 10, 2022 •
edited
Loading

I don't have permission to push so I can't create a PR:

this is to be expected, unless you are a maintainer of this package (eg have write permissions to nodkz/mongodb-memory-server), you will need a fork to create a PR

The correct URL is created (const downloadUrl = this.assignDownloadingURL(url);)

this understanding is wrong, assignDownloadingURL is to assign parameter 1 (named url) to a value in the class itself, not to / get the url, see

https://github.com/nodkz/mongodb-memory-server/blob/bbddd350626950dffa89592e740420d31f07df93/packages/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts#L520-L524

this function returns a string, but http(s).get also accepts a URL object, which should always be preferred as input

connect ECONNREFUSED 127.0.0.1:443

it seems like your request was forwarded to the localhost ip and https port, which very likely does not exist, or is at least not what is expected

things you can do / check:

  • your firewall on your system
  • your firewall on any system inbetween you and the internet (like router)
  • if you have something like pihole, check that it is not interfering
  • should it run in a sandbox, maybe check the permissions the sandbox has (like firejail / flatpak / a container)
  • try to upgrade your nodejs version to the latest available (and / or try to downgrade)
  • reset your system (not recommended)

Note: i do not know how Mac systems operate, so i cannot help in more detail

Also this issue seems to be related to #600 (where no more information was given and then closed, this issue here (#615) will remain open as a continuation)

PS: i changed the title to better reflect what this issue is about

@hasezoey hasezoey changed the title wrong URL value used for httpDownload method in MongoBinaryDownload Error: connect ECONNREFUSED 127.0.0.1:443 for downloading a binary Mar 10, 2022
@jchiare
Copy link
Author

jchiare commented Mar 10, 2022 •
edited
Loading

Thank you for your reply!

your firewall on your system

No firewall

your firewall on any system inbetween you and the internet (like router)

No other firewall (that I know of). This issue also happens to other colleagues (on mac M1) as well, and we are all working from home from different cities.

if you have something like pihole, check that it is not interfering

I don't

should it run in a sandbox, maybe check the permissions the sandbox has (like firejail / flatpak / a container)

I'm not running it in a sandbox - running it on my work mac with a residential ISP, nothing special router, no container

try to upgrade your nodejs version to the latest available (and / or try to downgrade)

I updated to v16.14.0 (latest LTS) and the same issue occurs

but http(s).get also accepts a URL object, which should always be preferred as input

If I look at the docs from nodejs, string is listed as a param first, so I'd expect that a string is preferred (from a node perspective) - but I'm definitely not sure.

The issue also persists in our CI/CD platform which uses the basic node docker image FROM node:16. I'll try to create a sample project to reproduce this issue on that image, so you can (maybe) reproduce locally.

@hasezoey
Copy link
Member

If I look at the docs from nodejs, string is listed as a param first, so I'd expect that a string is preferred (from a node perspective) - but I'm definitely not sure.

see docs: If options is a string, it is automatically parsed with [new URL()](https://nodejs.org/docs/latest-v16.x/api/url.html#new-urlinput-base)

in our case url is already a URL object, set with settings that can be defined with config / env variables

could you maybe give me the output of #600 (comment) ? (placed in the MMS node_module on your system, just before this line(s))

@jchiare
Copy link
Author

jchiare commented Mar 10, 2022

Thanks again for the reply - after I added the comment, it works now. So weird because I did not change anything. The code (same code) also works in our CI/CD.

I'll confirm with my colleagues to see if it works for them again. I think the only change that I've done is:

  • Delete my node_modules folder
  • run npm run install again

@hasezoey
Copy link
Member

Delete my node_modules folder
run npm run install again

definitely weird

if i understand correctly, you dont have the error anymore and cannot reproduce it anymore? if so, i would close this again because of insufficient information

@jchiare jchiare closed this as completed Mar 10, 2022
@caub
Copy link
Contributor

caub commented May 13, 2022 •
edited
Loading

That error happens repeatdly in gitlab-ci

node: 'v16.15.0'
npm: '8.5.5',
mongodb-memory-server: ^8.5.2

Couldnt download "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian10-5.0.3.tgz"! connect ECONNREFUSED 127.0.0.1:443
Starting the MongoMemoryServer Instance failed, enable debug log for more information. Error:
 Error: connect ECONNREFUSED 127.0.0.1:443
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 443
}

but whenever I clear the gitlab CI cache, and retry manually the job, it works

What's annoying is when another job is later automatically ran, for some reason it'll fail again

But I think it's more a gitlab issue than this package, just leaving this here for the record

@pavlonadolynskyi
Copy link

this can be related to other libraries, try to debug and find breaks it. For me it was old mailgun or mock-jwks

@pavlonadolynskyi pavlonadolynskyi mentioned this issue Jul 25, 2022
Closed
@mzoellner
Copy link

mzoellner commented Sep 29, 2022 •
edited
Loading

Just in case you stumble across this. For us the culprit was an old node-agent-base dependency (Related Issue). Looks like it's patching some basic node functions, which breaks (or broke) a lot of libraries.

In our case this dependency was installed due to an old mailgun-js version (thanks @pavlonadolynskyi for the hint).

We migrated from mailgun-js to mailgun.js and the problem went away.

Edit:
It turns out that moving the MONGOMS_DOWNLOAD_URL/MONGOMS_VERISON environment variables to before the npm ci command in our Dockerfile also does the trick.
Before we set those variables just before running npm run test, so the download of the binary happened inside the test run and after a require() to the faulty package which caused weird side effects. When those environment variables are set before npm ci, the binary is downloaded during npm package install and then cached, so it is available when running tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Needs More Information
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@mzoellner @pavlonadolynskyi @caub @hasezoey @jchiare