Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: HTTP urls do not support query parameters and do not support extensionless URLs #5929

Closed
1 task
nicolo-ribaudo opened this issue Nov 3, 2023 · 4 comments · Fixed by #5931
Closed
1 task
Labels
bug Something isn't working reproducible This issue can be successfully reproduced

Comments

@nicolo-ribaudo
Copy link
Contributor

nicolo-ribaudo commented Nov 3, 2023

Self-service

  • I'd be willing to implement a fix

Describe the bug

I am trying to add this tgz as a dependency: https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a

However, Yarn throws an error:

➤ YN0000: ┌ Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at kf.getResolverByDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1647)
    at kf.bindDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1036)
    at reduceDependency (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:28690)
    at Ha.reduceHook (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:392:3512)
    at _ (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:439:7045)
    at async Promise.allSettled (index 15)
    at async io (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:10398)
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 0s 135ms

I also tried to remove the query parameter, and it still throws:

➤ YN0000: ┌ Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn isn't supported by any available resolver
    at kf.getResolverByDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1647)
    at kf.bindDescriptor (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:391:1036)
    at reduceDependency (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:28690)
    at Ha.reduceHook (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:392:3512)
    at _ (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:439:7045)
    at async Promise.allSettled (index 15)
    at async io (/Users/nic/.cache/node/corepack/yarn/3.6.2/yarn.js:390:10398)
➤ YN0000: └ Completed
➤ YN0000: Failed with errors in 0s 142ms

For the case without the query parameter, I can add #.tgz at the end of the URL. This is meaningless (it's just a URL hash that the server ignores), but it makes Yarn happy: https://gitpkg.now.sh/acornjs/acorn/acorn#.tgz

However, this hack does not work when there is a query parameter (https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a#.tgz).

To reproduce

await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a",
  },
})).resolves.not.toThrow();
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn",
  },
})).resolves.not.toThrow();
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn#.tgz",
  },
}));
await expect(packageJsonAndInstall({
  dependencies: {
    "acorn": "https://gitpkg.now.sh/acornjs/acorn/acorn?c3ec7023f08bdd6c4eee0bf76bfa2e0b88af8e4a#.tgz",
  },
})).resolves.not.toThrow();

Environment

System:
    OS: macOS 14.0
    CPU: (8) arm64 Apple M2
  Binaries:
    Node: 20.8.0 - /private/var/folders/bg/d5r9tspx5hn87zp52mcl82dh0000gn/T/xfs-1a04181c/node
    Yarn: 3.6.2 - /private/var/folders/bg/d5r9tspx5hn87zp52mcl82dh0000gn/T/xfs-1a04181c/yarn
    npm: 10.1.0 - /usr/local/bin/npm
    bun: 1.0.0 - ~/.bun/bin/bun
  npmPackages:
    jest: ^29.6.2 => 29.6.2 

We couldn't upgrade to Yarn 4 yet, apologies if this has been already fixed :sweat_smile:

Additional context

Regardless of the query parameter, URLs extensions are meaningless. I could serve a JS file at http://example.com/foo.tar, or a tarball at http://example.com/foo.js. On the web, the file type is described by the mime type in the HTTP response and not by the "extension".

@nicolo-ribaudo nicolo-ribaudo added the bug Something isn't working label Nov 3, 2023
@yarnbot yarnbot added the unreproducible This issue cannot be reproduced on master label Nov 3, 2023
@yarnbot

This comment has been minimized.

@yarnbot yarnbot added reproducible This issue can be successfully reproduced and removed unreproducible This issue cannot be reproduced on master labels Nov 3, 2023
@yarnbot
Copy link
Collaborator

yarnbot commented Nov 3, 2023

This issue reproduces on master:

Error: expect(received).resolves.not.toThrow()

Received promise rejected instead of resolved
Rejected to value: [Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js install

➤ YN0000: · Yarn 4.0.1-dev
➤ YN0000: ┌ Resolution step
::group::Resolution step
➤ YN0001: │ Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at MultiResolver.getResolverByDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:73:13)
    at MultiResolver.bindDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:31:27)
    at Project.preparePackage (/github/workspace/packages/yarnpkg-core/sources/Project.ts:737:30)
    at startPackageResolution (/github/workspace/packages/yarnpkg-core/sources/Project.ts:834:21)
    at async Promise.allSettled (index 0)
    at Object.allSettledSafe (/github/workspace/packages/yarnpkg-core/sources/miscUtils.ts:89:19)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:939:9)
    at StreamReport.startProgressPromise (/github/workspace/packages/yarnpkg-core/sources/Report.ts:167:14)
    at Project.resolveEverything (/github/workspace/packages/yarnpkg-core/sources/Project.ts:816:5)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:1775:7)
::endgroup::
➤ YN0001: Error: acorn@https://gitpkg.now.sh/acornjs/acorn/acorn?[...] isn't supported by any available resolver
    at MultiResolver.getResolverByDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:73:13)
    at MultiResolver.bindDescriptor (/github/workspace/packages/yarnpkg-core/sources/MultiResolver.ts:31:27)
    at Project.preparePackage (/github/workspace/packages/yarnpkg-core/sources/Project.ts:737:30)
    at startPackageResolution (/github/workspace/packages/yarnpkg-core/sources/Project.ts:834:21)
    at async Promise.allSettled (index 0)
    at Object.allSettledSafe (/github/workspace/packages/yarnpkg-core/sources/miscUtils.ts:89:19)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:939:9)
    at StreamReport.startProgressPromise (/github/workspace/packages/yarnpkg-core/sources/Report.ts:167:14)
    at Project.resolveEverything (/github/workspace/packages/yarnpkg-core/sources/Project.ts:816:5)
    at <anonymous> (/github/workspace/packages/yarnpkg-core/sources/Project.ts:1775:7)
➤ YN0000: └ Completed
➤ YN0000: · Failed with errors in 0s 114ms
]
    at expect (/github/workspace/.yarn/cache/expect-npm-24.8.0-8c7640c562-9b3e8262b0.zip/node_modules/expect/build/index.js:138:15)
    at module.exports (evalmachine.<anonymous>:2:7)
    at /github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:57:19
    at executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:18:22)
    at executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:25:18)
    at ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-2.0.3-558f52b79f-69609f7d06.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:26:76)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-312910c71e.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

@arcanis
Copy link
Member

arcanis commented Nov 3, 2023

Duplicate of #2437, although this other thread is closed. We probably should improve the situation somehow.

@arcanis
Copy link
Member

arcanis commented Nov 3, 2023

Will be fixed with #5931

arcanis added a commit that referenced this issue Nov 3, 2023
**What's the problem this PR addresses?**

The v2 never supported referencing packages exposed as extensionless tgz
tarballs, despite other clients being fine with them. Us being stricter
here doesn't provide any value, and just hurts compatibility.

Fixes #5929

**How did you fix it?**

Adds support for extensionless urls, which are now interpreted as
regular tarballs.

**Checklist**
<!--- Don't worry if you miss something, chores are automatically
tested. -->
<!--- This checklist exists to help you remember doing the chores when
you submit a PR. -->
<!--- Put an `x` in all the boxes that apply. -->
- [x] I have read the [Contributing
Guide](https://yarnpkg.com/advanced/contributing).

<!-- See
https://yarnpkg.com/advanced/contributing#preparing-your-pr-to-be-released
for more details. -->
<!-- Check with `yarn version check` and fix with `yarn version check
-i` -->
- [x] I have set the packages that need to be released for my changes to
be effective.

<!-- The "Testing chores" workflow validates that your PR follows our
guidelines. -->
<!-- If it doesn't pass, click on it to see details as to what your PR
might be missing. -->
- [x] I will check that all automated PR checks pass before the PR gets
reviewed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working reproducible This issue can be successfully reproduced
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants
@arcanis @nicolo-ribaudo @yarnbot and others