New resolver implementation in Rust #8807

devongovett · 2023-02-02T15:50:26Z

This is a new implementation of Parcel's resolver. It includes a few new features to improve compatibility with other tools:

package.json exports and imports (Fixes Use package.json#exports map #4155, Fixes Import specifiers starting with a hash # resolve to the current asset #8839)
tsconfig.json baseUrl, paths, and moduleSuffixes (Fixes [TypeScript] baseUrl inside tsconfig.json not supporting. #202, fixes Implement TypeScript resolver plugin #4936)

It also replaces 3 separate node resolver implementations with a single one. Previously, we had separate implementations for the default resolver plugin vs in the NodePackageManager. Now they all use a single library, written in Rust, which provides lots of feature flags to enable turning on or off different resolver features. This allows us to match Node's behavior when needed, and optionally turn on additional features to match TypeScript, webpack, previous Parcel versions, etc. It will also enable future features like ESM plugins, and prepare us for the future when more of Parcel core is written in Rust.

Since it is written in Rust, most file system operations can happen natively if you are using a NodeFS (and Yarn PnP is not enabled). Otherwise, we pass in JS callbacks that are called to read files. The API in the @parcel/node-resolver-core package is now a wrapper around the Rust implementation providing the same API as before (as long as you weren't using internal methods). Yarn PnP, diagnostics, and some other things like checking dependency versions and resolving builtin polyfills are still done in JS since they are rare and not performance critical.

For package.json exports, we try to match the conditions that webpack applies based on Parcel's environment, though not all of them are supported. Currently, there is no way to specify custom conditions - not sure how we would want to do that.

There are a few questions about how some of these resolution features should work together. For example, should aliases apply to package.json exports or tsconfig paths? Would they go before or after? etc. See some comments in the code for more.

To do

Clean up code
Performance testing
Run a lot more test to ensure compatibility with other tools

… rust-resolver

devongovett · 2023-02-09T05:12:38Z

A follow up I'd like to do to improve performance is to create an instance of NodeResolver once per build rather than once per request. This way the cache can actually be reused. This will require a new method be introduced for Resolver plugins that runs on build start in order for the instance to be created. Not sure what to call that. Is it the same as loadConfig? If we added that, when would it run?

mischnic · 2023-02-18T13:26:27Z

packages/utils/node-resolver-rs/src/specifier.rs

+
+// https://url.spec.whatwg.org/#query-state
+fn parse_query<'a>(input: &'a str) -> (Option<&'a str>, &'a str) {
+  if !input.is_empty() && input.as_bytes()[0] == b'?' {


~~as_bytes()[0] == b'?' is probably incorrect with multibyte characters?~~ It can never be a starting byte in a multibyte utf8 character...

mischnic · 2023-02-18T14:10:16Z

packages/resolvers/default/src/DefaultResolver.js

-      // Extensions are always required in URL dependencies.
-      extensions:
-        dependency.specifierType === 'commonjs' ||
-        dependency.specifierType === 'esm'
-          ? ['ts', 'tsx', 'mjs', 'js', 'jsx', 'cjs', 'json']
-          : [],
-      mainFields: ['source', 'browser', 'module', 'main'],


This could be a breaking change for some plugins. Should we try to keep supporting this? Or was the plan to do a major bump on node-resolver-core anyway?

Yeah it's a bit hard to support custom main fields with the way package.jsons are parsed now (rust discards unknown fields). Extensions could be done but idk if there are any use cases other than replicating the default logic from here. I think the safest thing would be to bump a major anyway because none of the methods other than resolve exist anymore, and someone might have been using them.

mischnic · 2023-02-18T14:13:18Z

packages/utils/node-resolver-core/src/Wrapper.js

+        // TODO: invalidations?
+        return {isExcluded: true};


Changing aliases/exports/... could probably influence whether it's external or not?

mischnic · 2023-02-18T14:20:40Z

packages/utils/node-resolver-rs/src/lib.rs

+    /// The "exports" field in package.json.
+    const EXPORTS = 1 << 5;


That's for both exports and imports, right?

mischnic · 2023-02-18T14:25:58Z

packages/utils/node-resolver-rs/src/lib.rs

+  }
+
+  fn resolve_node_module(&self, module: &str, subpath: &str) -> Result<Resolution, ResolverError> {
+    // TODO: check if module == self


Still relevant?

The node docs have a section on this, and it's even got a special PACKAGE_SELF_RESOLVE function in the spec. But I don't really understand why it's called out specifically, because self references work even without this special handling just by searching the node_modules folders as with any other package. I added some tests for it but 🤷

mischnic · 2023-02-18T14:31:18Z

Should it also read from jsconfig.json?

parcel/packages/transformers/js/src/JSTransformer.js

Lines 184 to 187 in 9d1933b

    
           let tsconfig = await config.getConfigFrom<TSConfig>( 
        
             options.projectRoot + '/index', 
        
             ['tsconfig.json', 'jsconfig.json'], 
        
           );

# Conflicts: # Cargo.lock

devongovett · 2023-02-19T22:19:17Z

Should it also read from jsconfig.json?

Yeah maybe but it would add yet another file that needs to be checked for in every directory. Also other tools like esbuild don't check for it either. I want to start with supporting tsconfig.json and see if we get feedback about needing jsconfig.json too.

fregante · 2023-02-20T02:57:52Z

TypeScript and exports? 😱 🎉

jboler · 2023-02-23T15:51:55Z

Will this new resolver support import path aliases starting with @ ? e.g.

  // package.json
  "alias": {
    "@/**": "/src/$1"
  },

  // component.tsx
  import { thing } from '@/components/...'

fregante · 2023-02-23T16:01:26Z

Doesn't it already? I think you just need "@/**/*": "./src/$1/$2"

jboler · 2023-02-23T16:37:38Z

@fregante You're absolutely right - thanks for that!

nastynaz · 2023-06-29T17:47:24Z

Does this actually work? I've been stuck for 4 hours trying to get it working. I'm on parcel 2.9.0. Fresh repo with the following tsconfig:

{
  "compilerOptions": {
    "target": "es2022",
    "paths": {
      "shared/*": ["./src/shared/*"]
    },
    "include": ["src/**/*"]
  }
}

After running parcel build and then running the code with node, I get a "cannot find package 'shared'" error. The code doesn't exist but the imports are still trying to import a nonexistant 'shared' folder.

I've also tried adding various aliases to my package.json to no avail. If anyone could help me I'd greatly appreciate it!

fregante · 2023-06-29T17:57:59Z

I think it’s opt-in at the moment. You need to customize it in your parcelrc file:

https://parceljs.org/blog/v2-9-0/#new-resolver

nastynaz · 2023-07-01T00:52:18Z

@fregante Unfortunately that doesn't work.

@devongovett Since Parcel uses swc under the hood, is it possible to just change the .swcrc file/config it uses?

Because of this issue I went back to using swc. It's pretty easy to set up there, you just copy your tsconfig.json paths statement directly into the .swcrc:

{
  "jsc": {
    "parser": {
      "syntax": "typescript",
      "decorators": true
    },
    "target": "esnext",
    "paths": {
      "opensea/*": ["./src/shared/opensea/*"]
    }
  },
  "module": {
    "type": "commonjs"
  },
  "sourceMaps": true
}

* upstream/v2: Missing edge for multiple targets (parcel-bundler#8854) Split large runtime manifest into separate bundles (parcel-bundler#8837) Improvements to new resolver (parcel-bundler#8844) Fix published files for resolver New resolver implementation in Rust (parcel-bundler#8807) Update yarn.lock (parcel-bundler#8843) Bump napi-rs to latest (parcel-bundler#8838)

* upstream/v2: (128 commits) [webextension] Add support for `chrome_style` (#8867) Switch to SWC minifier by default (#8860) Use BitSet for bundler intersections (#8862) best key logic truncating package names (#8865) Add support for loadConfig to resolver plugins (#8847) Missing edge for multiple targets (#8854) Split large runtime manifest into separate bundles (#8837) Improvements to new resolver (#8844) Fix published files for resolver New resolver implementation in Rust (#8807) Update yarn.lock (#8843) Bump napi-rs to latest (#8838) Support .proxyrc.cjs (#8833) Sort global deps before injecting imports (#8818) Sort CSS module exports (#8817) fix: add extra information to unique bundles (#8784) Don't blow up HMR when <link />s don't have hrefs (#8800) v2.8.3 Changelog for v2.8.3 Address bug by updating an asset reference and merge conditions (#8762) ...

devongovett added 30 commits December 31, 2022 10:22

wip

3f114b7

Implement package.json exports and aliases

b48aa2d

More alias stuff

354348e

Refactor specifier parsing

5a2a642

Simplify exports and subpath handling

bbdd862

More tests, better specifier parsing

2474d00

Parse tsconfig.json

e598f74

Load extended tsconfigs

aa2b283

Improve and test tsconfig parsing

f7d5b5a

More tests for tsconfig

b645d41

Strip comments in tsconfig.json

23b9b79

Use feature flags

052503c

wip: typescript extensions

392cdd4

tsconfig moduleSuffixes

418d560

Test moduleSuffixes and TS extensions

28ea048

Don't apply tsconfig inside node_modules

7173b23

Merge branch 'rust-resolver' of github.com:parcel-bundler/parcel into…

3c3c884

… rust-resolver

Implement package.json and tsconfig.json cache

742c11f

Refactor into ResolveRequest

3d8079b

Avoid passing around tsconfig

2d5ff9a

cleanup

92e5437

Track invalidations

b333afe

wip: glob

a1689f3

more globbing

0374244

refactor

d473586

Implement glob aliases

94da887

Resolve side effects

f04a956

Start node bindings

ad9a986

Support custom file systems

a565e95

Implement source field

d507c75

mischnic reviewed Feb 18, 2023

View reviewed changes

devongovett mentioned this pull request Feb 18, 2023

Use package.json#exports map #4155

Closed

devongovett added 2 commits February 19, 2023 16:25

Merge branch 'v2' of github.com:parcel-bundler/parcel into rust-resolver

f85b257

# Conflicts: # Cargo.lock

updates

9f48266

Add flow types for resolver

5c2e1ed

devongovett merged commit 307268c into v2 Feb 19, 2023

devongovett deleted the rust-resolver branch February 19, 2023 23:52

This was referenced Feb 20, 2023

Implement TypeScript resolver plugin #4936

Closed

[TypeScript] baseUrl inside tsconfig.json not supporting. #202

Closed

Improvements to new resolver #8844

Merged

jakeboone02 mentioned this pull request Mar 27, 2023

Parcel to support package.json#exports soon andrewbranch/example-subpath-exports-ts-compat#7

Open

fregante mentioned this pull request May 1, 2023

Use Parcel instead of Webpack refined-github/refined-github#6597

Closed

mischnic mentioned this pull request May 11, 2023

Add support for import subpath patterns #7840

Closed

devongovett mentioned this pull request Oct 15, 2023

Importing typescript configured for ESM not supported by Parcel #7823

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New resolver implementation in Rust #8807

New resolver implementation in Rust #8807

devongovett commented Feb 2, 2023 •

edited by mischnic

Loading

devongovett commented Feb 9, 2023

mischnic Feb 18, 2023 •

edited

Loading

mischnic Feb 18, 2023

devongovett Feb 19, 2023

mischnic Feb 18, 2023

mischnic Feb 18, 2023

mischnic Feb 18, 2023

devongovett Feb 19, 2023

mischnic commented Feb 18, 2023

devongovett commented Feb 19, 2023

fregante commented Feb 20, 2023

jboler commented Feb 23, 2023

fregante commented Feb 23, 2023 •

edited

Loading

jboler commented Feb 23, 2023

nastynaz commented Jun 29, 2023

fregante commented Jun 29, 2023 •

edited

Loading

nastynaz commented Jul 1, 2023

		/// The "exports" field in package.json.
		const EXPORTS = 1 << 5;

New resolver implementation in Rust #8807

New resolver implementation in Rust #8807

Conversation

devongovett commented Feb 2, 2023 • edited by mischnic Loading

devongovett commented Feb 9, 2023

mischnic Feb 18, 2023 • edited Loading

Choose a reason for hiding this comment

mischnic Feb 18, 2023

Choose a reason for hiding this comment

devongovett Feb 19, 2023

Choose a reason for hiding this comment

mischnic Feb 18, 2023

Choose a reason for hiding this comment

mischnic Feb 18, 2023

Choose a reason for hiding this comment

mischnic Feb 18, 2023

Choose a reason for hiding this comment

devongovett Feb 19, 2023

Choose a reason for hiding this comment

mischnic commented Feb 18, 2023

devongovett commented Feb 19, 2023

fregante commented Feb 20, 2023

jboler commented Feb 23, 2023

fregante commented Feb 23, 2023 • edited Loading

jboler commented Feb 23, 2023

nastynaz commented Jun 29, 2023

fregante commented Jun 29, 2023 • edited Loading

nastynaz commented Jul 1, 2023

devongovett commented Feb 2, 2023 •

edited by mischnic

Loading

mischnic Feb 18, 2023 •

edited

Loading

fregante commented Feb 23, 2023 •

edited

Loading

fregante commented Jun 29, 2023 •

edited

Loading