no such file or directory #59

gajus · 2024-01-25T19:11:44Z

Expected Behavior

getExports is throwing an error:

{
  context: { format: 'module', importAttributes: {} },
  error: Error: ENOENT: no such file or directory, open '/Users/x/Developer/contra/gaia/node_modules/.pnpm/graphql-yoga@5.1.1_graphql@16.8.1/node_modules/graphql-yoga/esm/@graphql-yoga/logger'
      at async open (node:internal/fs/promises:633:25)
      at async readFile (node:internal/fs/promises:1242:14)
      at async getSource (node:internal/modules/esm/load:46:14)
      at async defaultLoad (node:internal/modules/esm/load:137:34)
      at async nextLoad (node:internal/modules/esm/hooks:750:22)
      at async getExports (/Users/x/Developer/contra/gaia/node_modules/.pnpm/import-in-the-middle@1.7.3/node_modules/import-in-the-middle/lib/get-exports.js:68:17)
      at async processModule (/Users/x/Developer/contra/gaia/node_modules/.pnpm/import-in-the-middle@1.7.3/node_modules/import-in-the-middle/hook.js:134:23)
      at async processModule (/Users/x/Developer/contra/gaia/node_modules/.pnpm/import-in-the-middle@1.7.3/node_modules/import-in-the-middle/hook.js:160:20)
      at async getSource (/Users/x/Developer/contra/gaia/node_modules/.pnpm/import-in-the-middle@1.7.3/node_modules/import-in-the-middle/hook.js:269:60)
      at async load (/Users/x/Developer/contra/gaia/node_modules/.pnpm/import-in-the-middle@1.7.3/node_modules/import-in-the-middle/hook.js:334:26) {
    errno: -2,
    code: 'ENOENT',
    syscall: 'open',
    path: '/Users/x/Developer/contra/gaia/node_modules/.pnpm/graphql-yoga@5.1.1_graphql@16.8.1/node_modules/graphql-yoga/esm/@graphql-yoga/logger'
  },
  url: 'file:///Users/x/Developer/contra/gaia/node_modules/.pnpm/graphql-yoga@5.1.1_graphql@16.8.1/node_modules/graphql-yoga/esm/@graphql-yoga/logger'
}

I added the following code to catch the above error.

let parentCtx;

try {
  parentCtx = await parentLoad(url, context)
} catch (error) {
  console.log({
    context,
    error,
    url
  });

  throw error;
}

The same code works if I remove --loader=import-in-the-middle/hook.mjs.

Happy to provide more context.

Specifications

Platform: Node.js v21

The text was updated successfully, but these errors were encountered:

gajus · 2024-01-25T19:18:52Z

~~Hard to tell from import URLs what's happening here, but I wonder if it is related to how pnpm works.~~

Confirmed that the above is not due to the way how node modules are linked.

gajus · 2024-01-25T19:29:11Z

The paths mentioned in the URLs do not actually exist, e.g.

{
  context: { format: 'module', importAttributes: {} },
  error: Error: ENOENT: no such file or directory, open '/Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription'
      at async open (node:internal/fs/promises:633:25)
      at async readFile (node:internal/fs/promises:1242:14)
      at async getSource (node:internal/modules/esm/load:46:14)
      at async defaultLoad (node:internal/modules/esm/load:137:34)
      at async nextLoad (node:internal/modules/esm/hooks:750:22)
      at async getExports (/Users/x/Developer/contra/gaia/node_modules/import-in-the-middle/lib/get-exports.js:66:17)
      at async processModule (/Users/x/Developer/contra/gaia/node_modules/import-in-the-middle/hook.js:134:23)
      at async processModule (/Users/x/Developer/contra/gaia/node_modules/import-in-the-middle/hook.js:160:20)
      at async processModule (/Users/x/Developer/contra/gaia/node_modules/import-in-the-middle/hook.js:160:20)
      at async getSource (/Users/x/Developer/contra/gaia/node_modules/import-in-the-middle/hook.js:269:60) {
    errno: -2,
    code: 'ENOENT',
    syscall: 'open',
    path: '/Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription'
  },
  url: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription'
}

There is no such directory as /Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription

The closest directly that exists is /Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/.

gajus · 2024-01-25T23:46:44Z

An example of file that triggers this is:

export * from 'styled-components';
//# sourceMappingURL=index.js.map

I can see that when processModule is trying to process it, it derives wrong URL for the module.

I added this line just before processModule:

console.log({
  modFile,
  normalizedModName,
  modUrl,
  modName,
});

which tells us:

{
  modFile: 'styled-components',
  normalizedModName: 'styledComponents',
  modUrl: 'file:///Users/x/Developer/contra/gaia/packages/styled-components/dist/styled-components',
  modName: '17062121655529e604482'
}

/Users/x/Developer/contra/gaia/packages/styled-components/dist/styled-components is wrong. It should be: /Users/x/Developer/contra/gaia/node_modules/styled-components/. But there is no way of knowing that just from analyzing the URLs, which is what this module does.

https://github.com/DataDog/import-in-the-middle/blob/c3c2c52c1915b47994af59d507c59029c1f1fae9/hook.js#L155-L166

I am confused how this works for other people use cases, since this module is clearly used by others.

gajus · 2024-01-25T23:55:07Z

So all of these failures are happening when resolving export * from ... pattern, e.g.

{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@envelop/core/esm/index.js',
  modFile: '@envelop/types',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@envelop/core/esm/@envelop/types',
  n: '* from @envelop/types'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/index.js',
  modFile: '@graphql-yoga/logger',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/logger',
  n: '* from @graphql-yoga/logger'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@floating-ui/react-dom-interactions/dist/floating-ui.react-dom-interactions.mjs',
  modFile: '@floating-ui/react-dom',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@floating-ui/react-dom-interactions/dist/@floating-ui/react-dom',
  n: '* from @floating-ui/react-dom'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/playwright-chromium/index.mjs',
  modFile: 'playwright-core',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/playwright-chromium/playwright-core',
  n: '* from playwright-core'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/packages/styled-components/dist/index.js',
  modFile: 'styled-components',
  modUrl: 'file:///Users/x/Developer/contra/gaia/packages/styled-components/dist/styled-components',
  n: '* from styled-components'
}

Tagging @bengl in case you have a better sense of what's breaking.

gajus · 2024-01-26T00:05:28Z

For what it is worth, if we update the logic to resolve import paths relative to the srcUrl, it resolves to correct modules (?).

-      const data = await processModule({
-        srcUrl: modUrl,
-        context,
-        parentGetSource,
-        ns: `$${modName}`,
-        defaultAs: normalizedModName
-      })
+      let data;
+      
+      try {
+        data = await processModule({
+          srcUrl: modUrl,
+          context,
+          parentGetSource,
+          ns: `$${modName}`,
+          defaultAs: normalizedModName
+        });
+      } catch (error) {
+        console.log({
+          srcUrl,
+          modFile,
+          modUrl,
+          n,
+        });
+      }
+
+      if (!data) {
+        const newModUrl = pathToFileURL(require.resolve(modFile, {
+          paths: [srcUrl]
+        })).toString();
+
+        console.log({
+          newModUrl
+        });
+
+        data = await processModule({
+          srcUrl: newModUrl,
+          context,
+          parentGetSource,
+          ns: `$${modName}`,
+          defaultAs: normalizedModName
+        });
+      }

{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@envelop/core/esm/index.js',
  modFile: '@envelop/types',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@envelop/core/esm/@envelop/types',
  n: '* from @envelop/types'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@envelop/types/cjs/index.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/index.js',
  modFile: '@graphql-yoga/logger',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/logger',
  n: '* from @graphql-yoga/logger'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@graphql-yoga/logger/cjs/index.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/subscription.js',
  modFile: '@graphql-yoga/subscription',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription',
  n: '* from @graphql-yoga/subscription'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@graphql-yoga/subscription/cjs/index.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/subscription.js',
  modFile: '@graphql-yoga/subscription',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/graphql-yoga/esm/@graphql-yoga/subscription',
  n: '* from @graphql-yoga/subscription'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@graphql-yoga/subscription/cjs/index.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@floating-ui/react-dom-interactions/dist/floating-ui.react-dom-interactions.mjs',
  modFile: '@floating-ui/react-dom',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@floating-ui/react-dom-interactions/dist/@floating-ui/react-dom',
  n: '* from @floating-ui/react-dom'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/@floating-ui/react-dom/dist/floating-ui.react-dom.umd.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/playwright-chromium/index.mjs',
  modFile: 'playwright-core',
  modUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/playwright-chromium/playwright-core',
  n: '* from playwright-core'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/playwright-core/index.js'
}
{
  srcUrl: 'file:///Users/x/Developer/contra/gaia/packages/styled-components/dist/index.js',
  modFile: 'styled-components',
  modUrl: 'file:///Users/x/Developer/contra/gaia/packages/styled-components/dist/styled-components',
  n: '* from styled-components'
}
{
  newModUrl: 'file:///Users/x/Developer/contra/gaia/node_modules/styled-components/dist/styled-components.cjs.js'
}

gajus · 2024-01-26T00:17:02Z

Looks like depending on how modules are installed (hoisted or non-hoisted), you will need to try both variants, i.e.

const newModUrl = pathToFileURL(require.resolve(modFile, {
  paths: [srcUrl]
})).toString();

and

const newModUrl = pathToFileURL(require.resolve(modFile)).toString();

gajus · 2024-01-26T15:26:33Z

This is the patch that we ended up with.

diff --git a/hook.js b/hook.js
index aa22356f370a34cd49ae3d5ea8fd076202e71bf5..074c388edf40f10e3e4274cf38c308371f7ce4fe 100644
--- a/hook.js
+++ b/hook.js
@@ -22,6 +22,48 @@ if (NODE_MAJOR >= 20 || (NODE_MAJOR === 18 && NODE_MINOR >= 19)) {
   getExports = ({ url }) => import(url).then(Object.keys)
 }
 
+function shouldIgnore (url) {
+  // In short, some packages are not compatible with this hook.
+  // I have attempted to patch this hook to work with every package, but that just sent me down a rabbit hole.
+  // See attempts and related issues here:
+  // * https://github.com/DataDog/import-in-the-middle/issues/60
+  // * https://github.com/DataDog/import-in-the-middle/issues/59
+  //
+  // However, for this hook to be useful, it does not need to work with every package.
+  // It just needs to work with the packages that we want OpenTelemetry to instrument.
+  const patterns = [
+    '@envelop/core',
+    '@floating-ui/react-dom-interactions',
+    'dataloader',
+    'graphql-yoga',
+    // Does not get patched properly.
+    // Produces errors such as:
+    // > TypeError: Cannot read properties of undefined (reading 'sendCommand')
+    // > at instrumentation_1.InstrumentationNodeModuleDefinition.moduleExports [as patch] (/Users/x/Developer/contra/gaia/node_modules/.pnpm/@opentelemetry+instrumentation-ioredis@0.36.1_@opentelemetry+api@1.7.0/node_modules/@opentelemetry/instrumentation-ioredis/build/src/instrumentation.js:40:78)
+    'ioredis',
+    'openai',
+    'playwright-chromium',
+    'styled-components',
+  ];
+
+  for (const pattern of patterns) {
+    if (url === pattern || url.startsWith(pattern + '/') || url.includes('/' + pattern + '/')) {
+      return true;
+    }
+  }
+
+  // If you miss a pattern, then you will get an error along the lines of:
+  //
+  // > Error: ENOENT: no such file or directory, open '/Users/x/Developer/contra/gaia/packages/styled-components/dist/styled-components'
+  //
+  // Usually, it is easy to tell what package is causing the issue by just looking at the path (e.g. 'styled-components' in the above example).
+  // However, other times, it is not so obvious. In those cases, you can uncomment the following line to get everything that was attempted up to the point of failure.
+  //
+  // console.log('url', url)
+
+  return false;
+}
+
 function hasIitm (url) {
   try {
     return new URL(url).searchParams.has('iitm')
@@ -230,6 +272,9 @@ function addIitm (url) {
 
 function createHook (meta) {
   async function resolve (specifier, context, parentResolve) {
+    if (shouldIgnore(specifier)) {
+      return parentResolve(specifier)
+    }
     const { parentURL = '' } = context
     const newSpecifier = deleteIitm(specifier)
     if (isWin && parentURL.indexOf('file:node') === 0) {
@@ -264,6 +309,9 @@ function createHook (meta) {
 
   const iitmURL = new URL('lib/register.js', meta.url).toString()
   async function getSource (url, context, parentGetSource) {
+    if (shouldIgnore(url)) {
+      return parentGetSource(url)
+    }
     if (hasIitm(url)) {
       const realUrl = deleteIitm(url)
       const { imports, namespaces, setters: mapSetters } = await processModule({
@@ -330,6 +378,9 @@ register(${JSON.stringify(realUrl)}, _, set, ${JSON.stringify(specifiers.get(rea
 
   // For Node.js 16.12.0 and higher.
   async function load (url, context, parentLoad) {
+    if (shouldIgnore(url)) {
+      return parentLoad(url)
+    }
     if (hasIitm(url)) {
       const { source } = await getSource(url, context, parentLoad)
       return {

jsumners-nr · 2024-02-01T12:45:32Z

I think hard coding a bunch of module identifiers is not the correct path forward. We should be able to concoct a replication that we can solve.

Fixes (potentially) #59, #62, #63 `parentResolve` is cached and later used when attempting to load bare specifiers.

timfish · 2024-05-31T00:57:04Z

This was fixed by #78

timfish · 2024-06-28T21:37:36Z

This was closed by #78.

Feel free to open another issue if this is not the case!

gajus mentioned this issue Jan 25, 2024

SyntaxError: Unexpected token #58

Closed

This comment was marked as off-topic.

Sign in to view

This comment was marked as outdated.

Sign in to view

gajus mentioned this issue Jan 26, 2024

Identifier '$Completions' has already been declared #60

Closed

meyer9 mentioned this issue Jan 31, 2024

Fails to resolve exports defined submodule when reexported by another package #63

Closed

chentsulin mentioned this issue May 17, 2024

Node SDK generating Error: ENOENT: no such file or directory getsentry/sentry-javascript#12059

Closed

3 tasks

timfish mentioned this issue May 23, 2024

fix: Resolve re-exports of external modules #78

Merged

AbhiPrasad mentioned this issue May 27, 2024

v8 Node OpenTelemetry Instrumentation Blockers getsentry/sentry-javascript#12242

Closed

bengl pushed a commit that referenced this issue May 28, 2024

fix: Resolve re-exports of external modules (#78)

bf3a4fb

Fixes (potentially) #59, #62, #63 `parentResolve` is cached and later used when attempting to load bare specifiers.

timfish closed this as completed Jun 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

no such file or directory #59

no such file or directory #59

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 25, 2024

This comment was marked as off-topic.

This comment was marked as outdated.

gajus commented Jan 25, 2024

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 26, 2024 •

edited

Loading

gajus commented Jan 26, 2024

gajus commented Jan 26, 2024

jsumners-nr commented Feb 1, 2024

timfish commented May 31, 2024

timfish commented Jun 28, 2024

no such file or directory #59

no such file or directory #59

Comments

gajus commented Jan 25, 2024 • edited Loading

Expected Behavior

Specifications

gajus commented Jan 25, 2024 • edited Loading

gajus commented Jan 25, 2024

This comment was marked as off-topic.

This comment was marked as outdated.

gajus commented Jan 25, 2024

gajus commented Jan 25, 2024 • edited Loading

gajus commented Jan 26, 2024 • edited Loading

gajus commented Jan 26, 2024

gajus commented Jan 26, 2024

jsumners-nr commented Feb 1, 2024

timfish commented May 31, 2024

timfish commented Jun 28, 2024

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 25, 2024 •

edited

Loading

gajus commented Jan 26, 2024 •

edited

Loading