Updating from v8.0.0 to v8.0.1 causes RangeError: Maximum call stack size exceeded #125
Comments
|
@joshhunt Any thoughts why the symlink traversal would cause this? Could they have a circular symlink in their paths? |
|
@blumendorf Can you please provide additional debugging details? It is unclear what globs are being provided or what path is being processed. You can just add |
|
@blumendorf Looking at the stack trace and comparing with the code, it seems like a bad filepath (perhaps too long?) is being provided to anymatch here: https://github.com/gulpjs/glob-stream/blob/master/index.js#L269-L270. If you add a We only added a symlink fix in 8.0.1, so it seems you have a symlink that is now being resolved and that causes the glob match to blow the stack. |
|
Thinking about it now - I guess recursive symlinks could cause an error with something like this. I notice pnpm is being used, which makes heavy use of symlinks. At first I was eager to say "just don't have circular symlinks", but it's not really your fault if they're coming from inside pnpm's structure.... A minimal reproduction repo, or some debugging to see what paths it's choking on, would go a long way. In the meantime I'll see if I can have a play around with reproducing. FYI - I introduced this symlink fix so I can get i18next-parser to follow symlinks, which is passing in our CI so far https://drone.grafana.net/grafana/grafana/169064/2/6 |
|
I've tried to reproduce this with... varying degrees of success. I've created this repo to help reproduce, which tests with both npm and pnpm, and i18next-scanner and i18next-parser https://github.com/joshhunt/glob-stream-i18next-scanner-repro If I run Note this is a different exception that both @blumendorf and @sunnysonx reporting having. I get the same results (including same "too many symbolic links" exception) on WSL2 w/Ubuntu 20.04.5 LTS (where the code is in the Linux VM, not traversing into the Windows mount), and on macOS Sonoma 14.3.1. I'm not sure where to go from here - could try to detect circular symbolic links, but as the exception I got was different from the original reporters, I'm not sure this would fix their issue. A minimal reproduction would go a long way 🙏 |
|
@joshhunt hey, sure, attaching a reproduction repo: After npm install, run If you get the same result, check Added also console logs between 269 and 270 lines, but didn't get much information, as it printed 12k+ paths, and none of them seem to be suspicious. But one strange thing, is that always the last row is from @material-ui, but maybe it actually crashed because of the path that should be the next one. WSL with ubuntu 22.* |
|
Was able to reproduce this thanks to @sunnysonx's repro. I suspect this will happen to anyone with many large package.json dependencies. @phated I think this may be due to now-and-later's mapSeries causing additional stack frames. Perhaps this issue has always existed, as a upper-bound on the number of files that can be walked, but the new walkdir implementation can only do significantly less, which can be easy to run into with node_modules directories. I tried to look through the code to see if there was a glaring recursion issue but nothing immediately stood out. Circular symlink issue still exists though and is totally on me. I'll try and get a fix for that in. |
|
I also get this error after upgrading to This is my code: function imageminify() {
const imgStreamOfficialIcons = gulp.src('material/icons_official/*.png')
.pipe(imagemin([
optipng({ optimizationLevel: 7 })
]))
.pipe(gulp.dest('material/icons_official_minified/'))
const imgStreamUnofficialIcons = gulp.src('material/icons_unofficial/*.png')
.pipe(imagemin([
optipng({ optimizationLevel: 7 })
]))
.pipe(gulp.dest('material/icons_unofficial_minified/'))
fs.readdirSync('material/icons_skills/').forEach(file => {
fs.renameSync('material/icons_skills/' + file, 'material/icons_skills/' + file.replace('.sub', ''))
})
const imgStreamSkillsIcons = gulp.src('material/icons_skills/*.png')
.pipe(imagemin([
optipng({ optimizationLevel: 7 })
]))
.pipe(gulp.dest('material/icons_skills_minified/'))
return merge(imgStreamOfficialIcons, imgStreamUnofficialIcons, imgStreamSkillsIcons)
}Error: Rolling back to I do have quite a lot of dependencies. |
|
I experienced this issue on macOS in a mixed Python / JS project which contained a hidden subdirectory ( The glob being passed to After removing this directory, the |
|
Minimal reproduction of the issue mentioned in the previous comment: Create a project with the following layout: gulpfile.mjs: import gulp from "gulp";
gulp.task("copy-files", () => gulp.src("src/*.txt").pipe(gulp.dest("build/")));Run from pathlib import Path
for i in range(1, 4000):
Path(f'tmp/{i}.txt').touch()Then run |
|
I got the same error when my folder in monorepo contained thousands of JSON files. This folder is in another project and my globs are absolute and should not be candidates. |
Disclaimer: I am not sure this is actually a glob-stream issue, but it seems to be the only moving part.
What we did:
We are using i18next parser and recently updated dependencies.
In the process, glob-stream got updated from 8.0.0 to 8.0.1.
This broke i18next-parser.
Manually reverting back to 8.0.0 in the lock file fixed the issue.
original dependency chain:
i18next-parser@8.12.0->vinyl-fs@4.0.0->glob-stream@8.0.0updated dependency chain:
i18next-parser@8.13.0->vinyl-fs@4.0.0->glob-stream@8.0.1What were you expecting to happen?
Updating glob-stream from 8.0.0 to 8.0.1 should not be breaking.
What actually happened?
We got a
RangeError: Maximum call stack size exceeded.Terminal output / screenshots
Please provide the following information:
node -v): 20.11.0npm -v): pnpm v8.15.4gulp -v): n/aThe text was updated successfully, but these errors were encountered: