-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Concurrent executions would race for global gitconfig #3294
Comments
We don't currently run dependabot-core in parallel and so the code has not really been designed with thread-safety in mind. For many ecosystems we end up creating many temp directories for different steps in the update process, all of which need access to this config. In theory I think we could create one parent dir that holds the git config, and create all the tmp-dirs for the different steps in there. I am afraid that this would require changes to how updates are ran in many ecosystems, and we don't really have the bandwidth to tackle this right now. There might be a clean way to do it that I can't think of right now and either way I'd be happy to review a PR that adds this change though. |
@jurre Thank you for the quick response and explanation. Now I understand the reasoning and design behind it. I will think about your suggestion and take a better look at the code. Please feel free to close this issue and perhaps give it a |
NVM I see there would be no local git repo cloned, for example in case of yarn. |
Yeah, we have some plans on eventually checking out a git repo for all ecosystems, but not sure on the timeline at this point |
Going to close this out for now, but feel free to reach out if I there's anything I can do to help |
* [`013f0262d`](npm/cli@013f026) [#3469](npm/cli#3469) fix(exitHandler): write code to logfile ([@wraithgar](https://github.com/wraithgar)) * [`0dd0341ac`](npm/cli@0dd0341) [#3474](npm/cli#3474) fix(ping): make "npm ping" echo a right time ([@aluneed](https://github.com/aluneed)) * [`d2e298f3c`](npm/cli@d2e298f) [#3484](npm/cli#3484) fix(deprecate): add undeprecate support ([@wraithgar](https://github.com/wraithgar)) ### DOCUMENTATION * [`9dd32d08e`](npm/cli@9dd32d0) [#3485](npm/cli#3485) fix(docs): remove npm package config override ([@wraithgar](https://github.com/wraithgar)) * [`a4e095618`](npm/cli@a4e0956) [#3486](npm/cli#3486) fix(docs): remove .hooks scripts ([@wraithgar](https://github.com/wraithgar)) * [`5f8ccccef`](npm/cli@5f8cccc) [#3483](npm/cli#3483) chore(tests): clean snapshot for lib/view.js tests ([@wraithgar](https://github.com/wraithgar)) * [`23ce3af19`](npm/cli@23ce3af) [#3460](npm/cli#3460) feat(ls): report *why* something is invalid ([@isaacs](https://github.com/isaacs)) * [`53f81af31`](npm/cli@53f81af) [#3450](npm/cli#3450) fix(docs): Improve phrasing of workspace example ([@lumaxis](https://github.com/lumaxis)) * [`78da60ffe`](npm/cli@78da60f) [#3454](npm/cli#3454) chore(linting): add bin and clean up lib/ls.js * [`54eae3063`](npm/cli@54eae30) [#3416](npm/cli#3416) chore(errorHandler): rename to exit handler ([@wraithgar](https://github.com/wraithgar)) * [`d0f50b156`](npm/cli@d0f50b1) [#3451](npm/cli#3451) chore(refactor): async npm.load ([@wraithgar](https://github.com/wraithgar)) * [`87f67d9ef`](npm/cli@87f67d9) [#3458](npm/cli#3458) chore(tests): expose real mock npm object ([@wraithgar](https://github.com/wraithgar)) * [`f3dce0917`](npm/cli@f3dce09) [#3459](npm/cli#3459) chore(config): snapshot config descriptions ([@wraithgar](https://github.com/wraithgar)) * [`6254b6f72`](npm/cli@6254b6f) [#3234](npm/cli#3234) [#3455](npm/cli#3455) @npmcli/package-json refactor ([@ruyadorno](https://github.com/ruyadorno)) * [`fe4138381`](npm/cli@fe41383) `@npmcli/arborist@2.6.4`: * bin: allow turning off timer display with --timers=false * fix: do not try to inflate a fresh lockfile * fix(diff): walk target children if root is a link * chore: @npmcli/package-json refactor * [`fce30e423`](npm/cli@fce30e4) [#3435](npm/cli#3435) fix(docs): rebuild config docs ([@wraithgar](https://github.com/wraithgar)) * [`ae285b391`](npm/cli@ae285b3) [#3408](npm/cli#3408) feat(ls): support `--package-lock-only` flag ([@G-Rath](https://github.com/G-Rath)) * [`c984fb59c`](npm/cli@c984fb5) [#3420](npm/cli#3420) feat(pack): add pack-destination config ([@wraithgar](https://github.com/wraithgar)) * [`40829ec40`](npm/cli@40829ec) [#2554](npm/cli#2554) [#3399](npm/cli#3399) fix(link): do not prune packages ([@ruyadorno](https://github.com/ruyadorno)) * [`102d4e6fb`](npm/cli@102d4e6) [#3417](npm/cli#3417) fix(workspaces): explicitly error in global mode ([@wraithgar](https://github.com/wraithgar)) * [`993df3041`](npm/cli@993df30) [#3423](npm/cli#3423) fix(docs): ls command usage instructions ([@gurdiga](https://github.com/gurdiga)) * [`dcc13662c`](npm/cli@dcc1366) [#3418](npm/cli#3418) fix(config): update link definition ([@wraithgar](https://github.com/wraithgar)) * [`b19e56c2e`](npm/cli@b19e56c) [#3382](npm/cli#3382) [#3429](npm/cli#3429) fix(ls): respect prod config for workspaces ([@ruyadorno](https://github.com/ruyadorno)) * [`c99b8b53c`](npm/cli@c99b8b5) [#3430](npm/cli#3430) fix(config): add flatOptions.npxCache ([@wraithgar](https://github.com/wraithgar)) * [`e5abf2a21`](npm/cli@e5abf2a) [#3386](npm/cli#3386) chore(libnpmdiff): added as workspace ([@ruyadorno](https://github.com/ruyadorno)) * [`c6a8734d7`](npm/cli@c6a8734) [#3388](npm/cli#3388) chore(refactor): finish passing npm context ([@wraithgar](https://github.com/wraithgar)) * [`d16ee452a`](npm/cli@d16ee45) [#3426](npm/cli#3426) chore(tests): use path.resolve ([@wraithgar](https://github.com/wraithgar)) * [`6b951c042`](npm/cli@6b951c0) `libnpmversion@1.2.1`: * fix(retrieve-tag): pass match in a way git accepts * [`de820a021`](npm/cli@de820a0) `npm-package-arg@8.1.5`: * fix: Make file: URLs (mostly) RFC 8909 compliant * [`16a95c647`](npm/cli@16a95c6) `@npmcli/arborist@2.6.3`: * fix(inventory) handle old and british forms of 'license' * fix: removes [_complete] check to apply correct metadata * ensure node.fsParent is not set to node itself * fix extraneous deps on load-actual * [`d341bd86c`](npm/cli@d341bd8) `make-fetch-happen@9.0.3`: * fix: implement cache modes correctly * [`c90612cf5`](npm/cli@c90612c) `libnpmexec@2.0.0`: * use new npxCache option * [`ef668ab57`](npm/cli@ef668ab) [#3368](npm/cli#3368) feat(diff): add workspace support ([@wraithgar](https://github.com/wraithgar)) * [`26d00c477`](npm/cli@26d00c4) [#3364](npm/cli#3364) fix(tests): mock writeFile in pack tests so we dont create 0 byte files in the repo ([@nlf](https://github.com/nlf)) * [`f130a81d6`](npm/cli@f130a81) [#3367](npm/cli#3367) fix(linting): add scripts, docs, smoke-tests ([@wraithgar](https://github.com/wraithgar)) * [`992799cd8`](npm/cli@992799c) [#3383](npm/cli#3383) fix(login): properly save scope if defined ([@wraithgar](https://github.com/wraithgar)) * [`844229519`](npm/cli@8442295) [#3392](npm/cli#3392) docs(workspaces): update using npm section Added examples of using `npm init` to bootstrap a new workspace and a section on how to add/manage dependencies to workspaces. ([@ruyadorno](https://github.com/ruyadorno)) * [`3654890fb`](npm/cli@3654890) remove ignored dep ([@nlf](https://github.com/nlf)) * [`a4a0e68a9`](npm/cli@a4a0e68) [#3362](npm/cli#3362) check less stuff into node_modules ([@isaacs](https://github.com/isaacs)) * [`7d5b049b6`](npm/cli@7d5b049) [#3365](npm/cli#3365) chore(package) Use a "files" list ([@isaacs](https://github.com/isaacs)) * [`e92b5f2ba`](npm/cli@e92b5f2) `npm-registry-fetch@11.0.0` * feat: improved logging of cache status * [`e864bd3ce`](npm/cli@e864bd3) [#3345](npm/cli#3345) fix(update-notifier): do not update notify when installing npm@spec ([@isaacs](https://github.com/isaacs)) * [`aafe23572`](npm/cli@aafe235) [#3348](npm/cli#3348) fix(update-notifier): parallelize check for updates ([@isaacs](https://github.com/isaacs)) * [`bc9c57dda`](npm/cli@bc9c57d) [#3353](npm/cli#3353) fix(docs): remove documentation for '--scripts-prepend-node-path' as it was removed in npm@7 ([@gimli01](https://github.com/gimli01)) * [`ca2822110`](npm/cli@ca28221) [#3360](npm/cli#3360) fix(docs): link foreground-scripts w/ loglevel ([@wraithgar](https://github.com/wraithgar)) * [`fb630b5a9`](npm/cli@fb630b5) [#3342](npm/cli#3342) chore(docs): manage docs as a workspace ([@ruyadorno](https://github.com/ruyadorno)) * [`54de5c6a4`](npm/cli@54de5c6) `npm-package-arg@8.1.4`: * fix: trim whitespace from fetchSpec * fix: handle file: when root directory begins with a special character * [`e92b5f2ba`](npm/cli@e92b5f2) `make-fetch-happen@9.0.1` * breaking: complete refactor of caching. drops warning headers, prevents cache indexes from growing for every request, correctly handles varied requests to the same url, and now caches redirects. * fix: support url-encoded proxy authorization * fix: do not lazy-load proxy agents or agentkeepalive. fixes the intermittent failures to update npm on slower connections. `npm-registry-fetch@11.0.0` * breaking: drop handling of deprecated warning headers * docs: fix header type for npm-command * docs: update registry param * feat: improved logging of cache status * [`23c50a45f`](npm/cli@23c50a4) `make-fetch-happen@9.0.2`: * fix: work around negotiator's lazy loading * [`c4ef78b08`](npm/cli@c4ef78b) [#3344](npm/cli#3344) fix(automation): update incorrect variable name in create-cli-deps-pr workflow ([@gimli01](https://github.com/gimli01)) * [`598a17a26`](npm/cli@598a17a) [#3329](npm/cli#3329) fix(libnpmexec): don't detach output from npm ([@wraithgar](https://github.com/wraithgar)) * [`c4fc03e9e`](npm/cli@c4fc03e) `@npmcli/arborist@2.6.1` * fixes reifying deps with mismatching version ranges between actual and virtual trees * [`9159fa62a`](npm/cli@9159fa6) `libnpmexec@1.2.0` * [`399ff8cbc`](npm/cli@399ff8c) [#3312](npm/cli#3312) feat(link): add workspace support ([@isaacs](https://github.com/isaacs)) * [`46a9bcbcb`](npm/cli@46a9bcb) [#3282](npm/cli#3282) fix(docs): proper postinstall script file name ([@KevinFCormier](https://github.com/KevinFCormier)) * [`83590d40f`](npm/cli@83590d4) [#3272](npm/cli#3272) fix(ls): show relative paths from root ([@isaacs](https://github.com/isaacs)) * [`a574b518a`](npm/cli@a574b51) [#3304](npm/cli#3304) fix(completion): restore IFS even if `npm completion` returns error ([@NariyasuHeseri](https://github.com/NariyasuHeseri)) * [`554e8a5cd`](npm/cli@554e8a5) [#3311](npm/cli#3311) set audit exit code properly ([@isaacs](https://github.com/isaacs)) * [`4a4fbe33c`](npm/cli@4a4fbe3) [#3268](npm/cli#3268) [#3285](npm/cli#3285) fix(publish): skip private workspaces ([@ruyadorno](https://github.com/ruyadorno)) * [`3c53d631f`](npm/cli@3c53d63) [#3307](npm/cli#3307) fix(docs): typo in package-lock.json docs ([@rethab](https://github.com/rethab)) * [`96367f93f`](npm/cli@96367f9) rebuild npm-pack doc ([@isaacs](https://github.com/isaacs)) * [`64b13dd10`](npm/cli@64b13dd) [#3313](npm/cli#3313) Drop stale Python 3<->node-gyp remark ([@spencerwilson](https://github.com/spencerwilson)) * [`7b56bfdf3`](npm/cli@7b56bfd) `cacache@15.2.0`: * feat: allow fully deleting indices * feat: add a validateEntry option to compact * chore: lint * chore: use standard npm style release scripts * [`dbbc151a3`](npm/cli@dbbc151) `npm-audit-report@2.1.5`: * fix(exit-code): account for null auditLevel default (#46) * [`5b2604507`](npm/cli@5b26045) chore(package-lock): update devDependencies ([@gar](https://github.com/Gar)) * [`3d5df0082`](npm/cli@3d5df00) [#3294](npm/cli#3294) chore(ci): move node release PR workflow to cli repo ([@gimli01](https://github.com/gimli01)) * [`0d1a9d787`](npm/cli@0d1a9d7) [#3227](npm/cli#3227) feat(install): add workspaces support to npm install commands ([@isaacs](https://github.com/isaacs)) * [`c18626f04`](npm/cli@c18626f) [#3250](npm/cli#3250) feat(ls): add workspaces support ([@ruyadorno](https://github.com/ruyadorno)) * [`41099d395`](npm/cli@41099d3) [#3265](npm/cli#3265) feat(explain): add workspaces support ([@ruyadorno](https://github.com/ruyadorno)) * [`fde354669`](npm/cli@fde3546) [#3251](npm/cli#3251) feat(unpublish): add workspace/dry-run support ([@wraithgar](https://github.com/wraithgar)) * [`83df3666c`](npm/cli@83df366) [#3260](npm/cli#3260) feat(outdated): add workspaces support ([@ruyadorno](https://github.com/ruyadorno)) * [`63a7635f7`](npm/cli@63a7635) [#3217](npm/cli#3217) feat(pack): add support to json config/output ([@mrmlnc](https://github.com/mrmlnc)) * [`faa12ccc2`](npm/cli@faa12cc) [#3253](npm/cli#3253) fix search description typos ([@juanpicado](https://github.com/juanpicado)) * [`2f5c28a68`](npm/cli@2f5c28a) [#3243](npm/cli#3243) fix(docs): autogenerate config docs for commands ([@isaacs](https://github.com/isaacs)) * [`ec256a14a`](npm/cli@ec256a1) `@npmcli/arborist@2.6.0` * [`5f15aba86`](npm/cli@5f15aba) `cacache@15.1.0` * [`b3add87e6`](npm/cli@b3add87) [#3262](npm/cli#3262) `npm-registry-client@10.1.2`: * fixed sso login token * [`076420c14`](npm/cli@076420c) [#3231](npm/cli#3231) feat(publish): add workspace support ([@wraithgar](https://github.com/wraithgar)) * [`370b36a36`](npm/cli@370b36a) [#3241](npm/cli#3241) feat(fund): add workspaces support ([@ruyadorno](https://github.com/ruyadorno)) * [`0c18e4f77`](npm/cli@0c18e4f) `@npmcli/arborist@2.5.0` * [`b551c6811`](npm/cli@b551c68) `libnpmfund@1.1.0` * [`de49f58f5`](npm/cli@de49f58) [#3216](npm/cli#3216) fix(contributing): link to proper cli repo ([@mrmlnc](https://github.com/mrmlnc)) * [`1d092144e`](npm/cli@1d09214) [#3203](npm/cli#3203) fix(packages): locale-agnostic string sorting ([@isaacs](https://github.com/isaacs)) * [`0696fca13`](npm/cli@0696fca) [#3209](npm/cli#3209) fix(view): fix non-registry specs ([@wraithgar](https://github.com/wraithgar)) * [`71ac93597`](npm/cli@71ac935) [#3206](npm/cli#3206) chore(github): Convert md issue template to yaml ([@lukehefson](https://github.com/lukehefson)) * [`6fb386d3b`](npm/cli@6fb386d) [#3201](npm/cli#3201) fix(tests): increase test fuzziness ([@wraithgar](https://github.com/wraithgar)) * [`f3a662fcd`](npm/cli@f3a662f) [#3211](npm/cli#3211) fix(tests): use config defaults ([@wraithgar](https://github.com/wraithgar)) * [`285976fd1`](npm/cli@285976f) `@npmcli/arborist@2.4.4` * fix(reify): properly save spec if prerelease * [`f9f24d17c`](npm/cli@f9f24d1) `libnpmexec@1.1.1` * fix(add): Specify 'en' locale to String.localeCompare * [`cb9f17499`](npm/cli@cb9f174) `glob@7.1.7` * force 'en' locale in string sorting * [`24b4e4a41`](npm/cli@24b4e4a) `ignore-walk@3.0.4` * Avoid locale-specific sorting issues * [`1eb7e5c7d`](npm/cli@1eb7e5c) `@npmcli/arborist@2.4.3` * guard against locale-specific sorting * [`a6a826067`](npm/cli@a6a8260) `npm-packlist@2.2.2`: * fix(sort): avoid locale-dependent sorting issues * [`701627c51`](npm/cli@701627c) [#3098](npm/cli#3098) feat(cache): Allow `add` to accept multiple specs ([@mjsir911](https://github.com/mjsir911)) * [`59171f030`](npm/cli@59171f0) [#3187](npm/cli#3187) feat(config): add workspaces boolean to user-agent ([@nlf](https://github.com/nlf)) * [`2c9b8713c`](npm/cli@2c9b871) [#3182](npm/cli#3182) fix(docs): fix broken links ([@wangsai](https://github.com/wangsai)) * [`88cbc8c44`](npm/cli@88cbc8c) [#3198](npm/cli#3198) fix(tests): reflect new libnpmexec logic * [`d01ce5e13`](npm/cli@d01ce5e) `libnpmexec@1.1.0`: * feat: add walk up dir lookup to satisfy local bins * [`81c1dfaaa`](npm/cli@81c1dfa) `@npmcli/arborist@2.4.2`: * fix(add): save packages in the right place * fix(reify): do not clean up nodes with no parent * fix(audit): support alias specs & root package names * [`87c2303ea`](npm/cli@87c2303) `@npmcli/git@2.0.9`: * fix(clone): Do not allow git replacement objects by default * [`99ff40dff`](npm/cli@99ff40d) `npm-packlist@2.2.0`: * feat(npmignore): Do not force include history, changelogs, notice * fix(package.json): add missing bin/index.js to files * [`c371f183e`](npm/cli@c371f18) [#3137](npm/cli#3137) [#3140](npm/cli#3140) fix(ls): do not warn on missing optional deps ([@isaacs](https://github.com/isaacs)) * [`861f606c7`](npm/cli@861f606) [#3156](npm/cli#3156) fix(build): make prune rule work on case-sensitive file systems ([@lpinca](https://github.com/lpinca)) * [`fb79d89a0`](npm/cli@fb79d89) `tap@15.0.6` * [`ce3820043`](npm/cli@ce38200) `@npmcli/arborist@2.4.1` * fix: prevent and eliminate unnecessary duplicates * fix: support resolvable partial intersecting peerSets * [`e479f1dac`](npm/cli@e479f1d) [#3146](npm/cli#3146) mention `directories.bin` in `bin` ([@felipecrs](https://github.com/felipecrs)) * [`7925cca24`](npm/cli@7925cca) `pacote@11.3.3`: * fix(registry): normalize manfest * [`b61eac693`](npm/cli@b61eac6) [#3130](npm/cli#3130) `@npmcli/config@2.2.0` * [`c74e67fc6`](npm/cli@c74e67f) [#3130](npm/cli#3130) `npm-registry-fetch@10.1.1` * [`efdd7dd44`](npm/cli@efdd7dd) Remove unused and incorrectly documented `--always-auth` config definition ([@isaacs](https://github.com/isaacs)) * [`4c1f16d2c`](npm/cli@4c1f16d) [#3095](npm/cli#3095) feat(init): add workspaces support ([@ruyadorno](https://github.com/ruyadorno)) * [`42ca59eee`](npm/cli@42ca59e) [#3086](npm/cli#3086) fix(ls): do not exit with error when all problems are extraneous deps ([@nlf](https://github.com/nlf)) * [`2aecec591`](npm/cli@2aecec5) [#2724](npm/cli#2724) [#3119](npm/cli#3119) fix(ls): make --long work when missing deps ([@ruyadorno](https://github.com/ruyadorno)) * [`42e0587a9`](npm/cli@42e0587) [#3115](npm/cli#3115) fix(pack): refuse to pack invalid packument ([@wraithgar](https://github.com/wraithgar)) * [`1c4eff7b5`](npm/cli@1c4eff7) [#3126](npm/cli#3126) fix(logout): use isBasicAuth attribute ([@wraithgar](https://github.com/wraithgar)) * [`c93f1c39e`](npm/cli@c93f1c3) [#3101](npm/cli#3101) chore(docs): update view docs ([@wraithgar](https://github.com/wraithgar)) * [`c4ff4bc11`](npm/cli@c4ff4bc) [npm/statusboard#313](npm/statusboard#313) [#3109](npm/cli#3109) fix(usage): fix refs to ws shorthand ([@ruyadorno](https://github.com/ruyadorno)) * [`83166ebcc`](npm/cli@83166eb) `npm-registry-fetch@10.1.0` * feat(auth): set isBasicAuth * [`e02bda6da`](npm/cli@e02bda6) `npm-registry-fetch@10.0.0` * feat(auth) load/send based on URI, not registry * [`a0382deba`](npm/cli@a0382de) `@npmcli/run-script@1.8.5` * fix: windows ComSpec env variable name * [`7f82ef5a8`](npm/cli@7f82ef5) `pacote@11.3.2` * [`35e49b94f`](npm/cli@35e49b9) `@npmcli/arborist@2.4.0` * [`95faf8ce6`](npm/cli@95faf8c) `libnpmaccess@4.0.2` * [`17fffc0e4`](npm/cli@17fffc0) `libnpmhook@6.0.2` * [`1b5a213aa`](npm/cli@1b5a213) `libnpmorg@2.0.2` * [`9f83e6484`](npm/cli@9f83e64) `libnpmpublish@4.0.1` * [`251f788c5`](npm/cli@251f788) `libnpmsearch@3.1.1` * [`35873a989`](npm/cli@35873a9) `libnpmteam@2.0.3` * [`23e12b4d8`](npm/cli@23e12b4) `npm-profile@5.0.3`
Package manager/ecosystem
Javascript, yarn and npm
Manifest contents prior to update
NA
Updated dependency
NA
What you expected to see, versus what you actually saw
We expect to see concurrent updating of dependencies for multiple repositories can succeed.
Instead we got exception complaining
$HOME/.gitconfig
does not exist when dependabot-core tries to move it to a backup.Images of the diff or a link to the PR, issue or logs
NA
Some early investigation
The problem is probably around https://github.com/dependabot/dependabot-core/blob/main/common/lib/dependabot/shared_helpers.rb#L156 and
dependabot-core/common/lib/dependabot/shared_helpers.rb
Line 258 in 6f40826
$HOME/.gitconfig
while the other process or thread is doing the same thing.Is it possible to configure git locally instead of globally so multiple repositories can be handled in parallel?
Thanks.
The text was updated successfully, but these errors were encountered: