Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deal with memory leaking through temporary R folders #401

Merged
merged 28 commits into from
Oct 13, 2023
Merged

Deal with memory leaking through temporary R folders #401

merged 28 commits into from
Oct 13, 2023

Conversation

EagleoutIce
Copy link
Member

No description provided.

@EagleoutIce
feat-fix: defere runNext work stealing step in parallel to the even…
29a4b7d 
…t queue to avoid too-deep call stack
@EagleoutIce
refactor: await promise resolve for parallel task defer
e26d0a4
@EagleoutIce
feat-fix: statistics-file newline sanitation should escape *all* newl…
5250fa9 
…ines
@EagleoutIce
refactor: 5ms delay for parallel queue to give the queue a little bit…
c97a892 
… time to breathe
@EagleoutIce
feat-fix: explicitly unlinking tempfiles
b1a93a4
@EagleoutIce
refactor: lazy tempfile provide in the summarizer (avoid on-load)
8a9d840
@EagleoutIce
refactor: refine exit events, beforeExit for the summarizer and on …
759b5f3 
…shell exit for the link
@EagleoutIce
feat-fix: try explicit tempdir handling in our session
1043794
@EagleoutIce
feat-fix: unlink tempfiles in R
104abcc
@EagleoutIce
feat: await compression in statistics helper
bc05db6
@EagleoutIce
feat-fix: do not increment counter on predicate fail
f6e7da4
@EagleoutIce
refactor: move writeout to successful only
a083e94
@EagleoutIce
refactor: jsonReplacer for the meta stats (for consistency)
519a486
@EagleoutIce
refactor: start marker for stat helper
e703ac5
@EagleoutIce
refactor: remove early file check again as this destroyed every sched…
c5f34fc 
…uling idea :c
@EagleoutIce
refactor: try to speedup parallel queue
780d96e
@EagleoutIce
refactor: try to speed up compression again (without precheck)
b3c9727
@EagleoutIce
feat: more memory-robust serialization of graph
06cf9df
@EagleoutIce
feat-fix: try to improve object recovery
f378a6e
@EagleoutIce
feat: further improve objectJson with recursion
ce38270
@EagleoutIce
refactor: we want something before the path regex marker
7c06f1c
@EagleoutIce
refactor: move arguments map to the beginning
a65772b
@EagleoutIce
refactor: try to further calls to check if file exists
d7aa413
@EagleoutIce
refactor: skip during file collection
d0abd92
@EagleoutIce
feat-fix: output dir name for target
2048a4b
@EagleoutIce
feat-fix: waiting for is now a set, should be printed with the json r…
422f6fc 
…eplacer
@EagleoutIce
lint-fix: remove todo comments (outsourced into issues)
0294ed8
@EagleoutIce EagleoutIce linked an issue Oct 13, 2023 that may be closed by this pull request
Memory leak in tmp folder for long-running parallel process! #395
Closed
@codecov
Copy link

codecov bot commented Oct 13, 2023

Codecov Report

Attention: 19 lines in your changes are missing coverage. Please review.

Files Coverage Δ
src/benchmark/stats/summarizer.ts 59.39% <100.00%> (+1.58%) ⬆️
src/r-bridge/shell.ts 89.65% <100.00%> (+0.63%) ⬆️
src/cli/common/script.ts 54.16% <50.00%> (+1.99%) ⬆️
src/statistics/output/statistics-file.ts 74.07% <0.00%> (ø)
src/core/print/dataflow-printer.ts 29.16% <26.08%> (-37.50%) ⬇️

📢 Thoughts on this report? Let us know!.

@EagleoutIce EagleoutIce merged commit 7818979 into main Oct 13, 2023
12 checks passed
@EagleoutIce EagleoutIce deleted the 395-memory-leak-in-tmp-folder-for-long-running-process branch October 13, 2023 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Memory leak in tmp folder for long-running parallel process!
1 participant
@EagleoutIce