Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make BagGenerator parallelism configurable #8602

Closed
qqmyers opened this issue Apr 12, 2022 · 0 comments · Fixed by #8606
Closed

Make BagGenerator parallelism configurable #8602

qqmyers opened this issue Apr 12, 2022 · 0 comments · Fixed by #8606
Assignees
Labels
TDL of interest to the Texas Digital Library
Milestone

Comments

@qqmyers
Copy link
Member

qqmyers commented Apr 12, 2022

Overview of the Feature Request
The generation of archival Bags currently uses 8 threads to read datafiles when generating the zipped Bag. As part of work for TDL related to #7493, the associated PR lowers the default to 2 threads and makes threading configurable via a :BagGeneratorThreads setting which can be used with the existing archiver classes.

What kind of user is the feature intended for?
Sysadmin

What inspired the request?
Interest in lowering the server load when archiving, particularly in the context of batch archiving as in #7493

Any related open or closed issues to this feature request? #7493

@qqmyers qqmyers added the TDL of interest to the Texas Digital Library label Apr 12, 2022
@qqmyers qqmyers self-assigned this Apr 12, 2022
pdurbin added a commit that referenced this issue Apr 14, 2022
@pdurbin pdurbin added this to the 5.11 milestone May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
TDL of interest to the Texas Digital Library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants