Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Circumvented bug that made tar fail on NFS file systems in get_dicom_script.pl and added option -id #409

Merged
merged 17 commits into from
Mar 18, 2019

Conversation

nicolasbrossard
Copy link
Collaborator

This PR provides a fix for the following use case:

Let's say a user wants to extract a very large set of C files. Since the extraction algorithm extracts all the wanted files in a temporary directory and then archives the entire set in a C<.tar.gz> file, the temporary directory has to be able to (temporarily) store a huge amount of a data. Now suppose that on the user's system, the temporary directory can only be on a NFS file system because of space issues. The script will fail since C has been known to incorrectly report that some files changed during the archiving process if the files are located on an NFS file system, stopping the script dead in its tracks.

The solution is to extract small portions of the data set at a time in the temporary directory, archive them and then immediately delete them from the temporary directory. This reduces the free space requirement imposed on this directory, which in turn provides a fix for the problem described above, assuming that the user has access to a non-NFS temporary directory with sufficient free space.

Copy link

@PapillonMcGill PapillonMcGill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

No tested of course

Copy link
Collaborator

@cmadjar cmadjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicolasbrossard See my one comment on the PR. Also, could we sit together tomorrow and you could show me how to properly test the PR? My /data directory is an NFS mount and I did not have any issue with your script before. Would not mind being able to reproduce the issue you found before approving the PR.

Thanks!

tools/get_dicom_files.pl Show resolved Hide resolved
tools/get_dicom_files.pl Outdated Show resolved Hide resolved
Copy link
Collaborator

@cmadjar cmadjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicolasbrossard I added a few comments based on your changes.

One additional comment: can you update the Description of the PR to say that you also modified the behaviour with the -p option and explain it? That will make my life easier at the time of writing the release notes. Thanks! :)

docs/scripts_md/get_dicom_files.md Outdated Show resolved Hide resolved
tools/get_dicom_files.pl Outdated Show resolved Hide resolved
tools/get_dicom_files.pl Outdated Show resolved Hide resolved
docs/scripts_md/get_dicom_files.md Outdated Show resolved Hide resolved
@nicolasbrossard nicolasbrossard changed the title Circumvented bug that made tar fail on NFS file systems in get_dicom_script.pl Circumvented bug that made tar fail on NFS file systems in get_dicom_script.pl and added optiopn -id Mar 18, 2019
@nicolasbrossard nicolasbrossard changed the title Circumvented bug that made tar fail on NFS file systems in get_dicom_script.pl and added optiopn -id Circumvented bug that made tar fail on NFS file systems in get_dicom_script.pl and added option -id Mar 18, 2019
docs/scripts_md/get_dicom_files.md Show resolved Hide resolved
docs/scripts_md/get_dicom_files.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@cmadjar cmadjar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works perfectly!

@cmadjar cmadjar merged commit 6ebfcce into aces:minor Mar 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants