Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Perl pipeline] Do not duplicate violation entry if it already exists #896

Merged

Conversation

cmadjar
Copy link
Collaborator

@cmadjar cmadjar commented Mar 2, 2023

Description

When the tarchiveLoader.pl script is rerun, it checks whether the file has already been registered into the files table before inserting a new entry in the files table but the pipeline does not check whether a violation has already been registered into the 3 violation tables (MRICandidateErrors, mri_protocol_violated_scans and mri_violations_log) and inserts duplicates of the same violation as many times as tarchiveLoader.pl is run on a DICOM archive.

This was fixed on the python side by #884 and #891. This PR fixes this for the Perl side as well using the same algorithm.

Reminder: a unique file entry is now determined by the unique combination of SeriesUID, EchoTime, PhaseEncodingDirection and EchoNumbers.

Resolves #890 as this is the only missing piece to close the issue.

check if entry already in mri_protocol_violated_scans
handle mri_violations_log duplicate
forgot to import MriViolationsLogOB in MRIProcessingUtility
fix bug where file was moved when it should not have
translate if else statement into perl...
forgot Echotime when inserting into mri_violations_log
commit markdown files
@cmadjar cmadjar force-pushed the do_not_duplicate_violation_if_already_exists_perl branch from 70746c4 to 238d07b Compare March 2, 2023 21:20
Copy link
Collaborator

@nicolasbrossard nicolasbrossard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General question: are integrity constraints going to be imposed (on the LORIS side) on the violations table so that duplicate inserts always fail?

uploadNeuroDB/NeuroDB/MRI.pm Show resolved Hide resolved
uploadNeuroDB/NeuroDB/MRIProcessingUtility.pm Show resolved Hide resolved
uploadNeuroDB/minc_insertion.pl Show resolved Hide resolved
@cmadjar
Copy link
Collaborator Author

cmadjar commented Mar 6, 2023

General question: are integrity constraints going to be imposed (on the LORIS side) on the violations table so that duplicate inserts always fail?

See aces/Loris#8382. However, I have an issue with adding a key for the mri_violations_log table. See here: aces/Loris#8382 (comment). I have no idea how to resolve this...

Also added last in the 3 if statements as requested. Thank you!

@cmadjar cmadjar merged commit 1e6fe05 into aces:main Mar 17, 2023
@cmadjar cmadjar deleted the do_not_duplicate_violation_if_already_exists_perl branch March 17, 2023 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure no duplicates can be inserted in the different violation tables
2 participants