[dcm2bids] Adds support for images that do not have an acquisition date #777
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmadjar
changed the title
cmadjar
added a commit
that referenced
this pull request
Jul 22, 2022
* Turn off column statistics option when running mysqldump (#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8
cmadjar
added a commit
that referenced
this pull request
Aug 16, 2022
* Turn off column statistics option when running mysqldump (#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787)
cmadjar
added a commit
that referenced
this pull request
Aug 17, 2022
* Turn off column statistics option when running mysqldump (#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * merge 24.0 release to 24.1-release (#791) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit
cmadjar
added a commit
that referenced
this pull request
Aug 22, 2022
* Turn off column statistics option when running mysqldump (#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * merge 24.0 release to 24.1-release (#791) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * Add PhaseEncodingDirection and EchoNumber to files and mri_violations_log insert statements (#785) * modify insertion into the mri_violations_log * modify is_file_unique function perl side * modify minc_insertion.pl script to add PhaseEncodingDirectin and EchoNumber to the file data object - perl side * check uniqueness of file based on seriesUID, EchoTime, EchoNumber and PhaseEncodingDirection in python scripts * populate PhaseEncDir, EchoTime, EchoNumber in insertion to files table for bids_import * set PhaseEncDir and EchoNb to none if not present in JSON file * update file non unique message * insert PhaseEncDir and EchoNb in mri_violations_log and files tables in python pipeline * fix missing parentheses * fix extra brackets in nifti_insertion.py * file matching type with regex instead of text as pybids returns either json or .json etc depending on it version * fix bug * fix phaseencodingdirection and echonumber set to null when inserting into files table for perl pipeline * Add phase encoding direction and echo number to mri protocol and mri protocol violated scans (#782) * add PhaseEncodingDirection and ScanOptions to mri_protocol and mri_protocol_violated_scans queries in the python code * update perl code to add the new fields in mri_protocol and mri_protocol_violated_scans * fix ScanOptions not being inserted correctly * use EchoNumber instead of ScanOptions as more generic * update doc * fix seriesUID not being inserted in mri_protocol_violated_scans * some print to figure out why scan is not inserted? * some print to figure out why scan is not inserted? * fix tiny bug * other prints * other prints * fix comparison of int and str for echo number comparison between protocol and json file content * Merge 24.0-release into 24.1-release (#796) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * fix table name to MRICandidateErrors instead of MriCandidateErrors as apparently, it makes a difference on MariaDB/Linux VMs while it just worked on local install on Mac... (#793) * Set DICOM dates to undef if the date does not follow proper DICOM standard (#794) * set date to undef if it does not follow proper DICOM formats * fix all dates set to NULL
cmadjar
added a commit
that referenced
this pull request
Oct 6, 2022
* Turn off column statistics option when running mysqldump (#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * merge 24.0 release to 24.1-release (#791) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * Add PhaseEncodingDirection and EchoNumber to files and mri_violations_log insert statements (#785) * modify insertion into the mri_violations_log * modify is_file_unique function perl side * modify minc_insertion.pl script to add PhaseEncodingDirectin and EchoNumber to the file data object - perl side * check uniqueness of file based on seriesUID, EchoTime, EchoNumber and PhaseEncodingDirection in python scripts * populate PhaseEncDir, EchoTime, EchoNumber in insertion to files table for bids_import * set PhaseEncDir and EchoNb to none if not present in JSON file * update file non unique message * insert PhaseEncDir and EchoNb in mri_violations_log and files tables in python pipeline * fix missing parentheses * fix extra brackets in nifti_insertion.py * file matching type with regex instead of text as pybids returns either json or .json etc depending on it version * fix bug * fix phaseencodingdirection and echonumber set to null when inserting into files table for perl pipeline * Add phase encoding direction and echo number to mri protocol and mri protocol violated scans (#782) * add PhaseEncodingDirection and ScanOptions to mri_protocol and mri_protocol_violated_scans queries in the python code * update perl code to add the new fields in mri_protocol and mri_protocol_violated_scans * fix ScanOptions not being inserted correctly * use EchoNumber instead of ScanOptions as more generic * update doc * fix seriesUID not being inserted in mri_protocol_violated_scans * some print to figure out why scan is not inserted? * some print to figure out why scan is not inserted? * fix tiny bug * other prints * other prints * fix comparison of int and str for echo number comparison between protocol and json file content * Merge 24.0-release into 24.1-release (#796) * Reload the mri_upload dictionary before checking if a tarchive has been validated (#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (#774) * fix regex search for excluded series description patterns (#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * fix table name to MRICandidateErrors instead of MriCandidateErrors as apparently, it makes a difference on MariaDB/Linux VMs while it just worked on local install on Mac... (#793) * Set DICOM dates to undef if the date does not follow proper DICOM standard (#794) * set date to undef if it does not follow proper DICOM formats * fix all dates set to NULL * set file number to undef if it is '' (#800) * [dcm2bids] Add missing get_scanner_candid function in imaging.py (#801) * add missing function called from database_config.py get_subject_ids function * add python function documentation * Add TaskName field to JSON side car file for modalities with task-* names (#802) * add taskName to JSON side car file for modalities with task-* names * add relevant comments * Add required IntendedFor BIDS field in fieldmap JSON files (#803) * add IntendedFor to fieldmaps automatically add visit_label in intended for paths * add method documentation * fix line too long * fix line too long * Update VERSION to 24.1.0 * Update release version in the README.md file * Fix batch_uploads_tarchive to redirect to dcm2bids pipeline when dcm2niix is set as a converter in the Configs (#804) * modify batch_uploads_tarchive so that it can run dcm2bids or dcm2mnc pipeline depending on the converter mentioned in the Config module * fix typo identified by Christine (#807) * [dcm2bids pipeline] Delete temporary directory created by the pipeline once it is done running (#806) * remove tmp dirs before exiting pipeline * add comments
zaliqarosli
pushed a commit
to zaliqarosli/Loris-MRI
that referenced
this pull request
Mar 9, 2023
* Turn off column statistics option when running mysqldump (aces#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (aces#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (aces#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (aces#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (aces#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (aces#774) * fix regex search for excluded series description patterns (aces#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (aces#787) * merge 24.0 release to 24.1-release (aces#791) * Reload the mri_upload dictionary before checking if a tarchive has been validated (aces#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (aces#774) * fix regex search for excluded series description patterns (aces#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (aces#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (aces#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit
zaliqarosli
pushed a commit
to zaliqarosli/Loris-MRI
that referenced
this pull request
Mar 9, 2023
* Turn off column statistics option when running mysqldump (aces#771) * turn off column statistics option when running mysqldump * add new option to the script to provide additional mysqldump options * run mass perldoc * fix perldoc * fix perldoc * check if date exists otherwise set it to null when inserting into the files table (aces#777) * [dcm2bids] Get scanner candidate's RegistrationProjectID based on the scanned candidate's visit ProjectID (aces#779) * add registrationProjectID to the function creating the candidate scanner * missed function call in base_pipeline * flake8 * Pull 24.0-release to 24.1-release (aces#788) * Reload the mri_upload dictionary before checking if a tarchive has been validated (aces#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (aces#774) * fix regex search for excluded series description patterns (aces#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (aces#787) * merge 24.0 release to 24.1-release (aces#791) * Reload the mri_upload dictionary before checking if a tarchive has been validated (aces#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (aces#774) * fix regex search for excluded series description patterns (aces#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (aces#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (aces#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * Add PhaseEncodingDirection and EchoNumber to files and mri_violations_log insert statements (aces#785) * modify insertion into the mri_violations_log * modify is_file_unique function perl side * modify minc_insertion.pl script to add PhaseEncodingDirectin and EchoNumber to the file data object - perl side * check uniqueness of file based on seriesUID, EchoTime, EchoNumber and PhaseEncodingDirection in python scripts * populate PhaseEncDir, EchoTime, EchoNumber in insertion to files table for bids_import * set PhaseEncDir and EchoNb to none if not present in JSON file * update file non unique message * insert PhaseEncDir and EchoNb in mri_violations_log and files tables in python pipeline * fix missing parentheses * fix extra brackets in nifti_insertion.py * file matching type with regex instead of text as pybids returns either json or .json etc depending on it version * fix bug * fix phaseencodingdirection and echonumber set to null when inserting into files table for perl pipeline * Add phase encoding direction and echo number to mri protocol and mri protocol violated scans (aces#782) * add PhaseEncodingDirection and ScanOptions to mri_protocol and mri_protocol_violated_scans queries in the python code * update perl code to add the new fields in mri_protocol and mri_protocol_violated_scans * fix ScanOptions not being inserted correctly * use EchoNumber instead of ScanOptions as more generic * update doc * fix seriesUID not being inserted in mri_protocol_violated_scans * some print to figure out why scan is not inserted? * some print to figure out why scan is not inserted? * fix tiny bug * other prints * other prints * fix comparison of int and str for echo number comparison between protocol and json file content * Merge 24.0-release into 24.1-release (aces#796) * Reload the mri_upload dictionary before checking if a tarchive has been validated (aces#783) * reload mri_upload object * remove debugging exit and print statements * fix minor bugs when dealing with scans.tsv files (aces#774) * fix regex search for excluded series description patterns (aces#786) * fix_return_statement_of_create_imaging_upload_dict_from_upload_id_function (aces#787) * [dcm2bids] Insert into MRICandidateErrors if there is a Candidate PatientName mismatch (aces#790) * insert into MRICandidateErrors when candidate mismatch or pname not matching between DICOMs and NIfTI * remove exit * fix table name to MRICandidateErrors instead of MriCandidateErrors as apparently, it makes a difference on MariaDB/Linux VMs while it just worked on local install on Mac... (aces#793) * Set DICOM dates to undef if the date does not follow proper DICOM standard (aces#794) * set date to undef if it does not follow proper DICOM formats * fix all dates set to NULL * set file number to undef if it is '' (aces#800) * [dcm2bids] Add missing get_scanner_candid function in imaging.py (aces#801) * add missing function called from database_config.py get_subject_ids function * add python function documentation * Add TaskName field to JSON side car file for modalities with task-* names (aces#802) * add taskName to JSON side car file for modalities with task-* names * add relevant comments * Add required IntendedFor BIDS field in fieldmap JSON files (aces#803) * add IntendedFor to fieldmaps automatically add visit_label in intended for paths * add method documentation * fix line too long * fix line too long * Update VERSION to 24.1.0 * Update release version in the README.md file * Fix batch_uploads_tarchive to redirect to dcm2bids pipeline when dcm2niix is set as a converter in the Configs (aces#804) * modify batch_uploads_tarchive so that it can run dcm2bids or dcm2mnc pipeline depending on the converter mentioned in the Config module * fix typo identified by Christine (aces#807) * [dcm2bids pipeline] Delete temporary directory created by the pipeline once it is done running (aces#806) * remove tmp dirs before exiting pipeline * add comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds support to insert images with no acquisition dates available. This is particularly pertinent when it comes to manage data supposed to be shared openly where the dates are considered identifying.
With this fix, the
filestable'sAcquisitionDatefield is set to the date available in the JSON if there is one, otherwise, it is set toNULL.