Add method to check and skip duplicate content uploads to S3 - for 24.1-release #1037
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmadjar
added a commit
that referenced
this pull request
Jan 18, 2024
* Update VERSION to 24.1.13 * Fixes for parameter file value lists (#1025) * rm conversion to str on value list * add check to ensure list is not empty * Add method to check and skip duplicate content uploads to S3 - for 24.1-release (#1037) * get Kim's code on 24.1-release branch * fix flake8 * fix flake8 * remove pyblake2 from pip requirements.txt file as LORIS-MRI uses hashlib now (#1038) * update version to 24.1.14 for bug fix release (#1039) * Add a script to correct incorrectly saved list in parameter file (#1034) * query parameter_file for the lists incorrectly saved * finalize the update query * finalize script help section --------- Co-authored-by: breen0074 <105391285+breen0074@users.noreply.github.com>
cmadjar
added a commit
that referenced
this pull request
Mar 15, 2024
* Update VERSION to 24.1.13 * Fixes for parameter file value lists (#1025) * rm conversion to str on value list * add check to ensure list is not empty * Add method to check and skip duplicate content uploads to S3 - for 24.1-release (#1037) * get Kim's code on 24.1-release branch * fix flake8 * fix flake8 * remove pyblake2 from pip requirements.txt file as LORIS-MRI uses hashlib now (#1038) * update version to 24.1.14 for bug fix release (#1039) * Add a script to correct incorrectly saved list in parameter file (#1034) * query parameter_file for the lists incorrectly saved * finalize the update query * finalize script help section * [EEG BIDS] Support assembly_bids source for datasets (LORIS 24.1) (#1046) * [EEG BIDS] Support assembly_bids source for datasets * flake8 * Add executable permission * QA comment * Add support for config prePackagedDownloadPath and EEGChunksPath * Update the bids_import script logic to explicitely add the --no-copy option * fix incorrect conflict resolution * fix incorrect conflict resolution * bids_import.py - Missing short option fix --------- Co-authored-by: breen0074 <105391285+breen0074@users.noreply.github.com> Co-authored-by: Laetitia Fesselier <laetitia.fesselier@mail.mcgill.ca>
cmadjar
added a commit
that referenced
this pull request
Mar 15, 2024
* Update VERSION to 24.1.13 * Fixes for parameter file value lists (#1025) * rm conversion to str on value list * add check to ensure list is not empty * Add method to check and skip duplicate content uploads to S3 - for 24.1-release (#1037) * get Kim's code on 24.1-release branch * fix flake8 * fix flake8 * remove pyblake2 from pip requirements.txt file as LORIS-MRI uses hashlib now (#1038) * update version to 24.1.14 for bug fix release (#1039) * Add a script to correct incorrectly saved list in parameter file (#1034) * query parameter_file for the lists incorrectly saved * finalize the update query * finalize script help section * [EEG BIDS] Support assembly_bids source for datasets (LORIS 24.1) (#1046) * [EEG BIDS] Support assembly_bids source for datasets * flake8 * Add executable permission * QA comment * Add support for config prePackagedDownloadPath and EEGChunksPath * Update the bids_import script logic to explicitely add the --no-copy option * fix flake8 * bids_import.py - Missing short option fix --------- Co-authored-by: breen0074 <105391285+breen0074@users.noreply.github.com> Co-authored-by: Laetitia Fesselier <laetitia.fesselier@mail.mcgill.ca>
cmadjar
added a commit
that referenced
this pull request
Mar 18, 2024
* Update VERSION to 24.1.13 * Fixes for parameter file value lists (#1025) * rm conversion to str on value list * add check to ensure list is not empty * Add method to check and skip duplicate content uploads to S3 - for 24.1-release (#1037) * get Kim's code on 24.1-release branch * fix flake8 * fix flake8 * remove pyblake2 from pip requirements.txt file as LORIS-MRI uses hashlib now (#1038) * update version to 24.1.14 for bug fix release (#1039) * Add a script to correct incorrectly saved list in parameter file (#1034) * query parameter_file for the lists incorrectly saved * finalize the update query * finalize script help section * [EEG BIDS] Support assembly_bids source for datasets (LORIS 24.1) (#1046) * [EEG BIDS] Support assembly_bids source for datasets * flake8 * Add executable permission * QA comment * Add support for config prePackagedDownloadPath and EEGChunksPath * Update the bids_import script logic to explicitely add the --no-copy option * Update set file path override and chunks logic (#1050) * Update set file path override and chunks logic * flake8 * Update set file path override and chunks logic - bugfix (#1051) --------- Co-authored-by: breen0074 <105391285+breen0074@users.noreply.github.com> Co-authored-by: Laetitia Fesselier <laetitia.fesselier@mail.mcgill.ca>
cmadjar
added a commit
that referenced
this pull request
Mar 18, 2024
* Update VERSION to 24.1.13 * Fixes for parameter file value lists (#1025) * rm conversion to str on value list * add check to ensure list is not empty * Add method to check and skip duplicate content uploads to S3 - for 24.1-release (#1037) * get Kim's code on 24.1-release branch * fix flake8 * fix flake8 * remove pyblake2 from pip requirements.txt file as LORIS-MRI uses hashlib now (#1038) * update version to 24.1.14 for bug fix release (#1039) * Add a script to correct incorrectly saved list in parameter file (#1034) * query parameter_file for the lists incorrectly saved * finalize the update query * finalize script help section * [EEG BIDS] Support assembly_bids source for datasets (LORIS 24.1) (#1046) * [EEG BIDS] Support assembly_bids source for datasets * flake8 * Add executable permission * QA comment * Add support for config prePackagedDownloadPath and EEGChunksPath * Update the bids_import script logic to explicitely add the --no-copy option * Update set file path override and chunks logic (#1050) * Update set file path override and chunks logic * flake8 * Update set file path override and chunks logic - bugfix (#1051) * fix bad resolution * fix bad resolution --------- Co-authored-by: breen0074 <105391285+breen0074@users.noreply.github.com> Co-authored-by: Laetitia Fesselier <laetitia.fesselier@mail.mcgill.ca>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as #1032 but going to the 24.1-release branch