-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added option to also re-insert deleted minc files (Same as PR#179) #193
Merged
Merged
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
fc2577f
added option to also re-insert
gluneau c1af357
Remove profile changes
gluneau 70a415e
removing IBIS specific fields
gluneau 9ea717b
No need to show another example of sql query
gluneau 034f8b8
adding documentation and making sql more generic
gluneau 136a12f
more updates
gluneau 4fda8a7
fix path to run from mri directory
gluneau e1bd202
Option to re-insert deleted minc files: Based off of PR 179
MounaSafiHarab dc67f78
remove the original file
MounaSafiHarab b44d7dc
better readme
MounaSafiHarab 5466c6c
Cecile comments
MounaSafiHarab 799c496
relative to example scripts directory
MounaSafiHarab 34ff07c
Modified the deletemincsqlwrapper.pl so that when no files are found …
cmadjar bc11abb
revert to t1 from adniT1
MounaSafiHarab 77fb427
remove the wrong file
MounaSafiHarab 1b87d6d
minor thing
MounaSafiHarab caa62de
chmod, run tarchiveLoader without seriesUID as an argument, and small…
MounaSafiHarab File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
### Example scripts | ||
|
||
- **deletemincsqlwrapper.pl** | ||
- An example script to delete multiple minc files fitting a common criterion from the database. | ||
- The script also provides the option to re-insert deleted scans with their seriesUID when using the `-insertminc` flag. | ||
- **Projects should modify the query as needed to suit their needs**. | ||
- For the example query provided (in `$queryF`), all inserted scans with types like `t1` or `t2`, having a `slice thickness` in the range of `4 mm` will be deleted. | ||
- A use case of this deletion query might be that initially the project did not exclude `t1` or `t2` modalities having 4 mm slice thickness, and subsequently, the | ||
study `mri_protocol` has been changed to add tighter checks on slice thickness. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
#!/usr/bin/perl | ||
use strict; | ||
use warnings; | ||
use Getopt::Tabular; | ||
no warnings 'once'; | ||
use Data::Dumper; | ||
use File::Basename; | ||
use File::Copy; | ||
use Term::ANSIColor qw(:constants); | ||
use NeuroDB::DBI; | ||
|
||
my $profile = undef; | ||
my $insertminc; | ||
|
||
my @opt_table = ( | ||
[ "Basic options", "section" ], | ||
[ | ||
"-profile", "string", 1, \$profile, | ||
"name of config file in ../../dicom-archive/.loris_mri" | ||
], | ||
[ | ||
"-insertminc", "boolean", 0, \$insertminc, "Re-insert the deleted minc" | ||
] | ||
); | ||
|
||
my $Help = <<HELP; | ||
****************************************************************************** | ||
Wrapper to minc_deletion.pl for bulk deletion based on a SQL customisable Query | ||
****************************************************************************** | ||
|
||
This script is a wrapper for deleting multiple mincs at a time and optionally | ||
re-inserting them. | ||
|
||
It will pause before for confirmation before deleting. | ||
|
||
Please note that this is an example script. Projects need to customize the query | ||
based on their needs. Please refer to the README in the tools/example_scripts/ | ||
directory for more details. | ||
|
||
HELP | ||
|
||
my $Usage = <<USAGE; | ||
usage: tools/example_scripts/deletemincsqlwrapper.pl -profile prod | ||
$0 -help to list options | ||
USAGE | ||
&Getopt::Tabular::SetHelp( $Help, $Usage ); | ||
&Getopt::Tabular::GetOptions( \@opt_table, \@ARGV ) || exit 1; | ||
|
||
{ package Settings; do "$ENV{LORIS_CONFIG}/.loris_mri/" . $profile} | ||
my $dbh = &NeuroDB::DBI::connect_to_db(@Settings::db); | ||
|
||
|
||
|
||
# Only the f.SeriesUID is really needed for minc_deletion, other fields are for information only | ||
# If you plan to re-insert, you'll also need ArchiveLocation | ||
my $queryF = <<SQL; | ||
SELECT DISTINCT f.fileid, f.SeriesUID, f.SessionID, f.file, t.ArchiveLocation, FROM_UNIXTIME(f.InsertTime), p.Value, q.QCStatus, c.Alias, m.Scan_type | ||
FROM files AS f | ||
LEFT JOIN parameter_file AS p using (FileID) | ||
LEFT JOIN parameter_type AS pt using (ParameterTypeID) | ||
LEFT JOIN files_qcstatus AS q using (FileID) | ||
LEFT JOIN session AS s ON (f.SessionID=s.ID) | ||
LEFT JOIN psc AS c ON (c.CenterID=s.CenterID) | ||
LEFT JOIN mri_scan_type AS m ON (m.ID=f.AcquisitionProtocolID) | ||
LEFT JOIN tarchive AS t ON f.TarchiveSource=t.TarchiveID | ||
WHERE pt.Name = 'acquisition:slice_thickness' | ||
AND p.Value LIKE '%4.%' | ||
AND (m.Scan_type LIKE '%t1%' OR m.Scan_type LIKE '%t2%') | ||
ORDER BY FROM_UNIXTIME(f.InsertTime) | ||
SQL | ||
|
||
|
||
my $sthF = $dbh->prepare($queryF); | ||
|
||
my $keepgoing = 1; | ||
my ($fF, $stdin, $i); | ||
|
||
printf ("%-6s", '| L# '); | ||
printf ("%-64s",'| SeriesUID'); | ||
printf ("%-20s",'| Value'); | ||
printf ("%-20s",'| Scan Type'); | ||
printf ("%-60s",'| File'); | ||
print "|\n"; | ||
|
||
$sthF->execute(); | ||
|
||
if ($sthF->rows > 0) { | ||
|
||
while ($fF = $sthF->fetchrow_hashref()) { | ||
|
||
$i++; | ||
|
||
printf ("%-6s", '| '. $i); | ||
printf ("%-64s",'| '. $fF->{'SeriesUID'}); | ||
printf ("%-20s",'| '. $fF->{'Value'}); | ||
printf ("%-20s",'| '. $fF->{'Scan_type'}); | ||
printf ("%-60s",'| '. $fF->{'file'}); | ||
print "|\n"; | ||
|
||
if ($keepgoing) { | ||
print "Press ENTER (or A and ENTER to do it all)\n"; | ||
$stdin = <STDIN>; | ||
if ($stdin eq "A\n") { | ||
print "Ok, I will keep going until it's done.\n"; | ||
$keepgoing = 0; | ||
} | ||
} | ||
|
||
my $minc_delete_cmd = "uploadNeuroDB/minc_deletion.pl -profile " . $profile . " -seriesuid " . $fF->{'SeriesUID'} . " confirm"; | ||
print $minc_delete_cmd . "\n"; | ||
my $minc_delete_log = `$minc_delete_cmd`; | ||
print $minc_delete_log . "\n"; | ||
|
||
if ($insertminc) { | ||
# Running tarchiveLoader on the archived tar as a whole will only insert new minc files that are not already in the files table | ||
my $tar_loader_cmd = "uploadNeuroDB/tarchiveLoader -profile " . $profile . " -verbose -globLocation " . $fF->{'ArchiveLocation'}; | ||
print $tar_loader_cmd . "\n"; | ||
my $tar_loader_log = `$tar_loader_cmd`; | ||
print $tar_loader_log . "\n"; | ||
} | ||
} | ||
} else { | ||
print "\n***No files were found that match the following query:***\n\n" . $queryF . "\n"; | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could be good to add a reference to the README section so that if people wants to have a better idea of what the script does and how to use it, they can refer to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added a line there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point, done