Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added option to also re-insert deleted minc files (Same as PR#179) #193

Merged
merged 17 commits into from
Oct 2, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 0 additions & 73 deletions tools/deletemincsqlwrapper.pl

This file was deleted.

11 changes: 11 additions & 0 deletions tools/example_scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
### Example scripts

- **deletemincsqlwrapper.pl**
- An example script to delete multiple minc files fitting a common criterion from the database.
- The script also provides the option to re-insert deleted scans with their seriesUID when using the `-insertminc` flag.
- **Projects should modify the query as needed to suit their needs**.
- For the example query provided (in `$queryF`), all inserted scans with types like `t1` or `t2`, having a `slice thickness` in the range of `4 mm` will be deleted.
- A use case of this deletion query might be that initially the project did not exclude `t1` or `t2` modalities having 4 mm slice thickness, and subsequently, the
study `mri_protocol` has been changed to add tighter checks on slice thickness.


124 changes: 124 additions & 0 deletions tools/example_scripts/deletemincsqlwrapper.pl
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Tabular;
no warnings 'once';
use Data::Dumper;
use File::Basename;
use File::Copy;
use Term::ANSIColor qw(:constants);
use NeuroDB::DBI;

my $profile = undef;
my $insertminc;

my @opt_table = (
[ "Basic options", "section" ],
[
"-profile", "string", 1, \$profile,
"name of config file in ../../dicom-archive/.loris_mri"
],
[
"-insertminc", "boolean", 0, \$insertminc, "Re-insert the deleted minc"
]
);

my $Help = <<HELP;
******************************************************************************
Wrapper to minc_deletion.pl for bulk deletion based on a SQL customisable Query
******************************************************************************

This script is a wrapper for deleting multiple mincs at a time and optionally
re-inserting them.

It will pause before for confirmation before deleting.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be good to add a reference to the README section so that if people wants to have a better idea of what the script does and how to use it, they can refer to it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a line there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, done

Please note that this is an example script. Projects need to customize the query
based on their needs. Please refer to the README in the tools/example_scripts/
directory for more details.

HELP

my $Usage = <<USAGE;
usage: tools/example_scripts/deletemincsqlwrapper.pl -profile prod
$0 -help to list options
USAGE
&Getopt::Tabular::SetHelp( $Help, $Usage );
&Getopt::Tabular::GetOptions( \@opt_table, \@ARGV ) || exit 1;

{ package Settings; do "$ENV{LORIS_CONFIG}/.loris_mri/" . $profile}
my $dbh = &NeuroDB::DBI::connect_to_db(@Settings::db);



# Only the f.SeriesUID is really needed for minc_deletion, other fields are for information only
# If you plan to re-insert, you'll also need ArchiveLocation
my $queryF = <<SQL;
SELECT DISTINCT f.fileid, f.SeriesUID, f.SessionID, f.file, t.ArchiveLocation, FROM_UNIXTIME(f.InsertTime), p.Value, q.QCStatus, c.Alias, m.Scan_type
FROM files AS f
LEFT JOIN parameter_file AS p using (FileID)
LEFT JOIN parameter_type AS pt using (ParameterTypeID)
LEFT JOIN files_qcstatus AS q using (FileID)
LEFT JOIN session AS s ON (f.SessionID=s.ID)
LEFT JOIN psc AS c ON (c.CenterID=s.CenterID)
LEFT JOIN mri_scan_type AS m ON (m.ID=f.AcquisitionProtocolID)
LEFT JOIN tarchive AS t ON f.TarchiveSource=t.TarchiveID
WHERE pt.Name = 'acquisition:slice_thickness'
AND p.Value LIKE '%4.%'
AND (m.Scan_type LIKE '%t1%' OR m.Scan_type LIKE '%t2%')
ORDER BY FROM_UNIXTIME(f.InsertTime)
SQL


my $sthF = $dbh->prepare($queryF);

my $keepgoing = 1;
my ($fF, $stdin, $i);

printf ("%-6s", '| L# ');
printf ("%-64s",'| SeriesUID');
printf ("%-20s",'| Value');
printf ("%-20s",'| Scan Type');
printf ("%-60s",'| File');
print "|\n";

$sthF->execute();

if ($sthF->rows > 0) {

while ($fF = $sthF->fetchrow_hashref()) {

$i++;

printf ("%-6s", '| '. $i);
printf ("%-64s",'| '. $fF->{'SeriesUID'});
printf ("%-20s",'| '. $fF->{'Value'});
printf ("%-20s",'| '. $fF->{'Scan_type'});
printf ("%-60s",'| '. $fF->{'file'});
print "|\n";

if ($keepgoing) {
print "Press ENTER (or A and ENTER to do it all)\n";
$stdin = <STDIN>;
if ($stdin eq "A\n") {
print "Ok, I will keep going until it's done.\n";
$keepgoing = 0;
}
}

my $minc_delete_cmd = "uploadNeuroDB/minc_deletion.pl -profile " . $profile . " -seriesuid " . $fF->{'SeriesUID'} . " confirm";
print $minc_delete_cmd . "\n";
my $minc_delete_log = `$minc_delete_cmd`;
print $minc_delete_log . "\n";

if ($insertminc) {
# Running tarchiveLoader on the archived tar as a whole will only insert new minc files that are not already in the files table
my $tar_loader_cmd = "uploadNeuroDB/tarchiveLoader -profile " . $profile . " -verbose -globLocation " . $fF->{'ArchiveLocation'};
print $tar_loader_cmd . "\n";
my $tar_loader_log = `$tar_loader_cmd`;
print $tar_loader_log . "\n";
}
}
} else {
print "\n***No files were found that match the following query:***\n\n" . $queryF . "\n";
}