Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to Cassis Template and XML and Minor corrections to isisimport #4490

Merged
merged 3 commits into from
May 25, 2021
Merged

Updates to Cassis Template and XML and Minor corrections to isisimport #4490

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a1450cc
Updates to cassis template. Corrections to isisimport to allow module…
May 25, 2021
4062383
Added disclaimer to xml file.
May 25, 2021
66ee931
Merge branch 'pds4' into testTemplate
amystamile-usgs May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 37 additions & 12 deletions isis/appdata/import/cassis.lbl.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
{% set IdArea="Product_Observational.Identification_Area" %}
{% set FileArea="Product_Observational.File_Area_Observational" %}
{% set threeDArray="Product_Observational.File_Area_Observational.Array_3D_Image" %}
{% set cart="Product_Observational.Observation_Area.Discipline_Area.cart_Cartography" %}

Object = IsisCube

Object = Core
{% if exists(threeDArray) %}
Group = Dimensions
Samples = {{ PO.File_Area_Observational.Array_3D_Image.Axis_Array.0.elements }}
Expand Down Expand Up @@ -33,8 +34,9 @@ Object = IsisCube
Type = Real
{% endif %}
ByteOrder = Lsb
{% if exists("Product_Observational.File_Area_Observational.Array_3D_Image.Element_Array.offset") %}
Base = {{ PO.File_Area_Observational.Array_3D_Image.Element_Array.offset }}

Base = {% if exists("Product_Observational.File_Area_Observational.Array_3D_Image.Element_Array.offset") %}
{{ PO.File_Area_Observational.Array_3D_Image.Element_Array.offset }}
{% else %}
{{ PO.File_Area_Observational.Array_3D_Image.Element_Array.value_offset }}
{% endif %}
Expand Down Expand Up @@ -68,14 +70,15 @@ Object = IsisCube
Type = Real
{% endif %}
ByteOrder = Lsb
{% if exists("Product_Observational.File_Area_Observational.Array_2D_Image.Element_Array.offset") %}
Base = {{ PO.File_Area_Observational.Array_2D_Image.Element_Array.offset }}
Base = {% if exists("Product_Observational.File_Area_Observational.Array_2D_Image.Element_Array.offset") %}
{{ PO.File_Area_Observational.Array_2D_Image.Element_Array.offset }}
{% else %}
{{ PO.File_Area_Observational.Array_2D_Image.Element_Array.value_offset }}
{% endif %}
Multiplier = {{ PO.File_Area_Observational.Array_2D_Image.Element_Array.scaling_factor }}
End_Group
{% endif %}
End_Object

Group = Instrument
SpacecraftName = {% if exists("Product_Observational.Observation_Area.Investigation_Area.name") %}
Expand Down Expand Up @@ -134,6 +137,9 @@ Object = IsisCube
{% if exists(IdArea + ".Producer_data") %}
ProducerName = "{{ PO.Identification_Area.Producer_data.Producer_full_name }}"
{% endif %}
{% if exists(IdArea + ".Product_Id") %}
ProductId = "{{ PO.Identification_Area.Product_Id }}"
{% endif %}
{% if exists(FileArea + ".File.creation_date_time") %}
ProductCreationTime = {{ PO.File_Area_Observational.File.creation_date_time }}
{% endif %}
Expand Down Expand Up @@ -193,7 +199,7 @@ Object = IsisCube
{% if exists(CassHeader) %}
ExposureTimePEHK = {{ PO.CaSSIS_Header.PEHK_HEADER.attrib_Exposure_Time }} <ms>
{% endif %}
{% if exists(CassHeader) %}
{% if exists(CassHeader + ".DERIVED_HEADER_DATA.PixelsPossiblySaturated") %}
PixelsPossiblySaturated = {{ PO.CaSSIS_Header.DERIVED_HEADER_DATA.PixelsPossiblySaturated._text }}
{% endif %}
{% if exists(CassHeader) %}
Expand All @@ -206,15 +212,9 @@ Object = IsisCube
FiltersAvailable = "{{ PO.CaSSIS_Header.CaSSIS_General.FILTERS_AVAILABLE }}"
{% endif %}
{% if exists(CassHeader) %}
FocalLength = {{ PO.CaSSIS_Header.CaSSIS_General.TELESCOPE_FOCAL_LENGTH._text }}
{% endif %}
{% if exists(CassHeader) %}
FocalLengthUnit = {{ PO.CaSSIS_Header.CaSSIS_General.TELESCOPE_FOCAL_LENGTH.attrib_Unit }}
{% endif %}
{% if exists(CassHeader) %}
TelescopeFNumber = {{ PO.CaSSIS_Header.CaSSIS_General.TELESCOPE_F_NUMBER }}
{% endif %}
{% if exists(CassHeader) %}
TelescopeType = "{{ PO.CaSSIS_Header.CaSSIS_General.TELESCOPE_TYPE }}"
{% endif %}
{% if exists(CassHeader) %}
Expand Down Expand Up @@ -365,6 +365,30 @@ Object = IsisCube
NaifFrameCode = -143400
End_Group

{% if exists(cart) %}
Group = Mapping
ProjectionName = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Map_Projection.cart_map_projection_name }}
CenterLongitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Map_Projection.cart_Equirectangular.cart_longitude_of_central_meridian }}
CenterLatitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Map_Projection.cart_Equirectangular.cart_latitude_of_projection_origin }}
Scale = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Planar_Coordinate_Information.cart_Coordinate_Representation.cart_pixel_scale_x }}
PixelResolution = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Planar_Coordinate_Information.cart_Coordinate_Representation.cart_pixel_resolution_x }} <meters/pixel>
MaximumLatitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Domain.cart_Bounding_Coordinates.cart_north_bounding_coordinate }}
MinimumLatitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Domain.cart_Bounding_Coordinates.cart_south_bounding_coordinate }}
MaximumLongitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Domain.cart_Bounding_Coordinates.cart_west_bounding_coordinate }}
MinimumLongitude = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Domain.cart_Bounding_Coordinates.cart_east_bounding_coordinate }}
UpperLeftCornerX = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Geo_Transformation.cart_upperleft_corner_x }} <meters>
UpperLeftCornerY = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Planar.cart_Geo_Transformation.cart_upperleft_corner_y }} <meters>
EquatorialRadius = {% if exists(cart + ".cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Geodetic_Model.cart_semi_major_radius") %}
{{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Geodetic_Model.cart_semi_major_radius }}
{% endif %}
PolarRadius = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Geodetic_Model.cart_polar_radius }} <meters>
LatitudeType = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Geodetic_Model.cart_latitude_type }}
LongitudeDirection = {{ PO.Observation_Area.Discipline_Area.cart_Cartography.cart_Spatial_Reference_Information.cart_Horizontal_Coordinate_System_Definition.cart_Geodetic_Model.cart_longitude_direction }}
TargetName = {{ PO.Observation_Area.Target_Identification.name }}
LongitudeDomain = 360
End_Group
{% endif %}

{% if exists(CassHeader) %}
{% set windowNumber=int(PO.CaSSIS_Header.FSW_HEADER.attrib_WindowCounter) + 1 %}
{% if windowNumber == 1 %}
Expand Down Expand Up @@ -417,3 +441,4 @@ Object = IsisCube
{% endif %}
End_Object
End

100 changes: 94 additions & 6 deletions isis/src/base/apps/isisimport/isisimport.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@

#include <inja/inja.hpp>
#include <nlohmann/json.hpp>
#include <QString>
#include <QtMath>

#include "CubeAttribute.h"
#include "FileName.h"
#include "iTime.h"
#include "OriginalXmlLabel.h"
#include "XmlToJson.h"
#include "ProcessImport.h"

Expand All @@ -16,6 +20,8 @@ using json = nlohmann::json;

namespace Isis {

static QString convertUniqueIdToObservationId(Pvl &outputLabel);

void isisimport(UserInterface &ui, Pvl *log) {
FileName xmlFileName = ui.GetFileName("FROM");

Expand All @@ -36,16 +42,26 @@ namespace Isis {

// To read the DN data
ProcessImport importer;
importer.SetInputFile(xmlFileName.removeExtension().addExtension("img").expanded());
if(xmlFileName.removeExtension().addExtension("dat").fileExists()){
importer.SetInputFile(xmlFileName.removeExtension().addExtension("dat").expanded());
}
else if (xmlFileName.removeExtension().addExtension("img").fileExists()) {
importer.SetInputFile(xmlFileName.removeExtension().addExtension("img").expanded());
}
else {
QString msg = "Cannot find image file for [" + xmlFileName.name() + "]. Confirm that the "
".dat or .img file for this XML exists and is located in the same directory.";
throw IException(IException::User, msg, _FILEINFO_);
}

// Set everything needed by ProcessImport
PvlGroup dimensions = newLabel.findObject("IsisCube").findGroup("Dimensions");
PvlGroup dimensions = newLabel.findObject("IsisCube").findObject("Core").findGroup("Dimensions");
int ns = toInt(dimensions["Samples"]);
int nl = toInt(dimensions["Lines"]);
int nb = toInt(dimensions["Bands"]);
importer.SetDimensions(ns, nl, nb);

PvlGroup pixels = newLabel.findObject("IsisCube").findGroup("Pixels");
PvlGroup pixels = newLabel.findObject("IsisCube").findObject("Core").findGroup("Pixels");
QString pixelType = pixels["Type"];
QString byteOrder = pixels["ByteOrder"];
double base = pixels["Base"];
Expand All @@ -55,24 +71,96 @@ namespace Isis {
importer.SetBase(base);
importer.SetMultiplier(multiplier);

// TODO: how to handle this?
importer.SetFileHeaderBytes(0);
// TODO: how to handle this?
importer.SetFileHeaderBytes(0);

CubeAttributeOutput &att = ui.GetOutputAttribute("TO");
Cube *outputCube = importer.SetOutputCube(ui.GetFileName("TO"), att);

OriginalXmlLabel xmlLabel;
xmlLabel.readFromXmlFile(xmlFileName);

importer.StartProcess();

// Write the updated label
outputCube->write(xmlLabel);

// Write the updated label
Isis::PvlObject &newCubeLabel = newLabel.findObject("IsisCube");
Isis::Pvl &outLabel(*outputCube->label());
Isis::PvlObject &outCubeLabel = outLabel.findObject("IsisCube");

for(int g = 0; g < newCubeLabel.groups(); g++) {
outCubeLabel.addGroup(newCubeLabel.group(g));
}

// Remove trailing "Z" from PDS4 .xml (on re-ingestion) and create YearDoy keyword in Archive group
PvlKeyword *startTime = &outLabel.findGroup("Instrument", Pvl::Traverse)["StartTime"];
QString startTimeString = startTime[0];
if (startTimeString.endsWith("Z", Qt::CaseInsensitive)) {
startTimeString.chop(1);
startTime->setValue(startTimeString);
}
iTime stime(startTimeString);
PvlGroup &archive = outLabel.findGroup("Archive", Pvl::Traverse);
PvlKeyword yeardoy("YearDoy", toString(stime.Year()*1000 + stime.DayOfYear()));
archive.addKeyword(yeardoy);

if (!outputCube->group("Archive").hasKeyword("ObservationId")){
convertUniqueIdToObservationId(outLabel);
}

importer.EndProcess();

return;
}

QString convertUniqueIdToObservationId(Pvl &outputLabel) {
if (outputLabel.findObject("IsisCube").hasGroup("Mosaic")) {
return ""; // translation file should auto translate this case to Mosaic group.
// For any other product, this ID goes in the Archive group.
}

QString target = "";
if (outputLabel.findObject("IsisCube").hasGroup("Instrument")) {
target = outputLabel.findGroup("Instrument", Pvl::Traverse)
.findKeyword("TargetName")[0];
}
else {
target = outputLabel.findGroup("Mapping", Pvl::Traverse)
.findKeyword("TargetName")[0];
}

PvlGroup &archiveGroup = outputLabel.findGroup("Archive", Pvl::Traverse);
QString uniqueId = archiveGroup.findKeyword("UniqueIdentifier")[0];

QString observationId = "";
BigInt uniqueIdDecimalValue = uniqueId.toLongLong();
BigInt operationPeriod = (uniqueIdDecimalValue & 1879048192);
operationPeriod /= qPow(2,28);
FileName transFile("$ISISROOT/appdata/translations/TgoCassisOperationPeriod.trn");
PvlTranslationTable transTable(transFile);
observationId = transTable.Translate("OperationPeriod", toString(operationPeriod));
BigInt orbitNumber = (uniqueIdDecimalValue & 268433408);
orbitNumber /= qPow(2,11);
observationId += "_";
observationId += QString("%1").arg(orbitNumber, 6, 10, QChar('0'));

int orbitPhase = (uniqueIdDecimalValue & 2044);
if (target.compare("mars", Qt::CaseInsensitive) == 0) {
orbitPhase /= qPow(2,2);
}
else {
orbitPhase = 900;
}
observationId += "_";
observationId += QString("%1").arg(orbitPhase, 3, 10, QChar('0'));

int imageType = (uniqueIdDecimalValue & 3);
observationId += "_";
observationId += toString(imageType);

archiveGroup += PvlKeyword("ObservationId", observationId);

return observationId;
}
}
13 changes: 9 additions & 4 deletions isis/src/base/apps/isisimport/isisimport.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,25 @@

<application name="isisimport" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://isis.astrogeology.usgs.gov/Schemas/Application/application.xsd">
<brief>
Convert from PDS 4 format (other formats will be supported in the future) using template XML files to an ISIS Cube.
Convert from PDS 4 format (other formats will be supported in the future) using template XML files to an ISIS Cube.
</brief>

<description>
<p>
NOTE: This program and the associated template files are incomplete. The documentation
describes what the eventual intended capabilities are. Many capabiliteis are not implemented
at this time.
</p>
<p>
Reads a PDS4 compatible label file and image file. The contents of the ISIS cube label file are
generated using the template file specified by the TEMPLATE parameter.
</p>
<p>
This application uses the <a href="https://pantor.github.io/inja">Inja templating engine</a>
to render the template file. The input data from the PDS4 label
to render the template file. The input data from the PDS4 label
is converted to JSON and then it is accessed via the Inja templating syntax.
See the <a href="https://pantor.github.io/inja">Inja documentation</a> for the full template
syntax.
syntax.
</p>
</description>

Expand Down Expand Up @@ -64,7 +69,7 @@
Isis Cube
</brief>
<description>
The ingested ISIS Cube.
The ingested ISIS Cube.
</description>
</parameter>

Expand Down