hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels #3698

foobarbecue · 2020-02-20T21:32:33Z

ISIS version(s) affected: 3.9.1

Description
When running hyb2onc2isis with any of the FITS images JAXA released for their July 2018 orbits, from http://darts.isas.jaxa.jp/pub/hayabusa2/onc_bundle/browse/ , hyb2onc2isis fails with:

**PROGRAMMER ERROR** No value or default value to translate for translation group [InstrumentId] in file [/data/JPL_Stereo/Isis/data/hayabusa2/translations/hyb2oncInstrument.trn].

How to reproduce
Install ISIS 4.0.0, download hyb2_onc_20180710_060508_tvf_l2a.fit and run hyb2onc2isis from=hyb2_onc_20180710_060508_tvf_l2a.fit to=hyb2_onc_20180710_060508_tvf_l2a.cub

Possible Solution
Labels on newer Hyabusa images are presumably different from labels used in older images against which hyb2onc2isis was developed.

Additional context
Initially discussed at https://astrodiscuss.usgs.gov/t/problem-injesting-hyabusa2-data/374/6

The text was updated successfully, but these errors were encountered:

foobarbecue · 2020-02-20T21:39:06Z

We could also fix some inconsistencies in the hyb2onc2isis docs regarding input file types at the same time.

AustinSanders · 2020-02-27T17:04:22Z

For reference, the related issue was resolved by adding a default value to the transformation file. It is likely that this will involve a similar fix.

foobarbecue · 2020-03-03T00:45:45Z

I started digging into this. I dumped a .fits label and compared to the .trn files.

Lots needs changing. I was able to figure out many of the changes, but some labels appear not to be present and so I just commented them out... probably the wrong approach. Now, hyb2onc2isis works but spiceinit fails with **ERROR** No Camera Kernels found for the image [hyb2_onc_20180801_094857_tvf_l2a.cub]

Where are these .trn files hosted, and how do I propose changes to them? The README in the same folder says "These translation files are maintained and archived on astrogit under the fy17_hayabusa2 repo."

In the meantime, here's a diff of the changes I made in order to get hyb2onc2isis to stop outputting errors:
diff.txt

edit: I also commented out the ImageNumber group in hyb2oncArchive.trn , but that didn't make it into the above diff.

jessemapel · 2020-03-03T16:44:45Z

@foobarbecue We're currently working to get those files pushed into the repo. For right now though, only the USGS ASC has access to make changes. We've got this issue slotted for work at the end of March so we'll take you're changes and start off of them getting this resolved.

The spiceinit error looks like it's caused by a lack of kernels. We only have kernels up to 2016 in $ISIS3DATA/hayabusa2/kernels/ck, I'm not sure why that is. @kberryUSGS do we have a kernel update script for Hayabusa 2?

Thanks for putting so much work into this, it'll help us get this resolved much easier.

krlberry · 2020-03-04T15:38:46Z

We do have a kernel update script for Hayabusa2. It's pulling kernels from http://www.darts.isas.jaxa.jp/pub/hayabusa2/spice_bundle/spice_kernels/, but the most recent cks there are from 2016. I'm not sure if there is a more up-to-date source.

jlaura · 2020-03-10T15:17:10Z

@scsides Do you want to check with the team and see if they have updated CKs that we can pull?

foobarbecue · 2020-03-13T00:37:24Z

I see kernels up to 2019 are now available at https://www.darts.isas.jaxa.jp/pub/hayabusa2/spice_bundle/spice_kernels , and sisdist.astrogeology.usgs.gov rsync. Great, I'll see if I get spiceinit to work now.

foobarbecue · 2020-03-13T00:49:58Z

Almost!

(isis3) aacurtis@lmmp-pipeline-ubuntu:/trek.devdata/ryugu/l2a/20180801$ spiceinit from=hyb2_onc_20180801_155438_tvf_l2a.cub
Group = Kernels
  NaifFrameCode             = -37100
  LeapSecond                = $base/kernels/lsk/naif0012.tls
  TargetAttitudeShape       = $hayabusa2/kernels/pck/pck00010.tpc
  TargetPosition            = $hayabusa2/kernels/tspk/de430.bsp
  InstrumentPointing        = ($hayabusa2/kernels/ck/hyb2_hkattrpt_2018_v02.b-
                               c, $hayabusa2/kernels/fk/hyb2_v14.tf)
  Instrument                = $hayabusa2/kernels/ik/hyb2_onc_v05.ti
  SpacecraftClock           = $hayabusa2/kernels/sclk/hyb2_20141203-20161231_-
                              v01.tsc
  InstrumentPosition        = Null
  InstrumentAddendum        = $hayabusa2/kernels/iak/hyb2oncAddendum0001.ti
  ShapeModel                = Null
  InstrumentPositionQuality = Unknown
  InstrumentPointingQuality = Reconstructed
  CameraVersion             = 1
  Error                     = "Can not find [BODY2162173_RADII] in text
                               kernels"
End_Group

Group = Kernels
  NaifFrameCode             = -37100
  LeapSecond                = $base/kernels/lsk/naif0012.tls
  TargetAttitudeShape       = $hayabusa2/kernels/pck/pck00010.tpc
  TargetPosition            = $hayabusa2/kernels/tspk/de430.bsp
  InstrumentPointing        = ($hayabusa2/kernels/ck/hyb2_aocsc_2018_v02.bc,
                               $hayabusa2/kernels/fk/hyb2_v14.tf)
  Instrument                = $hayabusa2/kernels/ik/hyb2_onc_v05.ti
  SpacecraftClock           = $hayabusa2/kernels/sclk/hyb2_20141203-20161231_-
                              v01.tsc
  InstrumentPosition        = Null
  InstrumentAddendum        = $hayabusa2/kernels/iak/hyb2oncAddendum0001.ti
  ShapeModel                = Null
  InstrumentPositionQuality = Unknown
  InstrumentPointingQuality = Reconstructed
  CameraVersion             = 1
  Error                     = "Can not find [BODY2162173_RADII] in text
                               kernels"
End_Group
**ERROR** Unable to initialize camera model.

jessemapel · 2020-03-13T14:47:07Z

That's a pck error. Ryugu isn't in pck10, try downloading this: https://www.darts.isas.jaxa.jp/pub/hayabusa2/spice_bundle/spice_kernels/pck/hyb2_ryugu_shape_v20190328.tpc and passing it via the pck parameter.

foobarbecue · 2020-03-21T03:19:58Z

Well, I'm not sure why the change but now I'm getting:

(isis3) aacurtis@lmmp-pipeline-ubuntu:/trek.devdata/ryugu/l2a/20180801$ spiceinit from=hyb2_onc_20180801_174821_tvf_l2a.cub
Group = Kernels
  LeapSecond                = $base/kernels/lsk/naif0012.tls
  TargetAttitudeShape       = $hayabusa2/kernels/pck/pck00010.tpc
  TargetPosition            = $hayabusa2/kernels/tspk/de430.bsp
  InstrumentPointing        = ($hayabusa2/kernels/ck/hyb2_hkattrpt_2018_v02.b-
                               c, $hayabusa2/kernels/fk/hyb2_v14.tf)
  Instrument                = $hayabusa2/kernels/ik/hyb2_onc_v05.ti
  SpacecraftClock           = $hayabusa2/kernels/sclk/hyb2_20141203-20161231_-
                              v01.tsc
  InstrumentPosition        = Null
  InstrumentAddendum        = $hayabusa2/kernels/iak/hyb2oncAddendum0001.ti
  ShapeModel                = Null
  InstrumentPositionQuality = Unknown
  InstrumentPointingQuality = Reconstructed
  CameraVersion             = 1
  Error                     = "PVL Keyword [NaifIkCode] does not exist in
                               [Group = Kernels]"
End_Group

Group = Kernels
  LeapSecond                = $base/kernels/lsk/naif0012.tls
  TargetAttitudeShape       = $hayabusa2/kernels/pck/pck00010.tpc
  TargetPosition            = $hayabusa2/kernels/tspk/de430.bsp
  InstrumentPointing        = ($hayabusa2/kernels/ck/hyb2_aocsc_2018_v02.bc,
                               $hayabusa2/kernels/fk/hyb2_v14.tf)
  Instrument                = $hayabusa2/kernels/ik/hyb2_onc_v05.ti
  SpacecraftClock           = $hayabusa2/kernels/sclk/hyb2_20141203-20161231_-
                              v01.tsc
  InstrumentPosition        = Null
  InstrumentAddendum        = $hayabusa2/kernels/iak/hyb2oncAddendum0001.ti
  ShapeModel                = Null
  InstrumentPositionQuality = Unknown
  InstrumentPointingQuality = Reconstructed
  CameraVersion             = 1
  Error                     = "PVL Keyword [NaifIkCode] does not exist in
                               [Group = Kernels]"
End_Group
**ERROR** Unable to initialize camera model.

jlaura · 2020-03-23T14:05:39Z

hyb2onc2isis needs updating to deal with new Hyabusa2 FITS image labels

krlberry · 2020-03-31T01:44:59Z

Hi @foobarbecue, the changes you recommended were merged in PR #3813 . I needed to make an additional change to fix the NaifIkCode error you experienced, see the change to Hayabusa2OncKernels.trn in the PR. The updated software will be available in the ISIS 4.1 release candidate, which is currently being worked on and I believe is slated to be released by the end of this week or early next week. Thank you again for the diff, it saved a lot of time!

After I was able to ingest your example Hayabusa2 fits file, to get the resulting cube spiceiniting, I did need to make some changes to the hayabusa2 kernels area. These changes will be available on the rsync server when the release candidate is released. If you'd like them earlier, I could also push an earlier update, just let me know!

foobarbecue · 2020-03-31T15:10:55Z

Wonderful! I will definitely use this in the next few days, if you are able to push the earlier kernel area update.

jessemapel · 2020-03-31T15:59:24Z

@foobarbecue We'll be pushing the most recent kernels we have as we've changed the data area around for 4.1. It looks like we have coverage up to beginning of 2019 currently.

jessemapel · 2020-04-03T19:31:36Z

@kberryUSGS Can this be closed?

foobarbecue · 2020-04-03T22:06:31Z

Hi, I just got around to trying this with the 4.1.0 RC and from spiceinit I get:

Error                     = "Can not find [BODY2162173_RADII] in text
                               kernels"

I don't need Hayabusa 1 kernels for Hayabusa2, do I?

jessemapel · 2020-04-03T22:16:08Z

The Hayabusa2 pck kerneldb isn't pulling in the ryugu PCK. Try adding PCK=$ISISDATA/hayabusa2/kernels/pck/hyb2_ryugu_shape_v20190328.tpc.

@kberryUSGS Is it okay if I manually update the kernel.db?

krlberry · 2020-04-05T18:54:44Z

@jessemapel I already updated the kernel.db to pick up the new pcks, but they weren't pushed to the rsync server until just now. I pushed just the hayabusa2 kernel changes to the rsync server today.

@foobarbecue Try updating your hayabusa2 data area using the following for use with the ISIS 4.1.0 RC:

cd $ISISDATA
rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/hayabusa2 . --dry-run

I added the --dry-run above so you can see if the changes proposed by rsync are reasonable. If the results for the above rsync look reasonable, run without the --dry-run to actually pull down the changes.

You should see that PCK file (among other changes) come down in your rsync.

The rsync module name has changed from isis3data to isisdata for data for use with ISIS 4.1.0 RC. This information (and more!) will be in the ISIS 4.1.0 RC release notes when the RC is officially released.

foobarbecue · 2020-04-05T23:05:18Z

Thanks guys, after updating kernels from the isisdata rsync I was able to successfully run hyb2onc2isis followed by spiceinit. I then tried hyb2onccal and got:

(isis4) aacurtis@lmmp-pipeline-ubuntu:/trek.devdata/ryugu/l2a/20180801$ hyb2onccal from=hyb2_onc_20180801_100809_tvf_l2a.cub to=from=hyb2_onc_20180801_100809_tvf_l2a.cal.cub
**ERROR** PVL Keyword [SelectedImageAreaY4] does not exist in [Group = Instrument].

foobarbecue · 2020-04-05T23:56:08Z

Think I figured it out, working on a quick patch

jlaura added enhancement New feature or request Missions Issues which are a priority for missions labels Feb 23, 2020

foobarbecue mentioned this issue Feb 26, 2020

kaguyatc2isis cannot ingest data provided by Jaxa online archive #3670

Closed

krlberry changed the title ~~hyb2onc2isis needs updating to deal with new Hyabusa2 FITS image labels~~ hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels Mar 5, 2020

jessemapel mentioned this issue Mar 12, 2020

Move the application data into the git repo #3727

Closed

krlberry self-assigned this Mar 26, 2020

krlberry mentioned this issue Mar 29, 2020

Update hyb2onc2isis to support newer image format #3813

Merged

11 tasks

krlberry mentioned this issue Mar 31, 2020

Update Hayabusa2 Kernel Download Script #3815

Closed

foobarbecue mentioned this issue Apr 6, 2020

Fix: translate 2nd coord of ROI to (X4, Y4), not (X2, Y2), as expected by hyb2onccal #3827

Merged

7 tasks

krlberry closed this as completed Apr 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels #3698

hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels #3698

foobarbecue commented Feb 20, 2020 •

edited

Loading

foobarbecue commented Feb 20, 2020

AustinSanders commented Feb 27, 2020

foobarbecue commented Mar 3, 2020 •

edited

Loading

jessemapel commented Mar 3, 2020

krlberry commented Mar 4, 2020 •

edited

Loading

jlaura commented Mar 10, 2020

foobarbecue commented Mar 13, 2020 •

edited

Loading

foobarbecue commented Mar 13, 2020

jessemapel commented Mar 13, 2020 •

edited

Loading

foobarbecue commented Mar 21, 2020 •

edited

Loading

jlaura commented Mar 23, 2020

krlberry commented Mar 31, 2020 •

edited

Loading

foobarbecue commented Mar 31, 2020

jessemapel commented Mar 31, 2020

jessemapel commented Apr 3, 2020

foobarbecue commented Apr 3, 2020 •

edited

Loading

jessemapel commented Apr 3, 2020

krlberry commented Apr 5, 2020 •

edited

Loading

foobarbecue commented Apr 5, 2020

foobarbecue commented Apr 5, 2020

hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels #3698

hyb2onc2isis needs updating to deal with new Hayabusa2 FITS image labels #3698

Comments

foobarbecue commented Feb 20, 2020 • edited Loading

foobarbecue commented Feb 20, 2020

AustinSanders commented Feb 27, 2020

foobarbecue commented Mar 3, 2020 • edited Loading

jessemapel commented Mar 3, 2020

krlberry commented Mar 4, 2020 • edited Loading

jlaura commented Mar 10, 2020

foobarbecue commented Mar 13, 2020 • edited Loading

foobarbecue commented Mar 13, 2020

jessemapel commented Mar 13, 2020 • edited Loading

foobarbecue commented Mar 21, 2020 • edited Loading

jlaura commented Mar 23, 2020

krlberry commented Mar 31, 2020 • edited Loading

foobarbecue commented Mar 31, 2020

jessemapel commented Mar 31, 2020

jessemapel commented Apr 3, 2020

foobarbecue commented Apr 3, 2020 • edited Loading

jessemapel commented Apr 3, 2020

krlberry commented Apr 5, 2020 • edited Loading

foobarbecue commented Apr 5, 2020

foobarbecue commented Apr 5, 2020

foobarbecue commented Feb 20, 2020 •

edited

Loading

foobarbecue commented Mar 3, 2020 •

edited

Loading

krlberry commented Mar 4, 2020 •

edited

Loading

foobarbecue commented Mar 13, 2020 •

edited

Loading

jessemapel commented Mar 13, 2020 •

edited

Loading

foobarbecue commented Mar 21, 2020 •

edited

Loading

krlberry commented Mar 31, 2020 •

edited

Loading

foobarbecue commented Apr 3, 2020 •

edited

Loading

krlberry commented Apr 5, 2020 •

edited

Loading