Show stk model

[javierggt]

Description

Changes to display the radiation model from STK in Kalman watch.

In order to add kalman drops data to the perigee detail page (in kalman_perigee_mon), the following attributes are added to the EventPerigee class:

• kalman_drops_nman
• kalman_drops_npnt
• predicted_kalman_drops

Also:

• added a notebook with the derivation of the scale factor between STK radiation and the IR flag data.
• increased the number of cached ACA images to 70, so it accommodates a typical call with a look-back of 45 days.
• added a simple gitignore with just a few entries.
• modified the task schedule so the last script can use the output of the second one
• removed unused function _reshape_to_n_sample_2d
• as part of removing the matplotlib code, the plot_kalman_drops function was removed. This function is referenced from several docstrings, so the docstrings still need to be fixed.

Note:

• the plotly html is produced in one script, but it is used in another script. This is how it was done before this PR (the png file being produced in a separate script).

Some changes were done on a separate PR (#18) and marged into this branch before merging this PR:

• the radiation model data is fetched from a file in SKA/data/stk.
• The new kalman_perigee_mon.EventPerigee.get_kalman_drops_nman function requires some parameters that would need to be passed all the way from the command line options to the called code in monitor_win_perigee.py. A config class was added to deal with this.
• Both kalman_perigee_mon.py and monitor_win_perigee.py produce plots/html, and both have functions to fetch data. To modify the perigee detail page, I noticed there would be a dependence cycle between the two. I copied or moved some of the functions to make things work.

Interface impacts

Changes the plot that appears in SSAWG trending summary, so the scraping code needs to be changed.

This PR requires the following Ska data: $SKA/data/stk/radiation_data.fits

Testing

see below

Unit tests

• No unit tests

Functional tests

Tests were done on PR #18, which was daisy-chained to this one. The squash-and-merge discarded that PR, so the testing is copied below:

Scripts were run in three different ways:

• emulating a single call, exactly as it is in the task schedule (except data-dir is different):

  kalman_watch_kalman_perigee_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/branch  \
      --make-html
  kalman_watch_monitor_win_perigee \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/branch
  kalman_watch_low_kalman_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/branch
  

  The output is here. In this case, the low-Kalman plot is empty because there is no data from previous runs.
• Using kalman_watch_monitor_win_perigee to produce a png of the monitoring window plot:

  kalman_watch_monitor_win_perigee \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/branch \
      --out-file mon_win_kalman_drops_-45d_-1d.png
  

  output: mon_win_kalman_drops_-45d_-1d.png
• successive calls to the script, imitating what the task schedule does, on a weekly basis to produce a history of data. Note that this is NOT exactly what is run in the crontab/task_schedule. This is because, in the crontab, the time range is automatically set to dates relative to the time the script is run. Also, the input/output file depends on this date. In this test, I ran with a changing date (not relative) so the file would change each time. The commands run are like these:

  kalman_watch_kalman_perigee_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/yearly \
      --make-html \
      --stop <stop>
  kalman_watch_monitor_win_perigee \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/yearly \
      --stop <stop> \
      --start <start> \
      --out-file=kalman_plot.html
  kalman_watch_low_kalman_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/yearly \
      --stop <stop> \
      --in-file=kalman_plot.html
  

  output
• A single call of the scripts, based on the latest flight data. Copied the weekly.2024-09-08_0015 snapshot of the kalman_watch3 data (found in data/mta4) into my data directory and ran:

  kalman_watch_kalman_perigee_mon \
     --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/run-once-on-latest \
     --make-html
  kalman_watch_monitor_win_perigee \
     --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/run-once-on-latest
  kalman_watch_low_kalman_mon \
     --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/run-once-on-latest
  

  output
• a single call using incomplete radiation data so not all observations should have predicted IR flag data:

  kalman_watch_kalman_perigee_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/no-rad-data  \
      --rad-data=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/rad_data_truncated.fits \
      --make-html
  kalman_watch_monitor_win_perigee \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/no-rad-data \
      --rad-data=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/rad_data_truncated.fits
  kalman_watch_low_kalman_mon \
      --data-dir=$SKA/www/ASPECT_ICXC/test_review_outputs/kalman_watch/pr-16/no-rad-data
  

  output. In this case, the last two perigees should have no predicted IR flag data.