2021.1

[javierggt]

ska3-flight 2021.1

Summary:

This is a routine update of the Ska3 runtime environment including a substantial number of package updates.

The ACIS load review tools will be briefed separately by ACIS.

Interface Impacts:

No breaking changes, but these are some improvements of interest to Ska3 users that will be discussed. A detailed list of changes can be found in the 'code changes' section below.

• kadi 5.4.0
  • Add Ska.ParseCM compatibility + read_backstop function
• maude 3.4.0
  • Allow access without credentials when possible
• parse_cm 3.5.2
  • Add docs and modernize code
  • Support full parsing of obs request per OP19
  • Add cmds property to return kadi.commands.CommandTable version
• proseco 4.10.0
  • Built-in function for diffing ACA catalogs
• ska.engarchive 4.50.1
  • Better caching in fetch
• ska_helpers 0.3.0
  • Add retry decorator and function wrapper
  • Add lru_cache_timed decorator

Testing:

This release is available for testing as ska3-flight version 2020.1rc3 from the test conda channel.

Linux (HEAD)

Test results are displayed in the dashboard

Several tests are noted as failing in the dashboard:

• acisfp_check-tests and psmc_check post_check_logs.log fails: “ErfaWarning: ERFA function “dtf2d” yielded 2 of “dubious year (Note 6)” and similar.

  This is a benign warning related to the astropy release being over a year old and being worried that a leap second might have occurred. It has not, so OK. We plan to update to a more recent astropy soon.

• dpa_check post_check_logs.log fails: 1dpamzt violates zero-feps limit of 12.00 degC…

  This is just our testing tool not expecting this new warning. We will fix.

• mica test fails because the runner does not have sybase access.

  We are working on sybase access for the test runner account. By-hand run is OK.

MacOSX standalone

All tests pass with exception of the same ERFA warnings.

Windows 10 standalone

All tests pass with one exception.

The hopper package fails tests because it has embedded strings in the unit tests that are separated by \n but the package is expected Windows line separators os.linesep. This is acceptable because the hopper package is currently only used by starcheck, which does not run on Windows. This testing is newly added in this release of Ska3.

Review:

All operations critical or impacting PR's are independently and carefully reviewed. For other PR's the level of detail for review is calibrated to operations criticality. Some PR's that are confined to aspect-team-specific processing may have little to no independent review.

Deployment:

• The 2021.1 ska3-flight release will be deployed on HEAD and GRETA ska3/flight after the release and products for the week are approved.

Code changes

ska3-core changes (2020.14 -> 2021.1rc3)

Updated Packages

• pytest: 5.4.3 -> 6.2.0

ska3-flight changes (2020.14 -> 2021.1rc3)

Updated Packages

• aca_hi_bgd: 0.1.4 -> 0.1.5 (0.1.4 -> 0.1.5)
  • PR 9 (jeanconn): Rearrange processing at end to avoid reprocessing
• aca_view: 0.4.1 -> 0.5.0 (0.4.1 -> 0.5.0)
  • PR 64 (javierggt): Improved data fetching
• aca_weekly_report: 0.1.3 -> 0.1.4 (0.1.3 -> 0.1.4)
  • PR 7 (jeanconn): Fix the type of the days-back option
• acdc: 4.7.0 -> 4.7.1 (4.7.0 -> 4.7.1)
  • PR 59 (taldcroft): Catch another source of Telemetry Gap Error
• acis_thermal_check: 3.4.0 -> 3.5.0 (3.4.0 -> 3.4.1 -> 3.5.0)
  • PR 36 (jzuhone): Make regression testing work on standalone platforms
  • PR 37 (jzuhone): Refactoring of limit handling, allowing for more customizations, use cxotime
• acisfp_check: 3.2.0 -> 3.5.0 (3.2.0 -> 3.3.0 -> 3.4.0 -> 3.5.0)
  • PR 27 (jzuhone): Refactor tests to match new signature
  • PR 28 (jzuhone): Update answers post-shiny
  • PR 29 (jzuhone): Simplification of code, use cxotime, fixing bugs with violations reporting
• backstop_history: 1.3.0 -> 1.4.0 (1.3.0 -> 1.4.0)
  • PR 18 (jzuhone): Replace Chandra.Time with cxotime
• bep_pcb_check: 3.0.0 -> 3.3.0 (3.0.0 -> 3.1.0 -> 3.2.0 -> 3.3.0)
  • PR 4 (jzuhone): Refactor tests to match new signature
  • PR 5 (jzuhone): Update answers for shiny
  • PR 6 (jzuhone): Update regression testing
• dea_check: 3.1.0 -> 3.4.0 (3.1.0 -> 3.2.0 -> 3.3.0 -> 3.4.0)
  • PR 28 (jzuhone): Refactor tests to match new signature
  • PR 29 (jzuhone): Update answers for shiny
  • PR 30 (jzuhone): Update regression testing
• dpa_check: 3.0.0 -> 3.3.0 (3.0.0 -> 3.1.0 -> 3.2.0 -> 3.3.0)
  • PR 26 (jzuhone): Refactor tests to match new signature
  • PR 27 (jzuhone): Update answers for shiny
  • PR 28 (jzuhone): Add customizations to support checking +12 degC/0 FEPs violations, other minor updates
• fep1_actel_check: 3.0.0 -> 3.3.0 (3.0.0 -> 3.1.0 -> 3.2.0 -> 3.3.0)
  • PR 4 (jzuhone): Refactor tests to match new signature
  • PR 5 (jzuhone): Update answers for shiny
  • PR 6 (jzuhone): Update regression testing
• fep1_mong_check: 3.0.0 -> 3.3.0 (3.0.0 -> 3.1.0 -> 3.2.0 -> 3.3.0)
  • PR 4 (jzuhone): Refactor tests to match new signature
  • PR 5 (jzuhone): Update answers for shiny
  • PR 6 (jzuhone): Update regression testing
• hopper: 4.5.0 -> 4.5.1 (4.5.0 -> 4.5.1)
  • PR 20 (jeanconn): Package/install the tests
• kadi: 5.4.0 -> 5.5.0 (5.4.0 -> 5.5.0)
  • PR 196 (taldcroft): Handle cmds.h5 resource unavailable error
  • PR 195 (taldcroft): Add Ska.ParseCM compatibility + read_backstop function
  • PR 197 (taldcroft): Add test of creating cmds archive and retry opening cmds.h5
• maude: 3.4.0 -> 3.5.0 (3.4.0 -> 3.5.0)
  • PR 30 (taldcroft): Use LazyDict from ska_helpers.utils
  • PR 29 (taldcroft): Allow access without credentials when possible
  • PR 31 (javierggt): added blobs_to_table function
• mica: 4.23.0 -> 4.24.0 (4.23.0 -> 4.24.0)
  • PR 249 (jeanconn): More centroid_dashboard tweaks
  • PR 250 (jeanconn): Minor V&V fixes
  • PR 251 (taldcroft): Add list of available content types to asp_l1.get_files
  • PR 253 (jeanconn): Increase an ASOL quat vs OBC quat test tolerance by 4 arcsecs in roll
  • PR 252 (jeanconn): L0 ingest issues (both shiny and repro V)
  • PR 254 (taldcroft): A few small V&V and report updates
  • PR 241 (taldcroft): Use vectorized Quat
• parse_cm: 3.5.2 -> 3.6.0 (3.5.2 -> 3.6.0)
  • PR 29 (taldcroft): Add docs and modernize code
  • PR 26 (taldcroft): fixed bug skipping the last line of an OBS block
  • PR 28 (taldcroft): Support full parsing of obs request per OP19
  • PR 30 (taldcroft): Add cmds property to return kadi.commands.CommandTable version
• proseco: 4.10.0 -> 4.11.0 (4.10.0 -> 4.11.0)
  • PR 349 (taldcroft): Built-in function for diffing ACA catalogs
• psmc_check: 3.0.0 -> 3.3.0 (3.0.0 -> 3.1.0 -> 3.2.0 -> 3.3.0)
  • PR 23 (jzuhone): Refactor tests to match new signature
  • PR 24 (jzuhone): Updating answers for shiny
  • PR 25 (jzuhone): Update regression testing
• ska.engarchive: 4.50.1 -> 4.51.0 (4.50.1 -> 4.51.0)
  • PR 209 (taldcroft): Better caching in fetch
  • PR 210 (taldcroft): Update docs shiny
  • PR 211 (taldcroft): Move sync archive update into normal task schedule
• ska_helpers: 0.3.0 -> 0.4.0 (0.3.0 -> 0.4.0)
  • PR 17 (taldcroft): Add retry decorator and function wrapper
  • PR 18 (taldcroft): Add retry and tests sub-packages
  • PR 20 (taldcroft): Add lru_cache_timed decorator
  • PR 19 (javierggt): add detail about all exceptions in ska_helpers.retry
• skare3_tools: 1.0.1 -> 1.0.2 (1.0.1 -> 1.0.2)
  • PR 73 (javierggt): deal with default branch name different than 'master'
  • PR 74 (javierggt): Improve info provided by ska3_update_summary
  • PR 76 (javierggt): skare3_tools.packages fixes for when ska3-matlab or ska3-flight are not in the conda channel
  • PR 75 (javierggt): added skare3-promote script
• testr: 4.5.0 -> 4.6.0 (4.5.0 -> 4.6.0)
  • PR 34 (taldcroft): Add functionality to skip tests via skip.yml in package dir

Related Issues

Fixes #546
Fixes #547
Fixes #548
Fixes #549
Fixes #550
Fixes #551
Fixes #552
Fixes #553
Fixes #556
Fixes #557
Fixes #558
Fixes #559
Fixes #560
Fixes #561
Fixes #562
Fixes #563
Fixes #564
Fixes #565
Fixes #566
Fixes #567
Fixes #569
Fixes #570
Fixes #571
Fixes #572
Fixes #574
Fixes #575
Fixes #576
Fixes #577
Fixes #578
Fixes #579
Fixes #580
Fixes #581
Fixes #582