v1.8.0

Raspberry-NOAA-V2 (‘RN2’) - Release 1.8.0

Supported on Raspian BUSTER.

‘Defaulting to gnuradio’

A number of users have expressed an interested to support a wider variety of SDR hardware devices beyond those interfaced to the original default rtl-sdr cards through the rtl-fm based commands used in the associated drivers.

In the original incarnation of RN2 the NOAA captures were driven directly through the audio_processor scripts to use rtl_fm directly, and the METEOR-M2 captures were driven using a python script that was originally generated in gnuradio and used the OSMOCOM source block, which in turn uses the rtl_fm binary to interface to the SDR card.

Since gnuradio can allow interchange of such ‘source blocks’ this methodology potentially opens up RN2 to support wider SDR hardware options.

This release DOES NOT seek to provide immediate access to the wider hardware options, but it DOES allow the community to begin to interface their own hardware by modifying the gnuradio composer files (.grc files included in the /audio_processors/ folder, to export python (.py) scripts similar to those included for the SDR, to copy the modifications that must then be made manually to the .py to interface the python workflow to the RN2 workflow (documented in the examples provided) and to then use alternative hardware to capture the source .wav or .s / .qpsk files that then can be used in the /image_processors/ scripts.

On day one most users defaulting to ‘gnuradio’ in their settings.yml will see little difference to RN2, but over time the community can now look to a common method to introduce new hardware support to the project.

More details on the deeper code is included in the new README included in the audio_processors folder, which includes sample .grc files and annotation in the current 'default' .py files showing where edits must be manually made AFTER exporting the python from gnuradio composer to make the scripts work with the RN2 settings.

Cosmetics

The spectrogram has been moved from SOX generation to FFMPEG generation, and also is represented vertically instead of horizontally. In the next iteration it may be suitable to flip it along with the NORTHBOUND passes so that as displayed in webpanel the timeline will alway flow in the same direction as the pass images shown (interference bars will then correlate in parallel between the two). It also means that colour-schema and other elements in the spectrograms can be more customised by those who want to do so.

Enhancements

We have enabled as many enhancements as possible in the NOAA workflows, and the webpanel now includes a hyperlink to a ‘details’ page that contains much of the documentation from the wxtoimg command line manual that relates to each enhancement type. The list shows only those that there are scripts in the /image_processors/ folder - there are a couple purposefully left out, but would be easy to introduce for those keen to do so.

Kudos

• @colinluthier for his continued support, critical eye and key work on various elements of the ‘plumbing’
• @Cadair for git fu, many general code contributions and for sanity checks on a nearly daily basis!

@everyone for testing and being patient!

What's Changed

• Add Missing Config to Schema and ffmpeg by @jekhokie in #337
• Add Advanced Scheduling by @jekhokie in #340
• script link incorrect by @b3rtdb in #376
• Auto Gain fix by @cdtush in #386
• Empty image bug patch by @colinluthier in #363
• Update push scripts so output is logged to the file by @Cadair in #387
• METEOR on TMPFS depending on free memory by @Clnmt in #359
• Prune older than patch by @colinluthier in #373
• Add a push provider to Matrix by @Cadair in #362
• put back missing space by @Cadair in #391
• French i18n by @Clnmt in #389
• Add Slack push feature by @MavEtJu in #375
• Fix config validation by @Cadair in #392
• FLIP capture only if PASS_DIRECTION is Northbound by @Clnmt in #398
• Disable sending of mail after "at" job execution by @Clnmt in #403
• Portuguese (PT) version by @airlomba in #426
• Support Chinese by @showdoi in #443
• Fix #400 by using which to find path for meteor_demod. by @ajcollins in #450
• Update receive_meteor.sh to use available memory rather than free memory by @ajcollins in #451
• Update logged stats in receive_meteor.sh to include available memory. by @ajcollins in #452
• Add libsox-fmt-mp3 to dependencies by @Miskerest in #453
• Histogram upgrade by @colinluthier in #361
• enabling gnuradio recording option for M2 and NOAA by @dom-robinson in #437

New Contributors

• @b3rtdb made their first contribution in #376
• @cdtush made their first contribution in #386
• @Cadair made their first contribution in #387
• @Clnmt made their first contribution in #359
• @MavEtJu made their first contribution in #375
• @airlomba made their first contribution in #426
• @showdoi made their first contribution in #443
• @ajcollins made their first contribution in #450
• @Miskerest made their first contribution in #453

Full Changelog: v1.7.0...v1.8.0

v1.7.0

Enhancements/Features:

• Update default thumbnail in webpanel to show MSA for daylight passes #301
• Add ability to output SDR settings from support script for better troubleshooting #296
• Add database table schema output to support script for better troubleshooting #304
• Add ability to overlay (optional) thermal image thermal gauge #306
• Add ability to delete passes from webpanel Admin interface (easier than atrm) #327
• Add ability to generate automated NOAA slide show in webpanel next to Satvis visualization #333

Bug Fixes:

• Pass missing frequency offset to record scripts #299
• Duplicate scheduled pass when schedule script run by hand/manually #269
• Ensure TLE files are downloaded as part of install if they are missing #318
• Remove future captures from database upon scheduling to avoid duplicates #317
• Ensure disabling schedule of a satellite reflects in database #291
• Fix t2sec.sh script to make it executable (was missing +x bit) #325

v1.6.0

NOTES:

• Several configuration parameters have been removed from the settings file in favor of modified variable names. Specifically, the following settings are no longer applicable and have no effect - please remove these from your config/settings.yml file and see new settings in config/settings.yml.sample that replace these:
  • schedule_noaa
  • schedule_meteor
  • receiver_gain
  • sun_min_elevation
  • sat_min_elevation
  • receiver_freq_offset
  • enable_bias_tee
  • sdr_device_id

Enhancements/Features:

• Add ability to choose specific satellites to schedule/capture #241
• Add ability to configure separate settings for each satellite pass to be captured #219
• Add support script that can be run to collect information that is useful for troubleshooting #244
• Add date output to test scanner script for easier use #248
• Enable direct to Twitter push from framework #254
• Increase output of support script for better support details #253
• Add ability to configure a custom date/time format in the webpanel #240
• Add ability to generate an azimuth/elevation polar graph for passes #261
• Add ability to generate a directional polar graph showing pass direction, side (E/W), and elevation #266
• Add ability to generate a histogram out of the NOAA Pristine image #281
• Add ability to output all scheduled jobs (captures) and their respective commands to the support script output #284
• Automatically create scanner test output in temp directory vs. relative paths, keeping things organized #289
• Add ability to "extend" image for annotation, avoiding annotation overlay on image capture content #264

Bug Fixes:

• Ensure existing stop script is terminated on test scanner completion #249
• Add a couple of missing parameters to the settings schema validation #278
• Fix bug where scheduler does not complete on a Pi reboot due to internet connectivity race condition #286

v1.5.0

NOTES:

• This release includes the ability to TLS-enable the webpanel. This should by NO MEANS imply that it is safe to expose the webpanel to the public internet as this framework and its components have not been security tested/vetted for reasonable security safeguards. Exposing the webpanel to the public internet is done at your own risk and with no assumptions of reasonable security.

Enhancements/Features:

• Add Pass Side (East/West) to Email and Discord push annotations #188
• Add enhanced (schema-based) configuration parameter checking functionality #182
• Add the ability to TLS-enable the webpanel using self-signed certs #137
• Add the ability to enable basic auth in the webpanel for the Admin functionality #46
• Add 'gain' as possible value in image annotation #203
• Add actual icon to favicon.ico for webpanel (meaningful browser tab icon) #205
• Add time/duration to log output to detail total time taken for capture in hours/minutes/seconds #190
• Add scheduling handler in case the Pi is rebooted (to restore all future 'at' jobs on reboot) #213
• Increase annotation font size in testing scanner output #204
• Add gain to annotation output of test scanner histogram #204

Bug Fixes:

• Fix bug preventing ability to disable ground station crosshairs #184
• Fix bug preventing ability to disable latitude/longitude lines in map #186
• Fix bug preventing auto-gain from being set #181
• Fix scanner scripts (testing) relative path issue #195
• Update configuration settings sample file to include a couple of missing parameters
• Fix bug preventing ability to disable state and country borders #236

v1.4.0

NOTES:

• Several configuration parameters have been removed from the settings file in favor of using a template for annotation. Specifically, the following settings are no longer applicable and have no effect - please remove these from your config/settings.yml file and instead edit the template ${NOAA_HOME}/config/annotation/annotation.html.j2 after running the install script:
  • show_sun_elevation
  • show_pass_direction

Enhancements/Features:

• Add Dutch language translation to Webpanel #146 #147
• Add French language translation to Webpanel #148
• Add auto-gain option for capture receives #165
• Add ability to customize image annotations using HTML template #94
• Add gain value for captures to the captured images pages on the webpanel #169
• Add testing scripts to scan and produce spectrogram of frequency ranges #172
• Add details about passes being east or west, and enable being able to add it to capture image annotation #159

Bug Fixes:

• Set Ground Station when clicking on satellite hyperlink for Satvis (to show pass information) #157
• Do not upload NOAA images that aren't a reasonable size (likely bad/missing images) #164
• Fix Bulgarian translations for webpanel #175
• Correct the word "Gain" for the Spanish translations in the webpanel #178

v1.3.0

Enhancements/Features:

• Add ability to compare config settings (assistance in upgrade process) #117
• Enable ability to add ground station location to image annotation (optional) #114
• Enable ability to add sun elevation to image annotation (optional) #114
• Add ability to enable/disable country borders, and associated color of borders #114
• Add ability to enable/disable state borders, and associated color of borders #114
• Add ability to specify NTP server in addition to default pool (e.g. stratum1 instances) #120
• Add Bulgarian language translation for webpanel #124
• Move tools scripts to sub-directory for better organization #97
• Add satellite azimuth and direction to webpanel and (optionally) image annotation #97
• Add ability to email an image of the webpanel pass list when the scheduler runs #123
• Click on satellite name to "focus on" satellite in Satvis when Satvis is enabled in pass list #126
• Add ability to post images to Discord channel #134
• Add ability to configure which NOAA enhancements to create for day/night passes #136

Bug Fixes:

• Correct $FLIP in receive_meteor.sh to ensure rotation is user-driven (not hard-coded) #113
• Check for images prior to pushing any capture #140

Notes:

• As part of moving the tools scripts, ensure that any cron jobs you have configured which leverage scripts that are now located in the scripts/tools/ sub-directory are updated for the correct path.

v1.2.0

Enhancements/Features:

• Add ability to crop the telemetry from NOAA images #79
• Add ability to set location of image annotations (global setting) #80
• Add documentation for troubleshooting Bias-Tee #38
• Add feature for backing up SQLite3 database to restore in case of corruption #60
• Add ability to disable WiFi power management (prevent network connectivity issues) #95
• Enhance Meteor-M 2 processing using gnuradio and in-memory processing #103
• Add ability to create pristine images from NOAA captures #100
• Add ability to email images to email address (e.g. for IFTTT processing) #104
• Add default email configs and details on handling GMail MFA #106

Bug Fixes:

• Fix bug where language files missing "Admin" resulted in no webpanel Admin page #76

v1.1.1

Enhancements/Features:

• Enable choice of SDR device when there are multiple available #57
• Add contribution guidelines to README #63
• Add troubleshooting for white Meteor-M 2 images due to sun elevation #59
• Add ability for thermal image processing for NOAA and handle when no image is present on file system #62
• Add prune scripts and configurations for "delete older than n" and "delete oldest n number" #61

v1.1.0

Enhancements/Features:

• Log to output file to enable better troubleshooting #27
• Add pass overlap indicator in webpanel when passes overlap in timeframe #39
• Update documentation for compatible hardware #54

Bug Fixes:

• Default to single channel output for spectrogram images #40
• Fix Python dependencies to remove "Warning" message during install (Ansible) run #44

raspberry-noaa-v2 1.0.0 (Initial Release!)

Initial release of V2 of the original raspberry-noaa framework. Major changes to support ongoing development from the community, and signed off as "ok to fork to V2" by the original creator of the raspberry-noaa framework (Nico).

This initial release is primarily a major refactor in the way the framework is installed and supported ongoing, allowing for a single-script upgrade for users and enhancing the ability to add features, manage the database schema, and upgrade the webpanel. Users now have a single-run install script and a single configuration file driving the entire install and upgrade process, meaning the process simply includes a "git pull" operation followed by running the install_and_update.sh script to get any/all new functionality since their last update!

Note: This initial release removes the ISS capture functionality and Twitter post capability. This was a purposeful effort to reduce scope and help enhance focus on the core functionality of the V2 release, and it is anticipated these functionalities may be re-added in the future.

Notable changes in this initial release (since the fork from the original version to this V2 version) are below.

Enhancements/Features:

• Webpanel refactor (MVC) for enhancements and mobile-friendly views #2
• Major scripts refactor for better code re-use (keeping functionality DRY) and organization #4
• Major installation updates and enhancements (single-click installer with single config file) #6
• Base station enable/disable and color in images #9
• Lat/lon lines enable/disable and color in images #10
• Documentation overhaul for easier navigation/content #11
• Simplification of scheduling scripts #14
• Simplification/consolidation of settings #16
• Satvis visualization added to webpanel "Passes" page #19
• Ability to delete captures (database record and respective images) from webpanel #24
• Major receive script refactor for pluggable audio and image processing #25
• Option to produce frequency spectrogram image #26

Bug Fixes:

• Fix NTP installation #37