Skip to content

Latest commit

 

History

History
170 lines (138 loc) · 5.77 KB

3_0_2.rst

File metadata and controls

170 lines (138 loc) · 5.77 KB

3.0.2

v3.0.2 - May 6, 2022

This is a minor release with minor bug fixes for v3.0.0

Changes

  • Change error to warning for Graphite timeseries is unavailable (4164: luminosity - cloudbursts)
  • Wrapped opentelemetry instrument functions in try. Fail gracefully if breaks it breaks (4514: Integrate opentelemetry)
  • Handle metrics with url encoded names try fetch with %25 if % fails (4374: webapp - handle url encoded chars)
  • If there is only 1 data point in the timeseries determine_data_frequency will not return a resolution so set a sensible default as it does not matter, it will be boring (4518: settings - LAST_KNOWN_VALUE_NAMESPACES)
  • Wrapped aligned_timeseries.append in try (4518)
  • Remove all nan from downsampled data because they break histogram_bins (3866: MIRAGE_ENABLE_HIGH_RESOLUTION_ANALYSIS)
  • If the downsampled data is not anomalous, remove the entry from (3866)
  • Prune entries older than 1 day from the trigger_history (3866)
  • Change to correct setex method so the keys expire (4536: Handle Redis failure)
  • Declare use_old_timestamp_keys before the ignore_submitted_timestamps (4536)
  • Clean up tmp timings and count files (3486: analyzer_batch)
  • waterfall alerts - handle single value timeseries (3734: waterfall alerts)
  • Added functions and API methods and documentation (4530: namespace.analysed_events)
  • Change from using skyline_functions correlate_or_relate_with to (4544: Handle EXTERNAL_SETTINGS in correlate_or_relate_with)
  • Deprecated correlate_or_relate_with in skyline_functions (4544, 3858: skyline_functions - correlate_or_relate_with)
  • Added as functions.metrics.correlate_or_relate_with with the ability to handle external settings (4544)
  • Added illuminance Redis hashes to track all anomalous events for future R&D (2580: illuminance)
  • Correct misspelt missing function in v3.0.0 (4530: namespace.analysed_events)
  • Handle cluster data in api metric endpoint (4530)
  • Use str for fp_id in cluster data (3890: metrics_manager - sync_cluster_files)

Upgrade notes

  • These update instructions apply to upgrading from v3.0.0 to v3.0.2 ONLY
  • There are no changes to the DB in v3.0.2
  • There are no changes to settings.py in v3.0.2

How to update from v3.0.0

  • Download the new release tag or clone/update to get it to a temp location, ready to be deployed.
  • Copy your settings.py from v3.0.0 and replace the v3.0.2 settings.py
  • Stop the Skyline apps, backup the database and Redis, move to the new version, start the Skyline apps.
# Get the new version
NEW_SKYLINE_VERSION="v3.0.2"    # Your new Skyline version
OLD_SKYLINE_VERSION="v3.0.0"    # Your old Skyline version

CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline"                 # Your Skyline path
NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"  # Your new Skyline path

mkdir -p "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
cd "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
git clone https://github.com/earthgecko/skyline .
git checkout "$NEW_SKYLINE_VERSION"

# settings.py
cp "$NEW_SKYLINE_PATH/skyline/settings.py" "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak"

# Create a new settings.py file in the new version based on your existing
# settings.py file
cat "${CURRENT_SKYLINE_PATH}/skyline/settings.py" > "$NEW_SKYLINE_PATH/skyline/settings.py"

# **TEST**
# Test your new settings.py BEFORE continuing with the upgrade,
# test_settings.sh runs the validate_settings.py that the Skyline apps run
# when they start
$NEW_SKYLINE_PATH/bin/test_settings.sh

# Stop/disable any/all service controls like monit, etc that are controlling
# Skyline services.

# NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
# or do not wish to run.
SKYLINE_SERVICES="analyzer
analyzer_batch
mirage
crucible
boundary
ionosphere
luminosity
panorama
webapp
vista
snab
flux
horizon"
for i in $SKYLINE_SERVICES
do
  systemctl stop "$i"
done
  • Move your current Skyline directory to a backup directory and move the new Skyline v3.0.2 with your new settings.py from the temp location to your working Skyline directory, (change your paths as appropriate) e.g.
mv "$CURRENT_SKYLINE_PATH" "${CURRENT_SKYLINE_PATH}.${OLD_SKYLINE_VERSION}"
mv "$NEW_SKYLINE_PATH" "$CURRENT_SKYLINE_PATH"

# Set permission on the dump dir
chown skyline:skyline "$CURRENT_SKYLINE_PATH"/skyline/webapp/static/dump
  • Start the all Skyline services (change as appropriate for your set up) e.g.
# NOTE ALL SKYLINE SERVICES ARE LISTED HERE, REMOVE TO ONES YOU DO NOT RUN
# apart from the new thunder Skyline app

# Start all other Skyline services
SKYLINE_SERVICES="horizon
flux
panorama
webapp
vista
analyzer
analyzer_batch
mirage
crucible
boundary
ionosphere
luminosity
snab
thunder"
for i in $SKYLINE_SERVICES
do
  systemctl start "$i"
  if [ $? -ne 0 ]; then
    echo "failed to start $i"
  else
    echo "started $i"
  fi
done
# Restart any/all service controls like monit, etc that are controlling
# Skyline services.
  • Check the logs
# How are they running
tail -n 20 /var/log/skyline/*.log

# Any errors - each app
find /var/log/skyline -type f -name "*.log" | while read skyline_logfile
do
  echo "#####
# Checking for errors in $skyline_logfile"
  cat "$skyline_logfile" | grep -B2 -A10 -i "error ::\|traceback" | tail -n 60
  echo ""
  echo ""
done

Congratulations, you are now running the best open source anomaly detection stack in the world (probably).