v0.22.0

Release date: 2023-05-09

Add JSON unmarshalling to SQL receiver, add unflatten to metadata
transformer as well, and allow banning with ban transformer by path.

v0.21.1

Release date: 2023-02-27

Add SIGHUP-handling to file-sender.

Add JSON-casting to cast transformer.

Upgrade to Go 1.20, and bump various libs past security lol.

v0.21.0

Release date: 2023-02-10

Add NATS sender and receiver (thanks Niklas Holmstedt!), add support for
"match any"/presence in switch-transformer and fix a minor issue where
blank string could never match.

v0.20.11

Release date: 2023-01-05

Releases v0.20.0 to v0.20.10 are all build-system/CI/CD-related tweaks and
basically irrelevant to end users.

The following is the release notes for v0.20.0:

This is a small, but important release that adds support for loadable
plugins, and also the LineFileAdvanced file reader that allows log
rotation-based file reading (e.g.: Move a file, send sigup, then parse it).

The plugin support is very much experimental, see docs/plugins.rst for
discussion. There's also an example-plugin at
https://github.com/telenornms/skogul-plugin-example which demonstrates how
to build an out-of-tree plugin.

Skogul v0.19.1

Release date: 2022-11-15

This is an "emergency release" of sorts, that REMOVES SQLITE SUPPORT.

We need to nuke this because it introduces what turned out to be unacceptable dependencies which can't be (easily) met on, e.g., CentOS 7.

This means Skogul 0.19.0 RPMS wont work on CentOS/RHEL 7, while Skogul 0.19.1 should.

Sorry for the inconvenience. We will re-visit SQLite when a more appropriate library can be used.

Skogul v0.19.0

Release date: 2022-11-14

Somewhat large release, but:

• Drone is on the fringe, so manual release. GH actions incoming.
• New SQLite sender
• New Experimental prometheus parser
• New unflatten transformer
• Cast transformer: Add casting of IPs to integers

Minor:

• Several minor chores from Håkon Solbjørg that increases consistency and correctness, with minor negative user-impact.
• Splunk: Send metadata as fields to spluk HEC
• Deperecate "Path" and use "File" instead for consistency (deprecation, old configs will keep working)
• Add KeepOriginal flag to the Data transformer
• Many log cleanups

All in all, this is a larger release than we'd like, but also largely isolated features.

Since this will be a manual release, it might deviate SLIGHTLY from the other releases, but should be mostly the same.

Skogul v0.18.0

Release date: 2022-09-20

Adds USP protocol buffer parser and AVRO encoder/parser time support, adds
worksarounds for Juniper -Inf transceivers, adds dummy parser.

USP

The USP parser will parse a protocol buffer-encoded USP message contained
in a record, and use the payload as JSON. This might change if hardware
surfaces that use USP differently.

AVRO

The AVRO encoder/parser now handles time fields, using a minor workaround.
Shouldn't require any changes for users.

Juniper

We now apply various hacks to fix #218, where juniper telemetry reports
-Infinity, which is impossible to encode as JSON. Intead, we silently
convert it to -40, horrifying any mathematicians in the room.

Dummy parser

The dummy parser is meant as a developer-tool where we can save un-parsed
bytes to disk, thereby allowing us to acquire test data for development.
Not useful in any production environment.

Skogul v0.17.0

Release date: 2022-08-24

In short:

• AVRO and GOB encoding and parsing
• SQL receiver
• Encoder-support in file sender
• Docker images!

Docker images - first release, so let's see how well this works :D

SQL receiver will periodically poll an SQL database for data, it is a bit
immature, but it should be solid. You write a query, the query needs a
field named 'time' (select foo as TIME ...) for the timestamp, and you also
select which columns are metadata - the rest will be data.

While GOB could theoretically improve performance, it's not possible in
practice because GOB isn't schemaless, but embeds a schema in the initial
packet. This means that GOB data depends on the receiver having the initial
packet, and there is no convenient way to achieve that except
re-initializing the encoder every time. The end result is that gob is no
better or worse then JSON in regards to performance, but, unlike JSON, it
can support every data type that Go supports, fully, without losing
accuracy.

The AVRO format is still missing support for timestamps, see #247, but
that's being worked on.

To use AVRO, you need to specify a schema. One is provided in
docs/examples/avro, which is a very generic schema. You can write a more
specific one as well, for your specific data, but so far there is no
support for time stamps. To be able to actually USE Avro today, you need to
add a transformer on the handler to add a timestamp.

Skogul v0.16.0

Release date: 2022-05-28

TL;DR:

• Kafka sender and receiver
• Default configuration file moved to /etc/skogul/conf.d in systemd units
• Enricher transformer MUST be used together with enricher updater sender
• "Wholefile" receiver will read a file and treat it as a single container
• Much needed docs/examples cleanup
• Dummy timestamp transformer

If you are using an unaltered systemd unit file and our RPM packages, you
will have to move your configuration from /etc/skogul/default.json into
/etc/skogul/conf.d/ - the file name can be whatever, as long as it ends in
.json - Skogul will load every configuration file present.

This better allows dynamic configuration through, e.g., Ansible and is
what we've been using for a while.

The Kafka sender has one little quirk: Since Kafka allows batching messages
natively, it was decided that it is better for overall performance to
encode each metric in a container as an individual message, instead of
encoding the entire container as one message. This will better exploit the
nature of Kafka and make it easier to use the data, hopefully.

The enricher also went through a major simplification: It no longer has a
native loader function, and instead you have to configure an enricher
updater sender. The configuration becomes slightly more cumbersome, but the
end result is a much more elegant solution.

Skogul v0.15.2

Release date: 2022-05-09

Adds plumbing for modular encoding, but more importantly, an enrichment
updater sender.

The sender "enrichmentupdater" and the corresponding changes to the
enrichment transformer will allow on-line updates to the enrichment
database using regular Skogul pipelines. You can load a JSON file at
start-up with an initial database of enrichment, then update that database
on the fly over an HTTP interface for example, if you set up a
receiver-pipeline that ends in an enrichment updater.

Both the enrichment updater and the enrichment transformer is STILL
experimental, but I'm feeling good about them, and they will be a major
addition to medium to large-scale Skogul deployments.

PS: I botched the NEWS in v0.15.1 so let's go lucky v0.15.2.