Skip to content

Alarm System Example

Jump to bottom
Ryan Slominski edited this page May 19, 2022 · 35 revisions

Extend JAWS with EPICS alarms.

See: jaws-epics2kafka

Message Formatting

The format of messages from epics2kafka and the JAWS are very different. To reconcile the differences Kafka provides several strategies:

  1. Connect converters - only converts between message packaging (serialization) such as AVRO, ProtocolBuffers, or JSON - does not modify fields
  2. Connect Single Message Transforms - can modify structure: fields, headers, and metadata, but cannot split stream into multiple topics or aggregate multiple messages and are not intended for heavy processing or external API calls.
  3. Kafka Streams
  4. Confluent ksqldb (built on Streams)
  5. Any other stream processing server (Apache confusingly has at least 8: Flink, Spark, Storm, Flume, Heron, Samza, Apex, and NiFi)

We use Connect converters in the configuration to ensure the messages are serialized with AVRO as JAWS requires. We use a custom Transform to convert epics2kafka messages into JAWS structure (and also provide hints to the AVRO converter about enums and unions as those are not directly supported by Kafka Connect Schema).

Channel Configuration

The alarms that JAWS is aware of are contained in the alarm-instances topic, and a subset of messages in the topic contain the field producer of type EPICSProducer indicating that the alarms are generated by epics2kafka. The Kafka Streams application registrations2epics reads the alarm-instances topic and populates the epics-channels command topic. This allows users of the alarm system to register alarms that are monitored by epics2kafka without having to also configure the epics2kafka command topic epics-channels manually.

Note: Confluent ksqldb might have been a good option for this task if it supported unions, but it does not, which the alarm-instances topic requires.

Clone this wiki locally