This repository contains all the bits for setting up the demo of the "Open sourcing the IoT : running EnMasse on Kubernetes" session at "Industry 4.0 & AgileIoT" by Paolo Patierno. Other than the source code following :
- Slide deck
- A video about the demo
This demo shows how it's possible to use a Kubernetes or OpenShift cluster running EnMasse and Apache Spark in order to have a way for ingesting data through a scalable messaging infrastructure and then processing such data using Spark Streaming jobs or Kafka Streams API. The proposed IoT scenario is made of :
- an AMQP publisher which sends simulated temperature values to a temperature address (backed by a queue or a Kafka topic)
- a Spark Streaming driver application which reads above values in order to process the maximum value in the latest 5 seconds, sending the processed value to a max address (backed by a queue or a Kafka topic)
- an AMQP receiver which gets filtered maximum temperature values reading from the max address
The following pictures shows how all the above components define the overall demo deployment using Kubernetes (running inside or outsice ACS), EnMasse and Apache Spark.
The same solution can be deployed on OpenShift using the Apache Spark provided by the radanalytics.io project.
Finally, this picture shows the deployment using Apache Kafka for storing ingested data but always through the AMQP protocol (so using an AMQP - Kafka bridge) and then a Kafka Streams application for real time analytics.
The description about the two different solutions can be found here :