Keda External Scaler with ActiveMQ Artemis

This sample has been updated to use KEDA v2 and ActiveMQ Artemis 2.15.0

This is a demonstration on how to use KEDA's external scaler to monitor ActiveMQ Artemis Queue.

The Keda external scaler calls metrics-provider GRPC server which will collect the metrics from ActiveMQ Artemis.

ActiveMQ Artemis is now included as part of the KEDA internal scaler

Note: use your own docker repository while building the project.

Pre-requisites:

• Kubernetes cluster
• Keda installed in the cluster.
• skaffold - used to build the images
• maven - use to build the java consumer and producer project.

Code organization

• consumer - a simple Springboot application which consumes messages from ActiveMQ Artemis. The queue name used is test. The consumerWindowSize is also set to 0 so that it will not buffer the messages to one consumer.
• producer - a simple Springboot application which pumps messages to the queue test
• metrics-provider - a Go application which serves as the GRPC server to serve the Keda External Scaler. It queries ActiveMQ Artemis MessageCount using the built -in jolokia endpoint exposed by Artemis.
• k8s-manifest - everything Kubernetes

Building the metrics provider for external scaler

You can get the proto file from the Keda github.

This project already contains the externalscaler.proto

Use protoc to autogenerate the Proto codes.

protoc -I externalscaler/ externalscaler/proto/externalscaler.proto --go_out=plugins=grpc:externalscaler

Note: We use kaniko in-cluster builder

Setup kaniko registry access secret

kubectl -n artemis create secret generic regcred --from-file $HOME/.docker/config.json

$ skaffold run -p metrics-provider

External Scaler as ActiveMQ Artemis sidecar

The docker image used taken from vromero/activemq-artemis-docker uses the hostname as its broker name, in order to avoid hardcoding the broker name, the metrics provider is deployed as a sidecar to ActiveMQ Artemis.

Added to the file k8s-manifest/artemis/deployment.yaml

  containers:
  - name: artemis-activemq-metrics-provider
    image: docker.io/balchu/artemis-ext-scaler:1.0.0
    args: ["--port","5050","--broker","$(POD_NAME)", "--user", "$(ARTEMIS_USERNAME)","--password","${ARTEMIS_PASSWOORD)"]
    imagePullPolicy: Always
    resources:
      requests:
        cpu: 100m
        memory: 10Mi   

Build and deploy the consumer.

Using skaffold and jib, simply execute the command below.

skaffold run -p consumer

Please make sure that you use your docker repository.

Deploy the External Scaler manifest

Now its time to setup the KEDA's external scaler.

kubectl apply -f k8s-manifest/externalscaler_scaledobject.yaml

The file looks like this.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: artemis-scaledobject
  namespace: artemis
spec:
  pollingInterval: 5   # Optional. Default: 30 seconds
  cooldownPeriod: 5  # Optional. Default: 300 seconds
  minReplicaCount: 0   # Optional. Default: 0
  maxReplicaCount: 30  # Optional. Default: 100  
  scaleTargetRef:
    name: artemis-consumer
  triggers:
  - type: external
    metadata:
      scalerAddress: artemis-activemq.artemis:5050
      queueLength: "10"
      brokerAddress: "test"
      queueName: "test"

Where:

• scalerAddress: is the location of the GRPC metrics provider host and port.
• queueLength: the target queue length.
• brokerAddress: An address represents a messaging endpoint. Within the configuration, a typical address is given a unique name, 0 or more queues, and a routing type.
• queueName: the name of queue to monitor.

Before pumping in messages, check the HPA.

Check the HPA

kubectl -n artemis get hpa
NAME                        REFERENCE                     TARGETS              MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-artemis-consumer   Deployment/artemis-consumer   <unknown>/20 (avg)   1         30        0          58m

At this point,HPA doesn't have the TargetAverageValue to scale up or down the pods. This can be observed by the <unknown>/20(avg)

Start the producer

The producer is a simple Springboot application.

If you are going to run the Springboot application using your IDE, make sure that you point to the host and port of the ActiveMQ Artemis.

Check the file application.yml

As an example: