Skip to content

Latest commit

 

History

History
74 lines (56 loc) · 3.38 KB

README.md

File metadata and controls

74 lines (56 loc) · 3.38 KB
Raw

Mesosphere DC/OS Kafka Installer

Support DC/OS Version Support DC/OS Version

An Ansible based installer for Kafka & Confluent Platform on DC/OS

This is a DC/OS Community project and is not supported or warrantied by Mesosphere.

screenshot

Features

  • Deploys Confluent Platform Kafka with full security - Active Directory/Kerberos GSSAPI & SASL_SSL authentication or with TLS & SSL
  • Supports deployment of multiple Kafka clusters for multi-tenant environments
  • DC/OS group/folder support for organising clusters such as /dev/123456-kafka/
  • DC/OS strict mode security out of the box (only strict at this time)
  • Deployment of an Active Directory server on AWS for testing
  • Dynamic generation of:
    • a batch script to add Active Directory users, principals and generate keytabs
    • options.json configs for every service
    • client-jass.conf for every service
    • endpoint dependencies for each service
  • Automation of:
    • generating and adding binary and text secrets to the DC/OS secret store
    • configuring DC/OS security service accounts and ACLs
  • Menu system to wrap the Ansible playbooks using GNU make
  • Janitor cleanup with one command
  • End to end client testing scripts with documented steps
  • Easy access to Confluent Control Center
  • Archive of each cluster's configuration assets

Planned Features

  • CCM support
  • Endpoint management
  • Configurable resources for JSON options
  • Apache Kafka support
  • MIT Kerberos support
  • Standalone monitoring deployment integrated with dcos-metrics
  • Performance benchmarking

Limitations

  • Strict mode only
  • Cluster identifier limited to 9 chars due to Active Directory Kerberos naming limitations
  • Confluent Connect does not provide security on its REST API endpoint, this is a limitation in the Confluent product
  • External (to DC/OS ) service discovery is work in progress

Design

Ansible does the heavy lifting, talking over localhost, to generate configuration files based on templates and talking directly to the DC/OS CLI to manage the deployment.

Really, all this is doing is automating the manual process and myriad of configurations required.

Why use this?

  • You're running a multi-tenant / private cloud environment based on DC/OS where you want to deploy multiple Confluent Platform Kafka clusters
  • You want to test the full Confluent Platform stack on a DC/OS strict mode cluster with all security features enabled
  • You want a convenient testing framework to validate the components are working correctly
  • You don't want to run through >50 manual steps

Documentation