Python data types for the Apache Kafka® Protocol.
- Exposes immutable dataclass entities for all protocol messages, generated from the same source as used internally in Apache Kafka®.
- Message classes are simple light-weight data containers that do not inherit anything or expose any methods other than a regular dataclass. Encoding and decoding is enabled by making all the necessary details about Kafka encoding introspectable.
- Widely compatible encoding and decoding of messages through the
IO[bytes]interface. - Test suite with focus on roundtrip property tests using Hypothesis, including compatibility testing against the internals of upstream Apache Kafka®.
$ pip install --require-virtualenv kioInstall development requirements.
$ pip install --require-virtualenv -e .[all]The test suite contains integration tests that expects to be able to connect to an
Apache Kafka® instance running on 127.0.0.1:9092. There is a Docker Compose file in
container/compose.yml that you can use to conveniently start up an instance.
$ docker compose up -d kafkaRun tests.
$ python3 -X dev -m pytest --covSetup pre-commit to run on push.
$ pre-commit install -t pre-pushWarning
Building the schema will delete the src/kio/schema directory and recreate it again, hence
all of the files under this directory will be deleted. Make sure to not put unrelated files
there and accidentally wipe out your own work.
Fetch, generate, and format schema.
$ make build-schema