All notable changes to Wallaroo will be documented in this file.
- Add support for windowing (PR #2735)
- Gradually back off when attempting to reconnect on data channel
- There is no longer a problem when using more workers than there are partitions
- No longer treat state computation stages as a special case. This results in fewer allocations and better performance
- Python 3 Support for Connectors
- Add parallel stateless steps to joining workers
- Streamlined Wallaroo Python API
- Connectors API Update
- Simplify the Python API for adding a computation to a pipeline
- Added Python 3 support for Wallaroo in our Docker image and in installations via Vagrant, Wallaroo Up (on Ubuntu (Xenial, Artful, and Bionic) and Debian (Stretch and Buster)), as well as for installation from source on systems with Python 3.5 or higher (python3-dev is also required).
- Deprecate giles receiver in favor of data receiver (PR #2341)
- Python's argparse and other libraries which require properly intialized python arguments should no longer fail in certain cases in machida
- Added support for Fedora 26/27, Amazon Linux 2, Oracle Linux, Ubuntu Artful, and Debian Jessie/Buster Linux distributions via Wallaroo Up
- Checkpointing protocol for regular checkpointing of application state
- Support for rollback to most recent checkpoint on recovery
- Support for replicated recovery data
- Added preview release of the Python Connectors API
- Added connector scripts for Kafka, Kinesis, RabbitMQ, Redis, S3, and UDP
- Added a template for a Postgres connector script
- Added Wallaroo Up to automate development environment setup on multiple Linux distributions
- Added support for Fedora 28, CentOS 7, and Debian Stretch Linux distributions via Wallaroo Up
- Added Vagrant as an option for trying out the Wallaroo Go
- Added Docker as an option for trying out the Wallaroo Go
- Ensure that parallel stateless computations always run independently (PR #2322)
- Filter none/nil in Decoder for Python/Go API's (PR #2259)
- Filter
Nonevalues in machida compute_multi results (PR #2179) - Fix busy-loop/scheduler yield bugs in TCPConnection-related classes (PR #2142)
- Fix errors in Python decorator API documentation (PR #2124)
- Improve autoscale performance and reliability, and fix related edge case bugs (PR #2122)
- Fix bug that caused Wallaroo to shut down when connecting a new source after a 3->2 shrink (PR #2072)
- Correctly remove boundary references when worker leaves (PR #2073)
- Correctly update routers after shrink (PR #2018)
- Only try to shrink if supplied worker names are still in the cluster (PR #2034)
- Fix bug when using cluster_shrinker with non-initializer (PR #2011)
- Ensure that all running workers migrate to joiners (PR #2027)
- Clean up recovery files during shrink (PR #2012)
- Ensure that new sources don't try to connect to old workers (PR #2004)
- Fail when control channel can't listen (PR #1982)
- Only create output file for Giles sender when writing (PR #1964)
- Add support for dynamic keys (PR #2265)
- Inform joining worker to shut down on join error (PR #2086)
- Add partition count observability query (PR #2081)
- Add support for multiple sinks per pipeline (PR #2060)
- Allow joined worker to recover with original command line (PR #1933)
- Add support for query requesting information about partition step distribution across workers (PR #2025)
- Add tool to allow an operator to shrink a Wallaroo cluster (PR #2005)
- Clean up external message protocol (PR #2032)
- Remove "name()" from StateBuilder interface (PR #1988)
- Add Machida with Resilience to Wallaroo in Docker
- Improve Python exception handling in user provided functions (PR #2194)
- Add Artful Aardvark Support (PR #2189)
- Add Wallaroo in Vagrant (PR #2183)
- Add documentation for Wallaroo in Docker on Windows(PR #2177)
- Go API: Fixed bug in state computation that return multiple results
- Kafka Client: Update to pony-kafka release 0.3.4 for bugfix regarding partial messages
- Do not force shrink count to a minimum of 1 (PR #1931)
- Fix bug that caused worker joins to fail after the first successful round. (PR #1927)
- Add "Running Wallaroo" section to book (PR #1914)
- New version of Python API based on decorators (PR #1833)
- Fix shrink autoscale query reply (PR #1862)
- Initial Go API (PR #1866)
- Turn off building with AVX512f CPU extensions to work around a LLVM bug (PR #1932)
- Updates to documentation
- Updates to documentation
- Get ctrl-c to shutdown cluster after autoscale (PR #1760)
- Send all unacked messages when resuming normal sending at OutgoingBoundary (PR #1766)
- Fix bug in Python word count partitioning logic (PR #1723)
- Add support for chaining State Partition -> Stateless Partition (PR #1670)
- Fix Sender to properly dispose of files (PR #1673)
- Create ProxyRouters to all required steps during initialization