Skip to content

A streaming SQL engine, a fast and lightweight alternative to ksqlDB and Apache Flink, πŸš€ powered by ClickHouse.

License

Notifications You must be signed in to change notification settings

bestpala/proton-haotian

Β 
Β 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Timeplus Proton – An open-source, fast and lightweight streaming SQL engine
A fast and lightweight streaming SQL engine, πŸš€ powered by ClickHouse

πŸ“„ DocumentationΒ Β  πŸš€ Live DemoΒ Β  🌎 Timeplus

Β  Β  ReleaseΒ  YouTubeΒ  SlackΒ  follow on LinkedInΒ  XΒ  LicenseΒ 

Why Use Timeplus Proton Β· Demo Video Β· Deployment Β· What's Next Β· Integrations Β· Contributing Β· Need help?

Timeplus Proton is a streaming SQL engine, a fast and lightweight alternative to ksqlDB or Apache Flink, πŸš€ powered by ClickHouse. It enables developers to solve streaming data processing, routing and analytics challenges from Apache Kafka, Redpanda and more sources, and send aggregated data to the downstream systems. Timeplus Proton is the core engine of Timeplus Enterprise.

πŸ’ͺ Why use Timeplus Proton?

  1. Apache Flink or ksqlDB alternative. Timeplus Proton provides powerful streaming SQL functionalities, such as streaming ETL, tumble/hop/session windows, watermarks, materialized views, CDC and data revision processing, etc.
  2. Fast. Timeplus Proton is written in C++, with optimized performance through SIMD. For example, on an Apple MacBookPro with M2 Max, Timeplus Proton can deliver 90 million EPS, 4 millisecond end-to-end latency, and high cardinality aggregation with 1 million unique keys.
  3. Lightweight. Timeplus Proton is a single binary (<500MB). No JVM or any other dependencies. You can also run it with Docker, or on an AWS t2.nano instance (1 vCPU and 0.5 GiB memory).
  4. Powered by the fast, resource efficient and mature ClickHouse. Timeplus Proton extends the historical data, storage, and computing functionality of ClickHouse with stream processing. Thousands of SQL functions are available in Timeplus Proton. Billions of rows are queried in milliseconds.
  5. Best streaming SQL engine for Kafka or Redpanda. Query the live data in Kafka or other compatible streaming data platforms, with external streams.

Proton Architecture See our architecture doc for technical details and our FAQ for more information.

🎬 Demo Video

2-minute short videoπŸ‘‡. Check out the full video at YouTube.

DataTalksProtonHandbrake.mp4

⚑ Deployment

A single binary:

curl https://install.timeplus.com/oss | sh

Once the proton binary is available, you can run Timeplus Proton in different modes:

  • Local Mode. You run proton local to start it for fast processing on local and remote files using SQL without having to install a full server
  • Config-less Mode. You run proton server to start the server and put the config/logs/data in the current folder proton-data. Then use proton client in the other terminal to start the SQL client.
  • Server Mode. You run sudo proton install to install the server in predefined path and a default configuration file. Then you can run sudo proton server -C /etc/proton-server/config.yaml to start the server and use proton client in the other terminal to start the SQL client.

For Mac users, you can also use Homebrew to manage the install/upgrade/uninstall:

brew install timeplus-io/timeplus/proton

Docker:

docker run -d --pull always -p 8123:8123 -p 8463:8463 --name proton d.timeplus.com/timeplus-io/proton:latest

Please check Server Ports to determine which ports to expose, so that other tools can connect to Timeplus, such as DBeaver.

Docker Compose:

The Docker Compose stack demonstrates how to read/write data in Kafka/Redpanda with external streams.

Timeplus Cloud:

Don't want to setup by yourself? Try Timeplus in Cloud

πŸ”Ž Usage

SQL is the main interface. You can start a new terminal window with proton client to start the SQL shell.

Note

You can also integrate Timeplus Proton with Python/Java/Go SDK, REST API, or BI plugins. Please check Integrations

In the proton client, you can write SQL to create External Stream for Kafka or External Table for ClickHouse.

You can also run the following SQL to create a stream of random data:

-- Create a stream with random data
CREATE RANDOM STREAM devices(
  device string default 'device'||to_string(rand()%4),
  temperature float default rand()%1000/10);

-- Run the streaming SQL
SELECT device, count(*), min(temperature), max(temperature)
FROM devices GROUP BY device;

You should see data like the following:

β”Œβ”€device──┬─count()─┬─min(temperature)─┬─max(temperature)─┐
β”‚ device0 β”‚    2256 β”‚                0 β”‚             99.6 β”‚
β”‚ device1 β”‚    2260 β”‚              0.1 β”‚             99.7 β”‚
β”‚ device3 β”‚    2259 β”‚              0.3 β”‚             99.9 β”‚
β”‚ device2 β”‚    2225 β”‚              0.2 β”‚             99.8 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

⏩ What's next?

To see more examples of using Timeplus Proton, check out the examples folder.

To access more features, such as sources, sinks, dashboards, alerts, and data lineage, try Timeplus Enterprise locally or create a workspace on Timeplus Cloud.

What features are available with Timeplus Proton versus Timeplus Enterprise?

Timeplus Proton Timeplus Enterprise
Deployment
  • Single-node Docker image
  • Single binary on Mac/Linux
  • Single node, or
  • Cluster
  • Kubernetes-based self-hosting, or
  • Fully-managed cloud service
Data sources
  • Random streams
  • External streams to Apache Kafka, Confluent Cloud, Redpanda
  • Streaming ingestion via REST API (compact mode only)
  • Everything in Timeplus Proton
  • External streams to another Timeplus Proton or Timeplus Enterprise deployment
  • WebSocket and HTTP Stream
  • NATS
  • CSV upload
  • Streaming ingestion via REST API (with API key and flexible modes)
  • Hundreds of connectors from Redpanda Connect
Data destinations (sinks)
  • External streams to Apache Kafka, Confluent Cloud, Redpanda
  • Everything in Timeplus Proton
  • External streams to another Timeplus Proton or Timeplus Enterprise deployment
  • Slack
  • Webhook
  • Hundreds of connectors from Redpanda Connect
Support
  • Community support from GitHub and Slack
  • Enterprise support via email, Slack, and Zoom, with a SLA

🧩 Integrations

The following drivers are available:

Integrations with other systems:

Documentation

We publish full documentation for Timeplus Proton at docs.timeplus.com alongside documentation for Timeplus Enterprise.

We also have a FAQ for detailing how we chose Apache License 2.0, how Timeplus Proton is related to ClickHouse, and more.

Contributing

We welcome your contributions! If you are looking for issues to work on, try looking at the issue list.

Please see the wiki for more details, and BUILD.md to compile Timeplus Proton in different platforms.

Need help?

Please use GitHub Discussions to share your feedbacks or questions for Timeplus Proton.

For filing bugs, suggesting improvements, or requesting new features, open GitHub Issues.

To connect with Timeplus engineers or inquire about Timeplus Enterprise, join our Timeplus Community Slack.

Licensing

Proton uses Apache License 2.0. See details in the LICENSE.

About

A streaming SQL engine, a fast and lightweight alternative to ksqlDB and Apache Flink, πŸš€ powered by ClickHouse.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 77.6%
  • Python 9.1%
  • Assembly 5.4%
  • Shell 4.4%
  • C 2.4%
  • CMake 0.6%
  • Other 0.5%