Skip to content

Commit

Permalink
Add document for subpartition runtime (#231)
Browse files Browse the repository at this point in the history
* Run benchmark for vthread runtime as well

* Add document for subpartition runtime
  • Loading branch information
kawamuray authored Mar 28, 2024
1 parent c8bab86 commit 479f043
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 0 deletions.
2 changes: 2 additions & 0 deletions cb/run-bm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,5 @@ $root_dir/cb/sysinfo.sh >$out_dir/sysinfo.json

run_with_opts "tasks_100k_latency_10ms_concurrency_20" --tasks 100000 --simulate-latency=10 --param=decaton.partition.concurrency=20
run_with_opts "tasks_1000k_latency_0ms_concurrency_20" --tasks 1000000 --simulate-latency=0 --param=decaton.partition.concurrency=20
run_with_opts "tasks_100k_latency_20ms_vthread" --tasks 100000 --simulate-latency=4 --latency-count=5 --param=decaton.subpartition.runtime=VIRTUAL_THREAD
run_with_opts "tasks_1000k_latency_0ms_vthread" --tasks 1000000 --simulate-latency=0 --param=decaton.subpartition.runtime=VIRTUAL_THREAD
1 change: 1 addition & 0 deletions docs/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Decaton Documents

- link:./why-decaton.adoc[Why Decaton]
- link:./getting-started.adoc[Getting Started]
- link:./runtime.adoc[Subpartition Runtime]
- link:./spring-integration.adoc[Spring Integration]
- link:./tracing.adoc[Tracing]
- link:./consuming-any-data.adoc[Use Decaton for consuming topics of non-Decaton tasks]
Expand Down
Binary file added docs/runtime-benchmark-conc-peak.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/runtime-benchmark-result.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
94 changes: 94 additions & 0 deletions docs/runtime.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
= Subpartition Runtime
:base_version: 8.0.0
:modules: processor

This document guides you what Subpartition Runtime is and how to use it.

Since `8.0.0` release, Decaton supports using different "Subpartition Runtime".
Subpartition is a core concept of Deacton to enable concurrent processing of one partition, and it has been implemented as the fixed count of queues with associated processing thread for each.
While it worked greatly to improve throughput of processing one partition, we also knew that sometimes the fixed number of threads left CPUs in idle, and limited throughput especially when the workload is heavily I/O intensive.
A typical workaround was to use `ProcessingContext#deferCompletion()` and use the returned `Completion` to asynchronously complete the task after returning from `DecatonProcessor#process()` method. It worked well in many production applications, but sometimes it caused a problem called "completion leak" leading the whole subscription to stuck and stop consuming the topic.

One possible solution for this problem is to use unlimited count of threads to process one partition, keeping serial processing constraints based on record keys, but creating such many platform (OS) threads is actually inefficient due to various costs like memory footprint and context switches, hence practically impossible. See <<Appendix>> for the performance characteristic with platform threads count.

Java 21 was released with https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html[Virtual Threads] support, making it possible to create individual thread for each record key - the unit of guaranteeing serial processing in Decaton.

As of now there are two runtimes supported.

* `THREAD_POOL` - process one partition with the platform threads of count `decaton.partition.concurrency`
* `VIRTUAL_THREAD` - process each different key on a different virtual thread. `decaton.partition.concurrency` configuration is meaningless with this runtime. **Requires JDK21 or higher**
== Usage

Just set `SubPartitionRuntime` by calling `SubscriptionBuilder#subPartitionRuntime()`. By default it is set to `SubPartitionRuntime.THREAD_POOL`.

[source,java]
----
SubscriptionBuilder.newBuilder("my-subscription")
.subPartitionRuntime(SubPartitionRuntime.VIRTUAL_THREAD)
...
.build();
----

IMPORTANT: When using the `VIRTUAL_THREAD`, be extremely careful for the use of `synchronized` primitive in your/library code that might be executed in `DecatonProcessor#process()` method, as it results "pinning" the virtual thread and could eliminate performance gain of using virtual thread completely.
Visit https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html#GUID-704A716D-0662-4BC7-8C7F-66EE74B1EDAD[Scheduling Virtual Threads and Pinned Virtual Threads] for the detail.

== Performance

As for the reference of performance characteristic of each runtime, here's the benchmarking result.

The machine spec used for the benchmark is as follows.

* CPU: Intel(R) Xeon(R) 2.20GHz, 8 core
* Memory: 16GiB

The benchmark was run using the https://github.com/line/decaton/tree/master/benchmark[benchmark module] with the following command line options.

[source]
----
./debm.sh \
--title "Decaton" \
--runner com.linecorp.decaton.benchmark.DecatonRunner \
--runs 3 \
--format json \
--tasks 10000 \
--warmup 100000 \
--simulate-latency=$latency \
--latency-count=5 \
--param=decaton.partition.concurrency=300 \
--param=decaton.subpartition.runtime={THREAD_POOL|VIRTUAL_THREAD}
----

image::runtime-benchmark-result.png[]

We can read from the result that the `VIRTUAL_THREAD` runtime provides much stable throughput around 23k tasks/sec regardless to the amount of duration taken for I/O in processing a task.


== Appendix

=== `decaton.partition.concurrency` tuning

As written previously in this document, having too much platform (OS) threads causes significant overhead and throughput growth is capped at some point.
To learn where is that point, a benchmark was run to determine at which `decaton.partition.concurrency` value do we hit throughput peak.

The following command is used with increasing concurrency value. The machine spec is as same as <<Performance>>.

[source]
----
./debm.sh \
--title "Decaton" \
--runner com.linecorp.decaton.benchmark.DecatonRunner \
--runs 2 \
--format json \
--tasks 10000 \
--warmup 100000 \
--simulate-latency=4 \
--latency-count=5 \
--param=decaton.partition.concurrency=$conc \
--param=decaton.subpartition.runtime=THREAD_POOL
----

image::runtime-benchmark-conc-peak.png[]

In this benchmark the processing logic simulates 5 continuous I/O with 4ms duration each. It is important to have multiple I/O during the benchmark because in real world applications there are more than one I/O performed in business logic, and that's where the cost of context switches come in.
From the result we can read that the maximum throughput could be achived when we have 300 threads for each partition, so in total, around 900 threads is maximum. (note that it could be very different on a different spec machine)

0 comments on commit 479f043

Please sign in to comment.