Tracking issue: short-term performance targets

[emilk]

The purpose of this issue is to list some performance targets and how close we are to reaching them.

This is the target we strive to reach on a modern laptop, e.g. a M1 MacBook Pro.

We currently lack benchmarks for most of these targets.
We know we are wildly off the mark on many of them.

Per-thread logging frequency

f32 scalar with one custom timeline.

Rust and C++: 1us per scalar = 1Mhz
Python: 100us per scalar = 10kHz

Status: We are off by ~3x at the moment.

Text logging should be similar.

Multi-threaded logging

Switching from one logging thread to N logging threads (N≤cores) should have at most 20% overhead, on any SDK.

Status: We know that we have problem with lock contention of Python

Video

• 4k video at 60 Hz RAW
• 4k video at 60 Hz with per-frame compression

The per-frame compression could e.g. be JPEG, JPEG2000, …

Status: Our SDK-side compressor is too slow, and it is also not multi-threaded

Latency

• No noticeable latency when streaming a web camera (<100ms)

Status: we're good

Data overhead

On-disk overhead and in-RAM overhead:

• 2x overhead: pain-limit
• 20% overhead: target

Overhead is measures as memory use divided by size of raw data (log data logged data + size of timeline values), ignoring caches.

Status: Right now our memory overhead for small datatypes (e.g. scalars) is enormous, because of arrow2 schema overhead.

Viewer performance

60 Hz when viewing:

• A static point cloud with custom colors, 1 million points large
• 60Hz 4k uncompressed video
• 60Hz 4k compressed video
• 10 time-scalar plots, with 10 lines per plot, each at 1kHz, viewing 1000 seconds of history

Links

• Report logging benchmarks for C++/Python/Rust to CI #4100
• All performance issues

[jleibs]

For both Viewer and SDK performance targets we need to be explicit about expected system load while achieving those targets. Our inclination to hit performance targets by moving work to additional threads very quickly puts us in a domain where we've left no CPU for the user to actually run their algorithms.

• See: CLI argument for setting max number of viewer threads #4931

For example in the use-case of 1Mhz scalar logging, there's a big difference between:

• 1 core @ 100% for everything
• 2 core @ 100% (1 logging + 1 batching)
• 4 cores @ 100% (1 logging + 1 batching + 1 ingesting + 1 viewing)