Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple sinks at once (tee logging) #1998

Open
teh-cmc opened this issue Apr 28, 2023 · 4 comments
Open

Support multiple sinks at once (tee logging) #1998

teh-cmc opened this issue Apr 28, 2023 · 4 comments
Labels
🐍 Python API Python logging API 🦀 Rust API Rust logging API

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Apr 28, 2023

Something along the lines of this pseudo code:

struct LogSinkBundle(Vec<Box<dyn LogSink>>)

impl LogSink for LogSinkBundle { ... }

also need to update CLI args helpers for both Rust and Python to support mixing --save with anything else.

@teh-cmc teh-cmc added 🐍 Python API Python logging API 🦀 Rust API Rust logging API labels Apr 28, 2023
@emilk emilk changed the title Support multiple sinks at once Support multiple sinks at once (tee logging) Jan 2, 2024
@emilk
Copy link
Member

emilk commented Jan 11, 2024

It is very useful to be able to dynamically add and remove sinks at runtime.

For instance:

  • you start rerun with no sinks
  • something happens, so you now start dumping to an .rrd on disk
  • you hit a rare corner case happens, so you call spawn and start streaming to the viewer, then stop that stream once you reached the end of the corner case

This mean we need an API along the following lines:

rerun = rr.new_recording(app_id=…, …)
rerun.add_sink(file_path="foo.rrd")
viewer_sink_id = rerun.add_sink(spawn=True)
…
rerun.remove_sink(viewer_sink_id)

When no sinks are attached, rerun.log(…) should be a no-op

@emilk
Copy link
Member

emilk commented Jan 16, 2024

An interesting case to consider is streaming to an .rrd and calling spawn. We could set up the native viewer to be able to stream-load an .rrd file that is being written to, requiring no extra WebSocket connection. That would also mean the rerun library without the WebSocket feature (and huge dependency tree) could still support spawn (see #4788)

@vmayoral
Copy link
Contributor

Adding to the thread another potential use case of interest: #6685

In a nutshell: Periodic calls to rr.save() to generate smaller/manageable log files. This is particular relevant in robotic systems generating tons of data from depth sensors and multiple cameras.

@vmayoral
Copy link
Contributor

Hello all, any updates on this topic? Rationale at #1998 (comment) makes a lot of sense to me as well. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐍 Python API Python logging API 🦀 Rust API Rust logging API
Projects
None yet
Development

No branches or pull requests

4 participants