Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Binary logging support #630

Open
jsturtevant opened this issue Jun 24, 2024 · 4 comments
Open

Enable Binary logging support #630

jsturtevant opened this issue Jun 24, 2024 · 4 comments

Comments

@jsturtevant
Copy link
Contributor

Containerd has pluggable logging which includes binary logging: https://github.com/containerd/containerd/blob/c7f7fa5390f761b83c3a69cd5709a8d04d912969/core/runtime/v2/README.md#logging

Currently we support fifo/file/npipe

@Mossaka
Copy link
Member

Mossaka commented Jun 25, 2024

What's the main benefits of supporting Binary logging?

@squillace
Copy link

the main benefit of binary logging is that you can do highly structured snapshots and various other customized things to not only compress but treat the log as a kind of database. So, for example, systemd and mysql (in fact, almost every db or db service) uses binary logging. Size is not the main concern, but "structured responsiveness".

outside of that, you can just compress the text log if size is your only concern (though ongoing size is typically smaller with binary).

In this case, we have a Windows customer who needs this to make runwasi a "drop-in" replacement for the current shim.

@jsturtevant
Copy link
Contributor Author

From containerd/containerd#6639:

Example policies that a custom log plugin might enforce include:
Log data is pre-processed to remove potentially sensitive information (e.g. via https://github.com/stripe/unilog)
Pre-processed log data is not allowed to be written to persistent disk.
Log lines may be persisted to remote storage at different sampling levels depending on content.

learned a little bit more about how binary loggers should work: they are expected to be called, and piping I/O in to the custom logger. An example the logger is in containerd/containerd#3085

@Mossaka
Copy link
Member

Mossaka commented Jul 2, 2024

Yeah sounds good. Perhaps this issue belongs to rust-extensions repo, though?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants