Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overwrite old data #19

Open
git-blame opened this issue Jan 18, 2025 · 2 comments
Open

Overwrite old data #19

git-blame opened this issue Jan 18, 2025 · 2 comments

Comments

@git-blame
Copy link

I find this package useful in several scenarios. Now I am looking to maintain a rolling log buffer. But I realized that this ring buffer does not overwrite old data like the traditional definition of a ring/circular buffer.

It is simple to implement a Writer wrapper to do this. For example, simply reading some bytes from the read end if needed. But I will have to ensure concurrency because although the buffer's functions are thread-safe, calling them separately are not.

But before I do, I want to confirm that overwriting data is not supported at this time. Or is there a mode I can set that allows this functionality?

@smallnest
Copy link
Owner

But I realized that this ring buffer does not overwrite old data like the traditional definition of a ring/circular buffer.

Why do you think it does not overwrite old data? Since It is a ringbuffer, it should be reuse the buffer.

@git-blame
Copy link
Author

In ringbuffer.Write code, if it is non-blocking, it returns ErrIsFull.

        err = r.setErr(err, true)
        if r.block && (err == ErrIsFull || err == ErrTooMuchDataToWrite) {
...
    if r.block && wrote > 0 {
        r.writeCond.Broadcast()
    }

    return wrote, r.setErr(err, true)

Also, I am testing with hclog which can write to both stderr and a Writer. Using a ringbuffer of 1024 bytes, I see that it has the 1st 1024 bytes of log data. If it is overwriting old data, I expect to see the last 1024 bytes of log.

End of logs on stderr:

...
2025-01-20T14:29:49.665Z [DEBUG] native_plugin: 2025/01/20 14:29:49 [DEBUG] plugin: plugin server: accept unix /tmp/plugin4106242905: use of closed network connection
2025-01-20T14:29:49.665Z [INFO]  plugin process exited: plugin=modules/native_plugin id=499058
2025-01-20T14:29:49.666Z [DEBUG] plugin exited
2025-01-20T14:29:49.666Z [INFO]  Stopping all pipeline plugins

logs in ringbuffer:

2025-01-20T14:29:47.642Z [DEBUG] Loading plugin: export
2025-01-20T14:29:47.642Z [DEBUG] starting plugin: path=modules/analytics_service_plugin args=[modules/analytics_service_plugin]
2025-01-20T14:29:47.644Z [DEBUG] plugin started: path=modules/analytics_service_plugin pid=499052
2025-01-20T14:29:47.644Z [DEBUG] waiting for RPC address: plugin=modules/analytics_service_plugin
2025-01-20T14:29:47.647Z [DEBUG] analytics_service_plugin: plugin address: address=/tmp/plugin1632902623 network=unix timestamp=2025-01-20T14:29:47.647Z
2025-01-20T14:29:47.648Z [DEBUG] using plugin: version=1
2025-01-20T14:29:47.650Z [DEBUG] Adding plugin: export
2025-01-20T14:29:47.650Z [DEBUG] Loading plugin: network_discover
2025-01-20T14:29:47.650Z [DEBUG] starting plugin: path=modules/native_plugin args=[modules/native_plugin]
2025-01-20T14:29:47.651Z [DEBUG] plugin started: path=modules/native_plugin pid=499058
2025-01-20T14:29:47.651Z [DEBUG] waiting for RPC address: plugin=modules/native_plugin
2025-01-20T14:29:47.654Z [DEBUG]

Note by the timestamp, etc. that ringbuffer contains the first logs until it is full. If it is circular, it should contain the last logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants