-
Notifications
You must be signed in to change notification settings - Fork 633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP chore(log): migrate std/log to use the Web Streams API #3969
Conversation
c576bb2
to
28c700f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test are now passing.
Big changes since the last review are:
- The log string is now encoded into a
Uint8Array
much earlier in the process, this means we only need to do it once rather than 2 or 3 times. - LogQueue is a new class which essentially converts synchronous calls to
.enqueue
into anAsyncIterable
. We use this to make sure:- All logs have been written when calling
flush
- We only process one log line at a time making sure we avoid race conditions with either
flushToBuffer
orrotateLogFiles
. I'm not 100% sure there would be race conditions now, but it could be easy to introduce in the future. - See here for further justification
- All logs have been written when calling
- Flush is now async, that means destroy is now async, this has a knock-on effect on a few things, especially tests.
- We now re-create the
Buffer
(not theArrayBuffer
just the wrapper) each time we flush to file. I'm not sure if it's intended behaviour but if we don't recreate theBuffer
it fails to write any data after the first flush.
@iuioiua would you mind taking a look at the changes? I've added some questions and TODOs in comments in the code.
I know LogQueue might seem a tad over-engineered, but it feels like a really nice abstraction and could be used for other things (see the comment about moving it). If the logic was implemented inline in the FileHandler class it would just add complexity to an already complex class.
I'm open to critical feedback, and looking forward to progressing this PR. I'll try to spend a few more hours on it over the holidays.
Due to the fact that the unload lifecycle event handler does not await promises, using the async streams API is very risky as the last flush on unload would not complete. As an alternative I've created this new PR #4021 that keeps to synchronous functions. It removes all |
Closing in favour of #4021. |
Tackling this issue: #3907
Whats left to do in this PR:
Initial Idea:
this._buf.readable.pipeTo(this._file.writable);
to pipe the buffer to the file once full.I'm not sure how much time I can dedicate to this over the next few weeks so if anyone wants to pick this up, you are welcome to.