-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: opening/reading from fifo/chardev files are blocking the thread #14255
Fix: opening/reading from fifo/chardev files are blocking the thread #14255
Conversation
Disk files are always opened in a blocking way because epoll (for instance) will always say that a file is ready for reach or write. But this is only for regular disk files, and that doesn't apply to pipes and characters devices for example. Opening `/dev/tty` (a character device) will block the current thread when trying to read from it until you start typing on the keyboard. IO::FileDescriptor was taking care to check the file type, but File always told it to be blocking. Worse, trying to open a fifo/pipe file will block the current thread until another thread or process also opens it, which means that we must determine whether to block _before_ trying to open a file (not change it afterwards). I also added a `blocking` parameter to `File.open` and `File.new`.
I'm not so sure whether |
I'm also conflicted: I'm not fond of having to call Maybe we could require to specify |
Yeah IIUC there are three aspects to patch which are all related, but I think they can be implemented largely independently? Can we separate them into different PRs?
I understand 1. should be relatively simple to resolve, just avoid overriding the implicit value from |
Hence my proposal: add a |
Are you sure that's enough? The event loop doesn't handle open calls. So this would just continue execution while the file descriptor is not open yet. I suppose any read or write operation will eventually suspend the fiber until the file descriptor is open. But are we sure that no other operation can cause any harm on a non-blocking file descriptor while it's not open yet? |
Good point:
Hum 🤔 That would only happen when we explicitly pass
|
Removes the automatic detection of the file type and blocking behavior before opening the file as we must support at least EINTR and ENXIO return values for `open(2)`. Blocking still defaults to true. Developers may pass nil for auto detection or false to force the nonblocking flag (after open). Opening a fifo without another thread/process also trying to open for read or write will still block the current thread (will be another PR).
The interpreter specs are failing. Do you know if that's related @ysbaddaden ? |
Interpreter specs should be fixed by #14287 |
Starting threads very likely requires support from the interpreter to create the thread (so it knows about it) to run the interpreted code in. This also fixes the "can't resume running fiber" exceptions that started appearing in the wait group pull request, because they were always run after the thread related specs.
Nope. I cherry-picked the commit from #14287 and this still fails, apparently in
EDIT: this is because I'm calling |
Note to self: asynchronous regular file I/O on Windows also depends on this since Windows disallows changing a file handle to the opposite blocking mode after creation |
@ysbaddaden Not sure if this solves my issue with raw mode: The following example does nothing at all. Am I thinking about this correctly? Seems to me that there is a problem with raw mode and non-blocking opening of /dev/tty.
|
@davidmatter I have no issues in a Linux terminal. Neither with this example or the one from https://forum.crystal-lang.org/t/help-with-channels-fibers/6354 after I add CTRL_C = '\u0003'
def read_tty_and_send(channel : Channel(Char))
File.open("/dev/tty", "r", blocking: false) do |tty|
while char = tty.raw &.read_char
if char == CTRL_C
channel.close
break
else
channel.send char
end
end
end
rescue ex
puts "Error reading from /dev/tty: #{ex.message}"
end
# Create a channel for Char
keystroke_channel = Channel(Char).new
# Start the tty reading in a separate fiber
spawn read_tty_and_send(keystroke_channel)
# Read from the channel
while char = keystroke_channel.receive?
print "Received keystroke: #{char.inspect}\r\n"
end |
Hey @ysbaddaden thanks for trying it out. So it was my darwin setup all along :) Seems we have an issue only on (my?) macOS. Chip: M1 Pro |
The following works perfectly fine. If you can give me a hint where the problem could be in the crystal stdlib, I can take a look.
|
This might be the problem, not sure about that, though. Maybe @gdamore knows some things about this. https://github.com/gdamore/tcell/blob/main/tty_unix.go#L60 |
Yeah, that's probably MacOS acting out. There's nothing wrong with your code, but maybe there's something in The difference with the C code is that |
Alright, this one works :)
|
This one does not, I'll try to dig deeper this weekend:
|
Note to self: There seems to be a problem with "Crystal::Scheduler.reschedule" in evented IO. Without the rescheduling it seems to work. Trying to find out why.
|
That seems to be the event loop: Basically, your change is making the read into a busy loop ( |
Disk files are always opened in a blocking way because epoll (for instance) will always say that a file is ready for read or write, but this is only true for regular disk files, and that doesn't apply to pipes and characters devices.
For example opening
/dev/tty
(a character device) will block the current thread when trying to read until we start typing on the keyboard. IO::FileDescriptor was taking care to check the file type, but File always told it to be blocking.Worse, trying to open a fifo (pipe) file will block the current thread until another thread or process also opens it, which means that we must determine whether to block before trying to open a file (not make it nonblocking afterwards).
I also added a
blocking
parameter toFile.open
andFile.new
. Specifying a boolean value will always open the file in blocking or nonblocking mode, avoiding a call tostat(2)
, though it shouldn't be an issue unless an application is opening lots of files.closes #8152