Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Darwin: FSWatch API #1333

Closed
wants to merge 21 commits into from
Closed

Conversation

kristate
Copy link
Contributor

@kristate kristate commented Aug 5, 2018

This is the Darwin Branch of #1294 ;

Highlights include new platform agnostic API.

}
}

// LinuxOsData MacOsData are not directly
// included into the struct because the compiler cannot resolve
// os specific members on other platforms -- this could be a bug.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewrk hey, is this a bug?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still thinking about this one. We have to decide whether name resolution is lazy or not. This is actually related to #21 and I'll use the insight from implementing that issue when figuring out how to deal with this situation.

@kristate
Copy link
Contributor Author

kristate commented Aug 5, 2018

I think we are in a pretty good place.

pwritev doesn't exist on darwin, but calling pwrite consecutively is recommended.

https://github.com/kristate/zig/blob/2d064bae9f8ca4d263ae58b04b484fa1a8314190/std/event/loop.zig#L676-L702

@andrewrk
Copy link
Member

andrewrk commented Aug 7, 2018

I think that we will use FSEvents on MacOS eventually. However, a pure kqueue solution (with an fd per file) will still be necessary on some of the BSDs. So we may as well implement that, because it's simpler and pushes off #1349 to solve later.

@kristate
Copy link
Contributor Author

kristate commented Aug 8, 2018

@andrewrk I agree about kqueue.
macOS is almost done, but we should also start to support BSD.

We should eventually have these implementations for FS watching:

  • FSEvents for Darwin
  • File Events Notification for Solaris
  • kqueue for the BSDs
  • inotify for linux (you have this already)
  • ReadDirectoryChangesW on Windows

I also found this list of limitations from the @emcrisostomo's fswatch project:

  • The FSEvents monitor, available only on OS X, has no known limitations,
    and scales very well with the number of files being observed.

  • The File Events Notification monitor, available on Solaris kernels and
    its derivatives, has no known limitations.

  • The kqueue monitor, available on any *BSD system featuring kqueue,
    requires a file descriptor to be opened for every file being watched. As a
    result, this monitor scales badly with the number of files being observed,
    and may begin to misbehave as soon as the fswatch process runs out of file
    descriptors. In this case, fswatch dumps one error on standard error for
    every file that cannot be opened.

  • The inotify monitor, available on Linux since kernel 2.6.13, may suffer
    a queue overflow if events are generated faster than they are read from the
    queue. In any case, the application is guaranteed to receive an overflow
    notification which can be handled to gracefully recover. fswatch
    currently throws an exception if a queue overflow occurs. Future versions
    will handle the overflow by emitting proper notifications.

  • The Windows monitor can only establish a watch directories, not files.
    To watch a file, its parent directory must be watched in order to receive
    change events for all the directory's children, recursively at any depth.
    Optionally, change events can be filtered to include only changes to the
    desired file.

@andrewrk
Copy link
Member

andrewrk commented Aug 8, 2018

I think the biggest challenge we face in order to support more target platforms is infrastructure - we absolutely must have CI building and testing for all supported platforms. See #835. Once we have automated testing set up for one of the BSDs, supporting it will be a piece of cake.

@kristate kristate changed the title [WIP] Darwin Branch of #1294 (async-fs) [WIP] Darwin: FSWatch API Aug 8, 2018
@andrewrk andrewrk added the work in progress This pull request is not ready for review yet. label Aug 20, 2018
@andrewrk andrewrk added this to the 0.4.0 milestone Aug 25, 2018
@andrewrk
Copy link
Member

Let's solve #1349 before we add any dependencies on frameworks to the standard library. We can re-evaluate using FSWatch later.

@andrewrk andrewrk closed this Sep 15, 2018
@andrewrk andrewrk modified the milestones: 0.4.0, 0.3.0 Sep 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
work in progress This pull request is not ready for review yet.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants