Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[barback] pub serve uses many file watcher on Mac OS #18141

Closed
andersjohnsen opened this issue Apr 10, 2014 · 14 comments
Closed

[barback] pub serve uses many file watcher on Mac OS #18141

andersjohnsen opened this issue Apr 10, 2014 · 14 comments
Assignees
Labels
closed-cannot-reproduce Closed as we were unable to reproduce the reported issue

Comments

@andersjohnsen
Copy link

Mac is not very efficient with many file watchers (either they are hard-limited, or very slow to start when using more than ~100).

Given that Mac supports recursive file watching, that should be consider instead.

@andersjohnsen
Copy link
Author

Marked this as blocking #18087.

@nex3
Copy link
Member

nex3 commented Apr 10, 2014

The directory watcher package calls new Directory(...).watch(recursive: true) once on the root directory. If that's not creating a recursive file watcher internally, that's up to dart:io to fix.


Removed Area-Pub label.
Added Area-IO label.

@andersjohnsen
Copy link
Author

It is creating a recursive watcher.


Removed Area-IO label.
Added Area-Pub label.

@nex3
Copy link
Member

nex3 commented Apr 10, 2014

As far as I know, calling new Directory(...).watch(recursive: true) on the root watched directory is the correct way to use that API to recursively watch that directory. If that's not true, please tell us how to correctly use the API. Otherwise, we're using the API correctly, so please don't re-assign this issue to Pub.


Removed Area-Pub label.
Added Area-IO label.

@andersjohnsen
Copy link
Author

If you read the title, you can see this is about the number of file watchers, not if they are recursive or not.

You have e.g. not mentioned how many you expect it to create.

This issue is about pub's usage of file watchers, not io's implementation. Reassigning to io makes no sense. Instead close it as WontFix, and I'll continue from there on.


Removed Area-IO label.
Added Area-Pub label.

@nex3
Copy link
Member

nex3 commented Apr 10, 2014

In the issue description, you explicitly call out recursive file watching as something pub should do.

In any case, "pub serve" will call Directory.watch once for lib/, once for asset/, and once for each directory the user explicitly lists -- usually just web/ and test/. It's possible that some of these watches are restarted once to work around issue #14849. The total number of watchers started ever should be in the single digits.


Added WontFix label.

@munificent
Copy link
Member

I'd like a little more context here.

Anders, did you see pub using too many watchers on Mac? How many? What command were you running? What was the package you ran it on?

WontFix isn't the right status here. Either there pub is using too many watchers in which case we should fix it, or it isn't, in which case this bug is just invalid. It's definitely not the case that we know pub uses a lot of watchers and are not deliberately fixing it.


Added NeedsInfo label.

@andersjohnsen
Copy link
Author

As reported in issue #18087, running 'pub serve' uses up too many resources.

See my comment here: https://code.google.com/p/dart/issues/detail?id=18087#c5.

The current limit I found on my machines was in the range of ~100 (actually more, but let's assume this range).

So, either 'pub serve' is using that many watchers, or the setup used when issue #18087 was seen, may have another issue.

Currently, all our samples using transformers are unable to fetch packages:

Package barback has no versions that match >=0.13.0-dev <0.13.0 derived from:

  • polymer depends on version >=0.9.0 <0.13.0
  • pub itself depends on version >=0.13.0-dev <0.14.0

I'll look at this again, once our samples are fixed.

@andersjohnsen
Copy link
Author

(marking as owner for now, so I don't forget)


Set owner to @Skabet.

@nex3
Copy link
Member

nex3 commented Apr 11, 2014

We haven't seen any complaints from other users matching those in issue #18087, nor does pub start anywhere near 100 watchers. You can verify this yourself by running "pub serve" in any package; whether the package uses transformers or not is irrelevant, so you can just avoid polymer packages until versioning is sorted out.

Given that, I think we should proceed assuming there was something unique to Yegor's package that triggered the issue.

@munificent
Copy link
Member

I'll look at this again, once our samples are fixed.

These should be fixed now.

I think we should proceed assuming there was something unique to Yegor's package that triggered the issue.

I'd still like to have more details here. As far as I know, you'd have to have a very pathologically strange package (like one with a hundred (!) path dependencies) to get that many watchers. That seems pretty unlikely, so maybe there is some other bug going on here.

@andersjohnsen
Copy link
Author

I see, and it makes a lot of sense if it's due to a special case package. What I noticed was the following:

  What steps will reproduce the problem?
  1. pub serve on any project with transformers

But maybe Yegor can shed some light on this.

@andersjohnsen
Copy link
Author

I'll close this, so we can move the discussion back to issue #18087.


Added CannotReproduce label.

@DartBot
Copy link

DartBot commented Jun 5, 2015

This issue has been moved to dart-lang/pub#952.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-cannot-reproduce Closed as we were unable to reproduce the reported issue
Projects
None yet
Development

No branches or pull requests

5 participants