feat: fs events #3452

bartlomieju · 2019-12-06T18:58:29Z

Prototype for file watching.

Uses https://docs.rs/notify/5.0.0-pre.2/notify/index.html

I used some code from @jcao219

cli/ops/fs.rs

bartlomieju · 2019-12-08T13:33:26Z

After more digging I'm not sure fs events can solved elegantly at the moment. There's no way to turn this sync API into async one.

Once we land Tokio 0.2 there's spawn_blocking function that can be coupled with recv_timeout. So postponing this PR until #3387 is done

bartlomieju · 2020-01-23T17:44:34Z

Not sure how to test it yet... Docs state that contents of event depend on the platform.

Example output for fs_watch_test.ts on macOS:

{"type":{"create":{"kind":"folder"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"content"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file1.txt"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"content"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file2.txt"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file1.txt"],"attrs":{}}
{"type":{"remove":{"kind":"file"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"content"}},"paths":["/private/var/folders/4k/qds_txm54r5ffc909vqvgdkw0000gn/T/715c24c7/file1.txt"],"attrs":{}}

Also serialization output is really questionable in my opinion...

ry · 2020-01-23T20:09:43Z

I think a platform dependent test would be fine. Probably a custom integration test would be the best way to test this. Create a temp dir, add file, touch it, etc - check events are issued in JS.

bartlomieju · 2020-01-23T21:32:00Z

Test case:

  const testDir = await Deno.makeTempDir();
  const file1 = testDir + "/file1.txt";
  const file2 = testDir + "/file2.txt";

  const watcher = Deno.watch(testDir, { recursive: true });
  // start capturing events in the background
  await Deno.writeFile(file1, new Uint8Array([0, 1, 2]));
  await Deno.writeFile(file2, new Uint8Array([0, 1, 2]));
  await Deno.remove(file1);
  await Deno.rename(file2, file1);
  await Deno.chmod(file1, 0o666);
  const f = await Deno.open(file1);
  f.close();

  setTimeout(() => {
    watcher.close();
  }, 750);

ubuntu:

{"type":{"create":{"kind":"file"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"any"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{}}
{"type":{"access":{"kind":"close","mode":"write"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/tmp/392eadc2/file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"any"}},"paths":["/tmp/392eadc2/file2.txt"],"attrs":{}}
{"type":{"access":{"kind":"close","mode":"write"}},"paths":["/tmp/392eadc2/file2.txt"],"attrs":{}}
{"type":{"remove":{"kind":"file"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"rename","mode":"from"}},"paths":["/tmp/392eadc2/file2.txt"],"attrs":{"tracker":2133}}
{"type":{"modify":{"kind":"rename","mode":"to"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{"tracker":2133}}
{"type":{"modify":{"kind":"rename","mode":"both"}},"paths":["/tmp/392eadc2/file2.txt","/tmp/392eadc2/file1.txt"],"attrs":{"tracker":2133}}
{"type":{"modify":{"kind":"metadata","mode":"any"}},"paths":["/tmp/392eadc2/file1.txt"],"attrs":{}}

macOS

{"type":{"create":{"kind":"folder"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file1.txt"],"attrs":{}}
{"type":{"remove":{"kind":"file"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"content"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file1.txt"],"attrs":{}}
{"type":{"create":{"kind":"file"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"rename","mode":"from"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"data","mode":"content"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"rename","mode":"from"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"metadata","mode":"ownership"}},"paths":["/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/679f3d6f/file1.txt"],"attrs":{}}

Windows

{"type":{"create":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file1.txt"],"attrs":{}}
{"type":{"create":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file2.txt"],"attrs":{}}
{"type":{"remove":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"rename","mode":"from"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file2.txt"],"attrs":{}}
{"type":{"modify":{"kind":"rename","mode":"to"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file1.txt"],"attrs":{}}
{"type":{"modify":{"kind":"any"}},"paths":["C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\9d0f8c00\\file1.txt"],"attrs":{}}

ry · 2020-01-23T21:53:48Z

feel free to make the test sloppy - i don't think nailing down the exact order is important at this stage

bartlomieju · 2020-01-23T21:59:45Z

feel free to make the test sloppy - i don't think nailing down the exact order is important at this stage

Sure, what about return type from watcher? What you see above is what notify serializes to

bartlomieju · 2020-02-18T01:16:02Z

This is getting stale; the only thing left to do in it is providing some unified interface between different OSes (see #3452 (comment)).

cli/ops/fs.rs

ry

Finally working!

nayeemrmn · 2020-02-21T18:31:42Z

cli/js/fs_events.ts

+
+export function fsEvents(
+  paths: string | string[],
+  options = { recursive: true }


This should be written as

{ recursive = true }: FsEventsOptions = {}

for full robustness.

magichim · 2020-03-01T14:02:24Z

Deno.watch feature is added at this time in 0.35 version?
@bartlomieju

bartlomieju · 2020-03-01T14:06:49Z

Yes https://github.com/denoland/deno/releases/tag/v0.35.0

ry · 2020-03-01T14:07:33Z

https://deno.land/typedoc/index.html#fsevents

prototype file watching

3385a59

bartlomieju commented Dec 6, 2019

View reviewed changes

cli/ops/fs.rs Outdated Show resolved Hide resolved

add test

775be5a

bartlomieju changed the title ~~feat: fs events~~ [WIP] feat: fs events Dec 6, 2019

bartlomieju closed this Jan 21, 2020

Merge branch 'master' into feat-fs_events

b2762df

bartlomieju reopened this Jan 23, 2020

bartlomieju added 2 commits January 23, 2020 17:32

fixes for new notify

1ba4acb

fixes

551c122

bartlomieju changed the title ~~[WIP] feat: fs events~~ feat: fs events Jan 23, 2020

bartlomieju added 2 commits January 23, 2020 17:57

remove debounce

05a31a3

add EventType enum

7bb79e6

bartlomieju added 3 commits January 23, 2020 21:26

debug tests

630603f

more debug logs

849ff0b

fmt

4dee826

bartlomieju mentioned this pull request Jan 26, 2020

fs events ops #1826

Closed

Merge branch 'master' into feat-fs_events

98aa812

bartlomieju added the help wanted label Feb 18, 2020

Merge branch 'master' into feat-fs_events

e6e264d

bartlomieju commented Feb 20, 2020

View reviewed changes

cli/ops/fs.rs Outdated Show resolved Hide resolved

bartlomieju commented Feb 20, 2020

View reviewed changes

cli/ops/fs.rs Outdated Show resolved Hide resolved

bartlomieju commented Feb 20, 2020

View reviewed changes

cli/ops/fs.rs Outdated Show resolved Hide resolved

update to latest State

4d67c97

ry added 2 commits February 20, 2020 19:37

fix

72a217b

manual

744f383

ry removed the help wanted label Feb 21, 2020

ry added 10 commits February 21, 2020 00:40

Merge branch 'master' into feat-fs_events

62ada69

x

d4bfd61

test

ac8e48f

use AsyncIterableIterator return method to close

22add5e

fix

648fbac

remove delay

63da3c7

Merge branch 'master' into feat-fs_events

b477372

fix

b22e093

fmt

f6efa52

try removing Deno.runTests()

f670d5e

ry approved these changes Feb 21, 2020

View reviewed changes

ry merged commit bd640bc into denoland:master Feb 21, 2020

bartlomieju deleted the feat-fs_events branch February 21, 2020 18:25

ry mentioned this pull request Feb 21, 2020

1.0 #2473

Closed

43 tasks

nayeemrmn reviewed Feb 21, 2020

View reviewed changes

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 21, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

284237d

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 24, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

e837ed5

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 24, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

8c2913f

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 24, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

0cbfd2e

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 31, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

d37aab4

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 31, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

a32b842

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 31, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

4d8d9da

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Jan 31, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

c98af1c

caspervonb pushed a commit to caspervonb/deno_std that referenced this pull request Feb 1, 2021

feat: Deno.fsEvents() (denoland/deno#3452)

4021875

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: fs events #3452

feat: fs events #3452

bartlomieju commented Dec 6, 2019 •

edited

Loading

bartlomieju commented Dec 8, 2019

bartlomieju commented Jan 23, 2020

ry commented Jan 23, 2020

bartlomieju commented Jan 23, 2020

ry commented Jan 23, 2020

bartlomieju commented Jan 23, 2020

bartlomieju commented Feb 18, 2020

ry left a comment

nayeemrmn Feb 21, 2020

magichim commented Mar 1, 2020

bartlomieju commented Mar 1, 2020

ry commented Mar 1, 2020

feat: fs events #3452

feat: fs events #3452

Conversation

bartlomieju commented Dec 6, 2019 • edited Loading

bartlomieju commented Dec 8, 2019

bartlomieju commented Jan 23, 2020

ry commented Jan 23, 2020

bartlomieju commented Jan 23, 2020

ry commented Jan 23, 2020

bartlomieju commented Jan 23, 2020

bartlomieju commented Feb 18, 2020

ry left a comment

Choose a reason for hiding this comment

nayeemrmn Feb 21, 2020

Choose a reason for hiding this comment

magichim commented Mar 1, 2020

bartlomieju commented Mar 1, 2020

ry commented Mar 1, 2020

bartlomieju commented Dec 6, 2019 •

edited

Loading