Considering the File System API

[andreubotella]

The File System API is a relatively recent web standard, now implemented in all browsers, that allows accessing and modifying files in the file system. All browsers allow access through this API to a virtual filesystem, unique to a page's origin, through await navigator.storage.getDirectory(), that you can't go outside of.

However, Chromium extends this by providing additional APIs (such as window.showOpenFilePicker()) that give you access to arbitrary files and directories in the file system. Although other browsers are opposed to implementing these APIs because of safety and privacy concerns, this is not an issue for server-side runtimes.

I have heard that this API is not a great fit for server-side environments, but (as someone who hasn't looked in depth at the API and considered it) those concerns are not clear to me. So I opened this issue to have a place to discuss what it's missing, whether we can work with WHATWG to fix that, and also to discuss WinterCG extensions similar to showOpenFilePicker().

[ekwoka]

I agree that I'm not too sure what the disagreement would be for the server context (outside of runtimes that inherently don't expose any FS as is).

Everything has unified around the node:fs system ,but the File System API seems to be quite a lot better thought out.

[Qard]

It lacks the full feature set and flexibility of the posix fs model, for example file encodings and permissions. Because the API is designed as a tree structure with methods on each element, such additional capabilities can not be added to it without violating the spec.

Additionally, the use of Web Streams and subsequently EventTarget makes the performance insufficient for Node.js at present.

It may be reasonable to have it as a secondary interface, but not really usable as the only way to read and write files.

[andreubotella]

Additionally, the use of Web Streams and subsequently EventTarget makes the performance insufficient for Node.js at present.

As far as I can tell, none of the FS APIs inherit from EventTarget. The only thing remotely connected is FileReader, and we're already not including that in the common minimum API even though we do include File.

As for streams, my understanding is that it is possible to achieve implementations of web streams with enough performance, so this would only be a temporary blocker for Node.js, right? After all, the usage of web APIs in Node.js is only ever going to increase, and web APIs won't be adopting Node.js's streams, so the pressure on Node.js's implementation to be more performant will do anything but decrease.

[Qard]

Web streams are based on EventTarget, so until the performance of that is resolved it's unlikely we'd be able to get at all comparable performance with the existing fs API.

As I said though, the API simply lacks the capability of the existing API though, and has quite a lot of complexity in its object model, so I don't expect to see a Node.js implementation of it any time soon.

[andreubotella]

Web streams are based on EventTarget, so until the performance of that is resolved it's unlikely we'd be able to get at all comparable performance with the existing fs API.

I don't think that's correct. The streams spec has no mention of EventTarget, and the only mentions of events in normative text are related to passing stream chunks through a worker boundary when transferring streams, which Node.js would do with EventEmitter.

As I said though, the API simply lacks the capability of the existing API though, and has quite a lot of complexity in its object model, so I don't expect to see a Node.js implementation of it any time soon.

Part of the point of this thread is to have a concrete list of capabilities that that API is missing, to see if we can work together with the WHATWG to make that possible.

[mk-pmb]

From a few minutes of reading MDN's OPFS docs and a quick scan of the Node.js fs API ToC, my impression is that polyfills will not be feasible for these Node.js features, unless you implement a full file system on top of OPFS:

[Toggle list]

filehandle.chmod(mode)
filehandle.chown(uid, gid)
filehandle.stat([options])
filehandle.utimes(atime, mtime)
fsPromises.access(path[, mode])
fsPromises.chmod(path, mode)
fsPromises.chown(path, uid, gid)
fsPromises.lchmod(path, mode)
fsPromises.lchown(path, uid, gid)
fsPromises.lutimes(path, atime, mtime)
fsPromises.link(existingPath, newPath)
fsPromises.lstat(path[, options])
fsPromises.readlink(path[, options])
fsPromises.realpath(path[, options])
fsPromises.rename(oldPath, newPath)
fsPromises.stat(path[, options])
fsPromises.symlink(target, path[, type])
fsPromises.utimes(path, atime, mtime)
fsPromises.watch(filename[, options])

A polyfill for fsPromises.mkdtemp(prefix[, options]) will probably be possible only in an insecure way.