Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to create clone/CoW copies #12902

Closed
candrews opened this issue May 8, 2017 · 8 comments
Closed

Provide a way to create clone/CoW copies #12902

candrews opened this issue May 8, 2017 · 8 comments
Labels
feature request Issues that request new features to be added to Node.js. fs Issues and PRs related to the fs subsystem / file system.

Comments

@candrews
Copy link

candrews commented May 8, 2017

Currently, copying files is implemented by reading the data of the source file and writing it to the new file, see fs-extra's implementation of copy. The problem is that this approach doesn't allow the OS to optimize file copying (such as by using direct disk-to-disk copy if it's available), nor does it allow the (optional or by default) use of reflink/CoW.

I'm specifically interested in making CoW (which is already available on some *nix file systems, including btrfs and zfs) available to nodejs.

IMHO, nodejs should add an fs.copy function (just like what fs-extra already does) which is implemented using the OS native file copying syscall. It should have an optional option parameter that allows specifying the copy should use CoW (if available), just like how cp --reflink=auto from coreutils works on Linux.

Note that this would be particularly benefit for cases where files are copied often - particularly big files. I'm thinking that this improvement would substantially benefit package managers (npm, yarn, etc).

@addaleax addaleax added feature request Issues that request new features to be added to Node.js. fs Issues and PRs related to the fs subsystem / file system. labels May 8, 2017
@eljefedelrodeodeljefe
Copy link
Contributor

Likely needs libuv implementation. Related: libuv/libuv#925

@Daniel15
Copy link

Daniel15 commented Jun 6, 2017

Also see some of the attempts at using native copy calls in Yarn:

yarnpkg/yarn#3290 which uses https://github.com/sciolist/fcopy (Unix via a native module)
yarnpkg/yarn#2960 (Win32 via node-ffi)

@richardlau
Copy link
Member

fs.copyFile was added by #15034.

@cjihrig
Copy link
Contributor

cjihrig commented Sep 8, 2017

FWIW, CoW is not supported yet. There is a PR to add it to libuv but I haven't landed it yet because I have yet to see it work on any of the CI machines.

@richardlau
Copy link
Member

Let's keep this open then.

@richardlau richardlau reopened this Sep 8, 2017
@coderaiser
Copy link
Contributor

Ponyfill fs-copy-file can be used for copying files. In node.js < v8.5.0 it copies files with help of streams and for v8.5.0 and next versions it uses native implementation of fs.copyFile.

@Daniel15
Copy link

Daniel15 commented Oct 4, 2017

@coderaiser - That looks like a useful module. Although, technically that's not a ponyfill as ponyfills never use the native API (as per the document you linked to) 😃

@cjihrig
Copy link
Contributor

cjihrig commented Apr 6, 2018

Forgot to link it, but this is closed in a16d88d.

@cjihrig cjihrig closed this as completed Apr 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Issues that request new features to be added to Node.js. fs Issues and PRs related to the fs subsystem / file system.
Projects
None yet
Development

No branches or pull requests

7 participants