sendfile syscall

[arobase-che]

Should we implement the sendfile syscall to TCPStream and/or UDPStream ?

Even if it's not totally portable (Linux and BSD, windows using transmitfile), I may be a convenient way to send a file effectively.

[hellow554]

What about macOS? At least all tier1 targets should be supported at least.
Also I'm not really convinced why this should be in std and not in an own crate. Maybe you can elaborate it further and convince me.

@rustbot modify labels: C-enhancement T-libs

[arobase-che]

Should work on macOS too but i never had used macOS. Mac OS sendfile documentation

---

I'm not sure it should be in std. Actually, it's a syscall so a very low level feature. As send, accept, bind, recvfrom, ... that are already in std::net. Other languages add it to the std (Python), Go use it internally in his « std » but doesn't provide a way to use it with a socket). But others languages as Haskell uses external lib. Some simply don't (D socket's)

I know i may be more convincing but I'm not sure myself ! In fact, i'm asking it.

It seems to me to be the right way to add it to the std. But maybe it's because I don't know how it could be implemented from an external crate.

[abonander]

This could be an internal specialization for io::copy(), and/or maybe exposed in std::os::unix::io as

pub fn send_file<R: AsRawFd, W: AsRawFd>(r: &mut R, w: &mut W) -> io::Result<u64> {...}

Windows actually has a painfully close equivalent but it's specifically to copy from a file handle to a socket (which are separate object types in the Windows API) whereas sendfile() works for any pair of file descriptors, so it couldn't be the same interface unless separate traits were used for the read side and the write side and one was only implemented by File and the other by the net stream types.

https://docs.microsoft.com/en-us/windows/desktop/api/mswsock/nf-mswsock-transmitfile

[abonander]

@arobase-che sendfile() is actually already available in the the libc crate but that's the unsafe extern bindings to the actual function in sys/sendfile.h.

It's of course completely possible to build a safe wrapper around it and it's actually pretty trivial:

extern crate libc;
use std::os::unix::io::AsRawFd;
use std::{io, ptr};


// mutability is not technically required but it fits API conventions
pub fn send_file<R: AsRawFd, W: AsRawFd>(r: &mut R, w: &mut W) -> io::Result<usize> {
    // parameter ordering is reversed
    // null pointer is an out-pointer for the offset after the read, if not null it doesn't update the file cursors which we actually do want
    // last argument is the maximum number of bytes to copy but 
    // the documentation says it stops at 2^31-1 regardless of arch
    match unsafe { libc::sendfile(w.as_raw_fd(), r.as_raw_fd(), ptr::null_mut(), usize::MAX) } {
        -1 => Err(io::Error::last_os_error()),
        copied => Ok(copied as usize), 
    } 
}

[abonander]

The libc wrapper function for reference: http://man7.org/linux/man-pages/man2/sendfile.2.html

[the8472]

This could be an internal specialization for io::copy()

That would be quite a rabbit hole. There are lots of special copy syscalls on linux.

splice: any -> pipe; pipe -> any
vmsplice: &'static [u8] -> pipe
sendfile: mmapable fd -> socket
copy_file_range: originally just regular -> regular on same file system, but support is expanding

[steveklabnik]

Triage: I don't think this was ever implemented.

[the8472]

There's a partial implementation via #75272, you can sendfile to a socket by using io::copy. It's currently linux-specific and would have to be generalized to work on other platforms.

[the8472]

There's a partial implementation via #75272, you can sendfile to a socket by using io::copy. It's currently linux-specific and would have to be generalized to work on other platforms.

The sendfile optimization for copying to a socket had had to be removed in #108283 since sendfile violates ordering guarantees implicit in the Read/Write traits.

So this will have to be implemented as a separate API which makes fewer promises.

[arobase-che]

The situation is pretty clear now. I think we can close the issue.
Thank you everybody.