-
-
Notifications
You must be signed in to change notification settings - Fork 605
Syscalls
By design, most applications running on OSv do not execute system calls when calling the libc functions. For example, an invocation of a mmap() is a direct local function call resolved by OSv dynamic linker that involves very few instructions and is therefore very fast. On Linux, the same call is way more expensive as it goes through a wrapper function in glibc which then invokes the system call SYS_mmap that involves a CPU ring and virtual address space switch among other things. This OSv optimization may not be as relevant as one would hope, especially when applications make few mmap() calls as is often the case, but this is a topic for another story.
Some applications like Golang or statically linked applications (see this for more details) bypass the libc layer and invoke systems calls directly using the SYSCALL (x86_64) or SVC (aarch64) instructions. To support those, OSv implemented the system call handler machinery in assembly for both x86_64 and aarch64.
Unlike Linux, where libc functions like mmap() delegate to the corresponding system calls (SYS_mmap in the example above), in OSv the opposite happens. Just like in Linux, OSv implements the SYSCALL and SVC instructions for x86_64 and aarch64 respectively (see syscall_entry in arch/x64/entry.S and handle_system_call in arch/aarch64/entry.S). This tricky low-level assembly code (~50 instructions) switches to a dedicated system call stack, saves all necessary registers, and delegates to syscall_wrapper() and eventually syscall() functions implemented in C++ in linux.cc. Finally, the syscall() function has a case statement that calls the relevant libc function implemented by OSv.
Sometimes there is no libc function that syscall() can directly invoke by a simple preprocessor statement like below:
SYSCALL2(listen, int, int);In those cases like futex() or getdents64() we implement relevant functions which we delegate to like so:
int futex(int *uaddr, int op, int val, const struct timespec *timeout,
int *uaddr2, uint32_t val3)
{
switch (op & FUTEX_CMD_MASK) {
...
}
SYSCALL6(futex, int *, int, int, const struct timespec *, int *, uint32_t);As one can tell, the system call invocations in OSv are slower and more expensive than the regular direct local function calls but should still be faster than in Linux because OSv does not have to switch the virtual memory mapping nor CPU ring.
Please note that even if OSv does not implement (aka expose) a specific system call by delegating to the relevant libc function, the syscall() function would log such a fact and return ENOSYS instead of crashing:
syscall(): unimplemented system call <nnn>
Here is a list of 73 system calls OSv exposes as of April 2023:
accept4bindclock_getresclock_gettimecloseconnectdup3epoll_create1epoll_ctlepoll_pwaitepoll_waiteventfd2exitexit_groupfcntlfdatasyncflockfstatfstatatfsyncftruncatefutexgetcwdgetdents64getgidget_mempolicygetpeernamegetpidgetrandomgetsocknamegetsockoptgettidgetuidioctllistenlseekmadvisemincoremkdirmkdiratmmapmunmapnanosleepopenopenatpipe2pread64pselect6pwrite64readreadlinkatrecvfromrecvmsgrenameatrt_sigactionrt_sigprocmasksched_getaffinitysched_setaffinitysched_yieldselectsendmsgsendtoset_mempolicysetsockoptsigaltstacksocketstatstatfssymlinkattgkillunameunlinkatwrite
Most of the time implementing (or exposing) new system calls is very trivial as long as there is a corresponding libc function in OSv (see the introduction section). Here is a list of 60 systems calls that should be trivial to expose (minus those that do not exactly map one-to-one to the libc functions):
acceptaccessalarmchdircreatdupdup2epoll_createeventfdfallocatefaccessatfchdirfstatfsfutimesatgetitimergetprioritygetrlimitgetrusagegettimeofdaykilllstatmprotectmsyncpausepipepollppollprctlreadlinkreadvrenamermdirsched_get_priority_maxsched_get_priority_minsendfilesethostnamesetitimersetprioritysetrlimitshmgetshmatshmctlshmdtshutdownsocketpairsymlinksyncsysinfotimetimerfd_createtimerfd_gettimetimerfd_settimetimestruncateumaskunlinkutimeutimensatutimeswritev