Skip to content

Commit

Permalink
luaext/Pexec: optimize setting CLOEXEC
Browse files Browse the repository at this point in the history
In case maximum number of open files limit is set too high,
Pexec() spends way too much time trying to set FD_CLOEXEC
for all those file descriptors, resulting in severe increase
of time it takes to execute rpm or dnf. For example, this
happens while running under Docker because:

> $ docker run fedora ulimit -n
> 1048576

One obvious fix is when open_max is too big, use procfs to get
the actual list of opened file descriptors and iterate over those.
My quick-n-dirty benchmark shows the /proc approach is about 10x
faster than iterating through a list of 1024 fds.

Note that the old method is still used in case /proc is not available.

This should fix:
- moby/moby#23137
- https://bugzilla.redhat.com/show_bug.cgi?id=1537564

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
  • Loading branch information
kolyshkin committed May 17, 2018
1 parent a104c9e commit e99d3b9
Showing 1 changed file with 43 additions and 11 deletions.
54 changes: 43 additions & 11 deletions luaext/lposix.c
Original file line number Diff line number Diff line change
Expand Up @@ -330,26 +330,58 @@ static int Pmkfifo(lua_State *L) /** mkfifo(path) */
}


static void set_cloexec(int fd)
{
int flags = fcntl(fd, F_GETFD);

if (flags == -1 || (flags & FD_CLOEXEC))
return;

fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
}

static void set_cloexec_all(void)
{
const int min_fd = 3; // don't touch stdin/out/err
int fd, open_max;

open_max = sysconf(_SC_OPEN_MAX);
if (open_max == -1) {
open_max = 1024;
}

// iterate over fds obtained from /proc
DIR *dir = opendir("/proc/self/fd");
if (dir == NULL) {
goto fallback;
}

struct dirent *entry;
while ((entry = readdir(dir)) != NULL) {
fd = atoi(entry->d_name);
if (fd >= min_fd)
set_cloexec(fd);
}
closedir(dir);

return;

fallback:
// iterate over all possible fds
for (fd = min_fd; fd < open_max; fd++)
set_cloexec(fd);
}

static int Pexec(lua_State *L) /** exec(path,[args]) */
{
const char *path = luaL_checkstring(L, 1);
int i,n=lua_gettop(L);
char **argv;
int flag, fdno, open_max;

if (!have_forked)
return luaL_error(L, "exec not permitted in this context");

open_max = sysconf(_SC_OPEN_MAX);
if (open_max == -1) {
open_max = 1024;
}
for (fdno = 3; fdno < open_max; fdno++) {
flag = fcntl(fdno, F_GETFD);
if (flag == -1 || (flag & FD_CLOEXEC))
continue;
fcntl(fdno, F_SETFD, FD_CLOEXEC);
}
set_cloexec_all();

argv = malloc((n+1)*sizeof(char*));
if (argv==NULL) return luaL_error(L,"not enough memory");
Expand Down

0 comments on commit e99d3b9

Please sign in to comment.