Optimization: use close_range(2) if available

[kevans91]

BPO	40423
Nosy	@gpshead, @miss-islington, @kevans91, @kevans91
PRs	bpo-40423: Optimization: use close_range(2) if available #22651
Files	cpython-close_range.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2020-10-19.20:38:17.508>
created_at = <Date 2020-04-28.14:10:08.486>
labels = ['expert-C-API', 'type-feature', '3.10']
title = 'Optimization: use close_range(2) if available'
updated_at = <Date 2020-10-19.20:38:17.507>
user = 'https://github.com/kevans91'

bugs.python.org fields:

activity = <Date 2020-10-19.20:38:17.507>
actor = 'gregory.p.smith'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2020-10-19.20:38:17.508>
closer = 'gregory.p.smith'
components = ['C API']
creation = <Date 2020-04-28.14:10:08.486>
creator = 'kevans91'
dependencies = []
files = ['49097']
hgrepos = []
issue_num = 40423
keywords = ['patch']
message_count = 6.0
messages = ['367531', '378445', '378448', '378449', '378452', '379011']
nosy_count = 4.0
nosy_names = ['gregory.p.smith', 'miss-islington', 'kevans', 'kevans91']
pr_nums = ['22651']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40423'
versions = ['Python 3.10']

[kevans91]

This is dependent on bpo-40422; the diff on top of that (PR19075) looks like the attached. Effectively, close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

PR will be sent after bpo-40422 is resolved.

[gpshead]

for reference, very recent Linux kernels appear to have gained a close_range syscall. http://lkml.iu.edu/hypermail/linux/kernel/2008.0/02649.html

Your diff isn't quite sufficient as is. When depending on a syscall that has a function provided by libc, the libc function may exist (thus HAVE_CLOSE_RANGE will be true at Python compile time) but the system the process is running on may not support the system call. So it'll return an EINVAL (or something like that) error.

Special handling of that error to add an else {...} falling back to the other codepath is necessary.

[kevans91]

Ah, I will fix this and then submit a PR, thanks... hopefully it returns ENOSYS.

[gpshead]

Ah, yeah ENOSYS is it. I had to do this trick in older subprocess versions for something else. Still visible here in the old 2.7 backport: https://github.com/google/python-subprocess32/blob/main/_posixsubprocess.c#L801

[miss-islington]

New changeset 1800c60 by Kyle Evans in branch 'master':
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
1800c60

[gpshead]

thanks Kyle!