Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/x86/crc32: add VPCLMULQDQ implementations of CRC-32 #341

Merged
merged 3 commits into from
Feb 19, 2024
Merged

Conversation

ebiggers
Copy link
Owner

This improves CRC-32 performance on some of the latest x86 CPUs. Three implementations are added: VPCLMULQDQ/AVX2 and VPCLMULQDQ/AVX512VL which use 256-bit vectors, and VPCLMULQDQ/AVX512F/AVX512VL which uses 512-bit vectors. To reduce downclocking effects, the implementation with 512-bit vectors isn't used on Intel CPUs 10th generation and older.

@ebiggers ebiggers force-pushed the dev branch 10 times, most recently from ab8661e to 7177cfb Compare February 19, 2024 06:02
This improves CRC-32 performance on some of the latest x86 CPUs.  Three
implementations are added: VPCLMULQDQ/AVX2 and VPCLMULQDQ/AVX512VL which
use 256-bit vectors, and VPCLMULQDQ/AVX512F/AVX512VL which uses 512-bit
vectors.  To reduce downclocking effects, the implementation with
512-bit vectors isn't used on Intel CPUs 10th generation and older.
@ebiggers ebiggers merged commit 6f4b83f into master Feb 19, 2024
52 checks passed
@ebiggers ebiggers deleted the dev branch February 19, 2024 06:42
@pps83
Copy link
Contributor

pps83 commented Nov 27, 2024

To reduce downclocking effects, the implementation with 512-bit vectors isn't used on Intel CPUs 10th generation and older.

this is CPU on my laptop:

image

it's 11th gen, but the 512-bit code gets disabled (while commit mentions 10th generation). I had to manually comment out the code that disables it. Otherwise, crc32 is much slower for me.

@ebiggers
Copy link
Owner Author

To reduce downclocking effects, the implementation with 512-bit vectors isn't used on Intel CPUs 10th generation and older.

this is CPU on my laptop:

image

it's 11th gen, but the 512-bit code gets disabled (while commit mentions 10th generation). I had to manually comment out the code that disables it. Otherwise, crc32 is much slower for me.

The concern is about applications that just call the CRC functions occasionally, resulting in only occasional use of 512-bit vectors. Older Intel CPUs do not handle that case well as they preemptively downclock and make everything slower. You can of course run a benchmark where you CRC things continually and show that the 512-bit vectors make things faster, but that is not necessarily enough to risk the negative effects in more realistic use cases.

I've intentionally drawn the line after Tiger Lake. The mention of 10th gen in the commit message was misleading, and I fixed the comment in the code months ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants