-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(fft): introduce cache efficient bit reverse shuffling #446
Conversation
Summary✅ Passed: 5581 🚧 Skipped
|
Wow that's a huge speedup. Regarding Mac M1s or M2s, the absence of speedup is probably explained by the extremely high memory bandwidth of the Macs. Have to check the numbers as I don't know them from the top of my head. Regarding Plonk, I'm curious about the cases where it's needed, by picking decimation in-time or decimation in frequency you can choose whether you start or end in canonical or bit-reversed domain. This repo has 3 variants out of 4 (no bit-reversed to bit-reversed) https://github.com/kwantam/fffft |
Yep for the M1, that was my conclusion too, plus the cache sizes + cache lines are significantly bigger. For PlonK, most of the time yes, we avoid bit reverse all together, but in one or two spots, we need to convert a polynomial to canonical regular form -- didn't bench the impact yet (probably no more than 5% total perf impact on PlonK prover) but it did stand out on profiling traces. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
I still don't get the arm64 peculiarities and the optimal vs. practical choices for the tile size, but I get the overall logic.
} | ||
|
||
func bitReverseCobra(v []fr.Element) { | ||
switch len(v) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are these empirical results?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in a sense; these methods are just generated with constant sized arrays and offsets for specific sizes.
Below a certain threshold, the naive version performs well, and after that threshold (2**27) I just didn't bother generating the methods since I don't think it's very common to bit reverse 270M+ vectors (but maybe...)
Description
Bit reversal permutation naive implementation does not scale very well due to its memory access patterns and cache associativity issues (since we are accessing elements by strides of powers of 2).
For PlonK prover that needs to do couple of these permutations on potentially large domains (> 2**25) this is very noticeable.
This PR introduces a "COBRA" bit shuffle permutation, derived from:
Larry Carter and Kang Su Gatlin, 1998
https://csaws.cs.technion.ac.il/~itai/Courses/Cache/bit.pdf
permutation in C++11 on the x86-64 architecture
Knauth, Adas, Whitfield, Wang, Ickler, Conrad, Serang, 2017
https://arxiv.org/pdf/1708.01873.pdf
https://github.com/mratsim/constantine/blob/d51699248db04e29c7b1ad97e0bafa1499db00b5/constantine/math/polynomials/fft.nim#L205
by Mamy Ratsimbazafy (@mratsim).
See code for details and benchmark section of this PR for numbers; but in practice, this is efficient for permutations over slices of field element > 2M, and on x86 architectures.
Type of change
How has this been tested?
bitreverese_test.go
in each curve specific package.How has this been benchmarked?
arm64
target we still use the "naive" method.hpc6a
and consumer gradeamd
chip are good; up to 70% faster for large sizes.On the
![BenchmarkBitReverse_hpc](https://private-user-images.githubusercontent.com/11349757/268088477-45d76cbc-4196-49f6-8caa-4892f6e778f6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ0MjIxMjksIm5iZiI6MTcyNDQyMTgyOSwicGF0aCI6Ii8xMTM0OTc1Ny8yNjgwODg0NzctNDVkNzZjYmMtNDE5Ni00OWY2LThjYWEtNDg5MmY2ZTc3OGY2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDE0MDM0OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThiYTFiYTRjMzQzN2I1ZDI3MjMzOGNmN2EwZWY5NjhiNDFkNTk3ZmJlMzMzZTBmZjI1Y2MxODJmNTQwMDczMWEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.1sOD9JOMYPks7cK2qJO4BSWVchY5cknwAn_sIHP5LLE)
hpc6a
for sizes going up to 2^28:Similarly on my amd-based desktop:
On the M1 macbook:
Checklist:
golangci-lint
does not output errors locally