-
Notifications
You must be signed in to change notification settings - Fork 99
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Operation Question] How to separate truncation and matmul operations #672
Comments
Hi @wilyub I don't think you could seperate matmul and truncate in Python bindings of SPU. You may have to dive into kernels of SPU runtime. |
Thanks for the advice. Any idea on where I could start looking? I'm not too familiar with the inner workings of SPU as I've only used it from python. Thanks! |
I would suggest having a look at https://github.com/secretflow/spu/blob/main/REPO_LAYOUT.md. |
@wilyub To bench the matmul only without the truncation, we can use "integer" matrix instead of the floating point matrix. |
I tested out the integer matmul and it works (no truncation shows up in the log). I was wondering what function should I call to get trunc_a to show up in the log by itself? I tried jnp.trunc and didn't get that result. Thanks! Also any update on why the nonlinearities seem so cheap compared to matmuls? |
Issue Type
Support
Modules Involved
MPC protocol, SPU runtime
Have you reproduced the bug with SPU HEAD?
Yes
Have you searched existing issues?
Yes
SPU Version
spu 0.9.0b1
OS Platform and Distribution
Linux Ubuntu 22.04
Python Version
3.11.5
Compiler Version
No response
Current Behavior?
I'd like to do some latency testing for multiplication and truncation operations. Here is a short overview of how it would look:
Input x and weight y are represented using 16 bits fixed point each. Therefore, when you do a multiplication, you expect them to have 32 bits representation in fixed point. However, suppose you have a ring size of 64 bits. You could then do another 32 bits * 32 bits multiplication before overflowing your ring size of 64 bits.
How can I do this in secretflow? In particular, I want to be able to control the fixed point representation size of each input to the matrix multiplication, and only truncate after specific multiplications (not every multiplication).
Thank you for your help!
Standalone code to reproduce the issue
Relevant log output
The text was updated successfully, but these errors were encountered: