-
Notifications
You must be signed in to change notification settings - Fork 685
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use primitive to implement "conj_physical", "real" and "imag" op #10281
Merged
levi131
merged 7 commits into
Oneflow-Inc:master
from
MarioLulab:luqi/add_conj_primitive
Jun 5, 2023
Merged
use primitive to implement "conj_physical", "real" and "imag" op #10281
levi131
merged 7 commits into
Oneflow-Inc:master
from
MarioLulab:luqi/add_conj_primitive
Jun 5, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MarioLulab
changed the title
use primitive to implement "conj_physical" op
use primitive to implement "conj_physical", "real" and "imag" op
May 22, 2023
levi131
reviewed
May 23, 2023
MarioLulab
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
May 23, 2023 03:47
levi131
approved these changes
May 23, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This was referenced May 30, 2023
L-Xiafeng
approved these changes
Jun 5, 2023
L-Xiafeng
pushed a commit
that referenced
this pull request
Jun 26, 2023
### Original requirements **Autotest**: We found that the previous testing of operators supporting complex tensor was not complete. We decided to reuse the real tensor operator tests to ensure completeness. Since complex tensor tests are supported in the `autotest` module from pr (#10027) , in this pr we applied the autotest module to the tests of complex tensor operators already available in Oneflow **Fix**: In addition, the autograd rules for some previous operators of complex numbers might not conform to the convention of ["Conjugate Wirtinger Derivative"](https://en.wikipedia.org/wiki/Wirtinger_derivatives). We have fixed these bugs in this pr at the same time. #### Main Works **Applying `autotest` module on existing operators that have already support complex tensor:** `Complex and Real Behave the Same Way`: means we don't need to add conjugate operation in op grad. Because regardless of whether the input data type involved in the operation is real or complex, the gradient result using the winterger derivative is the same as the real derivative rule, `Grad Not Supported in OF`: means the grad of this op is not supported in oneflow - broadcast_elementwise_binary | Op | complex type | Backend | Using autotest | conjugate Wirtinger derivative | |:-----:|:-------------:|:------:|:------:|:------:| | Add | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the Same Way | | Mul | cp64, cp128 | CPU, CUDA | DONE | DONE | | Sub | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the Same Way | | Equal | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the Same Way | | NotEqual | cp64, cp128 | CPU, CUDA | DONE | Grad Not Supported in OF | - broadcast_elementwise_unary | Op | complex type | Backend | Using autotest | conjugate Wirtinger derivative | |:----------:|:-------------:|:------:|:------:|:------:| | Cast | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the Same Way | - other exisiting operations | Op | complex type | Backend | Using autotest | conjugate Wirtinger derivative | |:----------:|:-------------:|:------:|:------:|:------:| | constant_pad | cp64, cp128 | CPU, CUDA | Done | Complex and Real Behave the Same Way | | reduce_sum | cp64, cp128 | CPU, CUDA | TO-DO | Complex and Real Behave the Same Way | ## 注意: 复数基础设施建设系列 pr: 1. 使用 primitive 来实现 conj, real 等常见复数算子: #10281 2. 将现有支持复数数据类型的算子测例迁移到 autotest 模块中,以使复数算子复用实数算子的测试用例:#10284 3. 继续拓展支持复数数据类型的算子,比如 matmul, sqrt, div 等:#10269 依赖关系: 本 pr 基于:[pr1](https://github.com/Oneflow-Inc/oneflow/pull/10281),需要在 merge [pr1](#10281) 后,再 Merge 本 pr
L-Xiafeng
added a commit
that referenced
this pull request
Jun 30, 2023
…10269) ### Original requirements AI for Science models FNO use a series of operators such as sqrt, mm, div, etc. for complex tensor. So we need to support complex tensor for these ops. #### Main Works **Support existing ops for complex tensor in this pr as below:** - broadcast_elementwise_binary | Op | complex type | Backend | Progress | |:-----:|:-------------:|:------:|:------:| | scalar_div | cp64, cp128 | CPU, CUDA | Done | | broadcast_div | cp64, cp128 | CPU, CUDA | Done | - broadcast_elementwise_unary | Op | complex type | Backend | Progress | |----------|:-------------:|:------:|:------:| | sqrt | cp64, cp128 | CPU, CUDA | Done | | negative | cp64, cp128 | CPU, CUDA | Done | - other exisiting operations | Op | complex type | Backend | Progress | |----------|:-------------:|:------:|:------:| | matmul | cp64, cp128 | CPU, CUDA | Done | | batch_matmul | cp64, cp128 | CPU, CUDA | Done | | broadcast_matmul | cp64, cp128 | CPU, CUDA | Done | | matrix_vector_product | cp64, cp128 | CPU, CUDA | Done | | reduce_sum_like | cp64, cp128 | CPU, CUDA | Done | ## 注意: 复数基础设施建设系列 pr: 1. 使用 primitive 来实现 conj, real 等常见复数算子: #10281 2. 将现有支持复数数据类型的算子测例迁移到 autotest 模块中,以使复数算子复用实数算子的测试用例:#10284 3. 继续拓展支持复数数据类型的算子,比如 matmul, sqrt, div 等:#10269 依赖关系: 本 pr 基于:[pr1](#10281) 和 [pr2](#10284) ,需要在 merge [pr1](#10281) 和 [pr2](#10284) 后,再 Merge 本 pr --------- Co-authored-by: levi131 <limaolin@oneflow.org> Co-authored-by: levi131 <83750468+levi131@users.noreply.github.com> Co-authored-by: lijunlin <70465472+L-Xiafeng@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original requirements
现有的 conj kernel, real kernel, imag kernel 使用 KernelUtil 来进行组织,在 kernel 层面复数数据类型和实数数据类型的调用 conj 无法实现统一。故移除原有 KernelUtil 的实现,使用 ElementwiseUnary 的 primitive 进行实现
kConj
这一 UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 conj,并移除原有 conj 的 KernelUtilkReal
和kRealGrad
UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 real 和 real_grad,并移除原有 real 和 real_grad 的 KernelUtilkImag
和kImagGrad
UnaryOp,使用 ElementwiseUnary 的 Primitive 实现 imag 和 imag_grad,并移除原有 imag 和 imag_grad 的 KernelUtil注意:
复数基础设施建设系列 pr:
依赖关系:
本 pr 基于:pr2 和 pr3 的基础,请优先 merge 此 pr