Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add FFT operators #10027

Merged
merged 190 commits into from
May 17, 2023
Merged

Add FFT operators #10027

merged 190 commits into from
May 17, 2023

Conversation

MarioLulab
Copy link
Contributor

@MarioLulab MarioLulab commented Mar 23, 2023 •
edited
Loading

Original requirements

Some AI for Science related models(FNO, AFNO, PINO) use FFT operations. So we need to support these.

Main Works

related research: https://github.com/Oneflow-Inc/OneTeam/issues/1927

  1. use complex64, complex128 data type
  2. develop some basic operators which support fft.fft, fft.ifft, fft.fft2, fft.ifft2, fft.rfft, fft.irfft, etc. :
    • fft_c2c
    • fft_r2c
    • fft_c2r
  3. support FFT functional interface:
    • fft.fft (CPU, CUDA Done)
    • fft.ifft (CPU, CUDA Done)
    • fft.fft2 (CPU, CUDA Done)
    • fft.ifft2 (CPU, CUDA Done)
    • fft.fftn (CPU, CUDA Done)
    • fft.ifftn (CPU, CUDA Done)
    • fft.rfft (CPU, CUDA Done)
    • fft.irfft (CPU, CUDA Done)
    • fft.rfft2 (CPU, CUDA Done)
    • fft.irfft2 (CPU, CUDA Done)
    • fft.rfftn (CPU, CUDA Done)
    • fft.irfftn (CPU, CUDA Done)
    • fft.hfft (CPU, CUDA Done)
    • fft.ihfft (CPU, CUDA Done)
    • fft.hfft2 (CPU, CUDA Done)
    • fft.ihfft2 (CPU, CUDA Done)
    • fft.hfftn (CPU, CUDA Done)
    • fft.ihfftn (CPU, CUDA Done)
  4. extend autotest module for complex tensor by adding include_complex option. If set include_complex = True, random_tensor will generate complex tensor with prob of 0.5. In the future, we can use this option to reuse the real tensor test cases to test the complex tensor

Future Works

  1. use LRU Caches to speed up cufft running time

levi131 and others added 30 commits March 7, 2023 16:29
init commit for add cfloat and cdouble dtype
0281680
code polish
9b915e7
@levi131
Merge branch 'master' into lml/complex
faf2076
Merge remote-tracking branch 'upstream/master' into lml/complex
ca41336
save work
c602d02
Merge remote-tracking branch 'upstream/master' into lml/complex
661df1e
save work
9d5b054
merge upstream master
9466b8e
update work
3896bd1
add oneflow.complex64 and oneflow.complex128
797d483
fix bug for complex128
e25f0a4
fix bug for get item for scalar complex tensor
d4295de
update format
34cb7ff
update work state
d3287a8
update format
6b6e884
rm some useless code
f66c2d4
Merge remote-tracking branch 'upstream/master' into lml/complex
07df8de
fix format
ffbca31
save work state
46f7d3c
add complex64 and complex128 for cpu primitive
e17b942
refine format
ea74be6
skip test tensor cuda on CPU only CI
cc59ba2
add import os
d22f576
@MarioLulab
Add c2c, r2c, c2r Op.
fbb4c3e
rm default value for ComplexDoubleAttr
2c6af0f
Merge remote-tracking branch 'upstream/master' into lml/complex_tenso…
95c418c 
…r_and_complex_attr
rm unused construct function for calss scalar
02d0656
refine class Scalar and add transform for ComplexDoubleAttr
795ae7d
refine format
b00334d
add set active_tag
fac2e1f
@MarioLulab MarioLulab requested a review from oneflow-ci-bot May 10, 2023 02:41
@MarioLulab MarioLulab requested review from oneflow-ci-bot and removed request for oneflow-ci-bot May 15, 2023 03:53
levi131
levi131 approved these changes May 16, 2023
View reviewed changes
Copy link
Contributor

@levi131 levi131 left a comment •
edited by MarioLulab
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

目前多卡测试发现随机挂问题，测试中采用skip_unless_1n1d策略。多卡随机挂问题等定位到之后另外提pr修复。

daquexian
daquexian approved these changes May 17, 2023
View reviewed changes
oneflow/user/kernels/fft_kernels.cpp Outdated Show resolved Hide resolved
mergify bot and others added 2 commits May 17, 2023 02:49
@mergify
Merge branch 'master' into luqi/dev_fft_based_complex
35feec6
@MarioLulab @daquexian
Update oneflow/user/kernels/fft_kernels.cpp
d368edc 
use pass by value to optimize ShapeView passing

Co-authored-by: daquexian <daquexian566@gmail.com>
@levi131 levi131 enabled auto-merge (squash) May 17, 2023 03:03
@levi131 levi131 merged commit 85a4b8d into Oneflow-Inc:master May 17, 2023
This was referenced May 19, 2023
Merged
Merged
L-Xiafeng pushed a commit that referenced this pull request Jun 26, 2023
@MarioLulab
Applying autotest module on existing complex operators (#10284)
fb423fa 
### Original requirements
**Autotest**: We found that the previous testing of operators supporting
complex tensor was not complete. We decided to reuse the real tensor
operator tests to ensure completeness. Since complex tensor tests are
supported in the `autotest` module from pr
(#10027) , in this pr we
applied the autotest module to the tests of complex tensor operators
already available in Oneflow

**Fix**: In addition, the autograd rules for some previous operators of
complex numbers might not conform to the convention of ["Conjugate
Wirtinger
Derivative"](https://en.wikipedia.org/wiki/Wirtinger_derivatives). We
have fixed these bugs in this pr at the same time.

#### Main Works
**Applying `autotest` module on existing operators that have already
support complex tensor:**

`Complex and Real Behave the Same Way`: means we don't need to add
conjugate operation in op grad. Because regardless of whether the input
data type involved in the operation is real or complex, the gradient
result using the winterger derivative is the same as the real derivative
rule,

`Grad Not Supported in OF`: means the grad of this op is not supported
in oneflow

- broadcast_elementwise_binary
| Op | complex type | Backend | Using autotest | conjugate Wirtinger
derivative |
  |:-----:|:-------------:|:------:|:------:|:------:|
| Add | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the
Same Way |
  | Mul |  cp64, cp128   |  CPU, CUDA | DONE | DONE |
| Sub | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the
Same Way |
| Equal | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the
Same Way |
| NotEqual | cp64, cp128 | CPU, CUDA | DONE | Grad Not Supported in OF |


- broadcast_elementwise_unary
| Op | complex type | Backend | Using autotest | conjugate Wirtinger
derivative |
  |:----------:|:-------------:|:------:|:------:|:------:|
| Cast | cp64, cp128 | CPU, CUDA | DONE | Complex and Real Behave the
Same Way |


- other exisiting operations
| Op | complex type | Backend | Using autotest | conjugate Wirtinger
derivative |
  |:----------:|:-------------:|:------:|:------:|:------:|
| constant_pad | cp64, cp128 | CPU, CUDA | Done | Complex and Real
Behave the Same Way |
| reduce_sum | cp64, cp128 | CPU, CUDA | TO-DO | Complex and Real Behave
the Same Way |


## 注意：
复数基础设施建设系列 pr:
1. 使用 primitive 来实现 conj, real 等常见复数算子：
#10281
2. 将现有支持复数数据类型的算子测例迁移到 autotest
模块中，以使复数算子复用实数算子的测试用例：#10284
3. 继续拓展支持复数数据类型的算子，比如 matmul, sqrt, div
等：#10269
依赖关系：
本 pr 基于：[pr1](https://github.com/Oneflow-Inc/oneflow/pull/10281)，需要在
merge [pr1](#10281) 后，再 Merge
本 pr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@levi131 levi131 levi131 approved these changes

@daquexian daquexian daquexian approved these changes

@BBuf BBuf Awaiting requested review from BBuf

@hjchen2 hjchen2 Awaiting requested review from hjchen2 hjchen2 is a code owner

@jackalcooper jackalcooper Awaiting requested review from jackalcooper jackalcooper is a code owner

@liujuncheng liujuncheng Awaiting requested review from liujuncheng liujuncheng is a code owner

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
automerge feature op
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@MarioLulab @daquexian @levi131