[Lang] [test] Copy-free interaction between Taichi and PaddlePaddle

[0xzhang]

Related issue = hackathon

paddle.fluid.core.Tensor()._ptr() is only avaliable on develop branch. I encounter 2 problems with develop package in build process.

1. I started with CUDA-configured WSL as my development environment. After installing the develop version package on CPU and CUDA. import paddle causes an Illegal Instruction error and exits the Python environment. So I gave up using WSL.
2. Then I developed directly on Windows . Firstly, I used the develop version of paddlepaddle-gpu. Although the actual file is less than 2GB. When I import paddle I got error Windows FATAL Exception: access violation. A .pyd file is too large for the page to load. My computer has 16GB of memory, and rebooting still doesn't fix the problem.

Therefore, I tried the CPU version of paddlepaddle-0.0.0 and it worked.
Refer to the implementation of PyTorch in Taichi. My implementation should support the interactions in CPU-CPU/CPU-GPU/GPU-CPU/GPU-GPU between Taichi and Paddle. But for now, on my development enviroment only CPU-CPU test could be verified.

[netlify]

✅ Deploy Preview for docsite-preview ready!

Name	Link
🔨 Latest commit	34b1fef
🔍 Latest deploy log	https://app.netlify.com/sites/docsite-preview/deploys/627229bc6b13510008edda0f
😎 Deploy Preview	https://deploy-preview-4886--docsite-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[0xzhang]

I use commands below to run test.

$Env:TI_ENABLE_PADDLE=1
python .\setup.py install
python .\tests\run_tests.py -v -k test_paddle_io -a cpu -t 1

[jim19930609]

just one more concern, all other parts LGTM!

[jim19930609]

For the concern regarding host_v's lifetime, tested with the following codes:

import sys 
import paddle

def callback(a):
    def foo():
        print("inside callback: ", sys.getrefcount(a))
        y = a + 1 
    return foo 

callbacks = []
x = paddle.to_tensor([5])

print("Init ref_count for x: ", sys.getrefcount(x))

callbacks.append(callback(x))
print("after appension: ", sys.getrefcount(x))

del x
callbacks[0]()

Result:

Seems like passing object as function argument does increase its ref count, therefore should be safe.

[0xzhang]

Task List

Implementation

python\taichi\lang\util.py

• has_paddle()
• to_paddle_type()
• to_taichi_type()

python\taichi\lang\kernel_impl.py

• get_paddle_callbacks()
• get_function_body()
• match_ext_arr()

python\taichi\lang\matrix.py

• to_paddle()

python\taichi\lang\field.py

• ScalarField.to_paddle()
• Field.to_paddle(): raise NotImplementedError like Field.to_torch()
• Field.from_paddle(): from_numpy(), raise NotImplementedError like Field.from_torch()

python\taichi\lang\mesh.py

• MeshElementField.to_paddle()
• MeshElementField.from_paddle()

python\taichi\lang\struct.py

• StructField.to_paddle()
• StructField.from_paddle()

Test

tests\python\test_paddle_io.py

• test_io_devices()
• test_io()
• test_io_2d()
• test_io_3d()
• test_io_simple()
• test_io_zeros()
• test_io_struct()
• test_fused_kernels()
• test_devices()
• test_shape_matrix()
• test_shape_vector()
• test_paddle_zero()
• test_paddle_view()

tests\python\test_get_external_tensor_shape.py

• test_get_external_tensor_shape_access_paddle()

tests\python\test_f16.py

• test_to_paddle()
• test_from_paddle()

tests\python\test_api.py

• user_api[ti.Field]: 'from_paddle', 'to_paddle'
• user_api[ti.MatrixField]: 'from_paddle', 'to_paddle'
• user_api[ti.ScalarField]: 'from_paddle', 'to_paddle'
• user_api[ti.StructField]: 'from_paddle', 'to_paddle'

Doc

docs\lang\articles\basic\external.md

• from_paddle()/to_paddle()

docs\lang\articles\misc\global_settings.md

• export TI_ENABLE_PADDLE=1

[ailzhang]

Hey @0xzhang , great work! Thanks a lot for implementing this feature!
Btw I'm requesting reviews here since the tests are not exercised in CI due to missing paddle package. I want to make sure we don't accidentally merge this before it's tested ;) Would you mind adding paddle into requirements_dev.txt so that we have that in the CI? Thanks a lot!

2022-04-29T13:13:18.3060568Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_to_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3068611Z tests/python/test_f16.py::test_to_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3073852Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_to_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3081612Z tests/python/test_f16.py::test_from_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3087680Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_from_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3095160Z tests/python/test_f16.py::test_from_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3101501Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_from_paddle[arch=cuda-1] 

[0xzhang]

Hey @0xzhang , great work! Thanks a lot for implementing this feature! Btw I'm requesting reviews here since the tests are not exercised in CI due to missing paddle package. I want to make sure we don't accidentally merge this before it's tested ;) Would you mind adding paddle into requirements_dev.txt so that we have that in the CI? Thanks a lot!

2022-04-29T13:13:18.3060568Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_to_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3068611Z tests/python/test_f16.py::test_to_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3073852Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_to_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3081612Z tests/python/test_f16.py::test_from_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3087680Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_from_paddle[arch=cuda-0] 
2022-04-29T13:13:18.3095160Z tests/python/test_f16.py::test_from_paddle[arch=cuda-1] 
2022-04-29T13:13:18.3101501Z [gw3] [ 39%] SKIPPED tests/python/test_f16.py::test_from_paddle[arch=cuda-1] 

Thanks for your review! I quite agree with your request. I also think untested integration is inappropriate. I was concerned that setting paddles in requirements_dev.txt would require the developer to install a larger installation package. The package of CUDA version on develop branch is almost 461MB space needed.

How the torch's installation be seted? I can't find where torch's installation is set, but torch was tested in CI.

And the pip install command of paddlepaddle-gpu-0.0.0 is below. How should I add it in requirements_dev.txt. I'm not sure about this.

python -m pip install paddlepaddle-gpu==0.0.0.post112 -f https://www.paddlepaddle.org.cn/whl/windows/gpu/develop.html

[0xzhang]

I found the setup in .github/workflows/scripts. I'll give it a try

[0xzhang]

Paddle's develop package

OS	Platform	Python
Windows	CPU	3.6, 3.7, 3.8, 3.9, 3.10
	CUDA	3.6, 3.7, 3.8, 3.9, 3.10
macOS	CPU	3.8/3.9 macos-10.14 x86_64; 3.6/3.7 macos-10.6 intel; 3.7 macos-10.14 intel
	CUDA	❌
Linux	CPU	3.6, 3.7, 3.8, 3.9
	CUDA	3.6, 3.7, 3.8, 3.9

CI Machine

Machine	OS	Platform	Python	Result
linux (CPU) (ubuntu-latest, py39, ON, cpu,cc)	✅	CPU	✅	✅
linux (CPU) (ubuntu-latest, py310, ON, cpu,cc)	✅	CPU	❌	❌
macos (CPU) (macos-10.15, 3.7, OFF, ON, cpu)	❌	CPU	✅	❌
GPU	✅	GPU	✅	Illegal instruction
Windows	✅	CPU/GPU	✅	PermissionDenied ?
Apple M1 (macos-latest, 3.8)	❌	CPU/GPU	✅	❌

Due to the limitations of the develop package, only linux (CPU) (ubuntu-latest, py39, ON, cpu,cc) can be tested at present. And Windows machine suddenly cannot complete the Cleanup process.

I'm not sure about error Illegal instruction. This error has also appeared on my WSL environment.

The current situation is a bit tough as it always seems to fail to test the GPU version 😢.

[0xzhang]

Good news. Windows machine work again.
Bad news. Windows machine failed again and again 😢. I create a new issue(#4895) about this problem.

Because of the usage of paddle.Tensor.fluid.core._ptr(). I should dependent on Paddle's develop branch.
According to the status of official installation packages on develop branch. I modifiy the test process in win_test.ps1 and unix_test.sh.

1. It is not compatible with macOS versions. Therefor testcases on macOS will not be tested for now.
2. GPU version's package on Linux will get an error Illegal Instruction. I encounter this problem on my WSL as well. I think the problem is most likely from the installation package. So CUDA version on Linux also will not be tested for now.

Overall, only two machines are currently available for testing. And Windows machine can test both on CPU and GPU. Hope the Windows machine could work fine! 🙏

And run_tests.py doesn't always work fine on my Windows. So I couldn't run the test for Paddle on GPU. I create a new issue(#4897) about this problem.

CI Machine

Machine	OS	Platform	Python	Testable
linux (CPU) (ubuntu-latest, py39, ON, cpu,cc)	✅	CPU	✅	✅
linux (CPU) (ubuntu-latest, py310, ON, cpu,cc)	✅	CPU	❌	❌
macos (CPU) (macos-10.15, 3.7, OFF, ON, cpu)	❌	CPU	✅	❌
GPU	✅	GPU	✅	❌
Windows	✅	CPU/GPU	✅	✅
Apple M1 (macos-latest, 3.8)	❌	CPU/GPU	✅	❌

[ailzhang]

Finally windows passed! Thanks a ton for your contribution!