Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POWER9 AC922 GPUDirect 39Gb/s only #66

Open
raph38130 opened this issue Mar 31, 2020 · 0 comments
Open

POWER9 AC922 GPUDirect 39Gb/s only #66

raph38130 opened this issue Mar 31, 2020 · 0 comments

Comments

@raph38130
Copy link

Hi,
test system is IBM AC922, 2 x TESLA V100, Connectx-5EN back to back with a x86_64 DELL R740

using RoCEv2 (UC queue pair, WRITE verbs), I have 97Gb/s BW to AC922 CPU memory but only 39 Gb/s to TESLA memory
nv_peer_mem and nv_rsync_mem module are loaded, nvidia-persistenced started
MLNX OFED installed

same issue with either custom code or perftest/ib_write_bw.

observed BW between 2xDELL GPU Quadro P6000 as expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant