refine_error_info_with_diff_device_input_and_fix_typo

[clackhan]

当遇到 多个 input placement 或则不一致时，报错信息对齐pytorch

[clackhan]

这里是要保留graph信息还是完全和eager对齐？需要 @strint 确认一下

[strint]

这里还提供了graph op信息、提供了修改建议。

这样部分对齐就挺好。

[hjchen2]

close https://github.com/Oneflow-Inc/OneTeam/issues/1332

[github-actions]

Speed stats:

[github-actions]

CI failed when running job: Build cu102_xla. PR label automerge has been removed

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8054/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.7ms (= 12969.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 149.1ms (= 14905.1ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.15 (= 149.1ms / 129.7ms)

OneFlow resnet50 time: 78.5ms (= 7849.5ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.3ms (= 8333.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.06 (= 83.3ms / 78.5ms)

OneFlow resnet50 time: 54.1ms (= 10821.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.3ms (= 11859.0ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.10 (= 59.3ms / 54.1ms)

OneFlow resnet50 time: 42.1ms (= 8414.4ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 40.8ms (= 8155.0ms / 200, input_shape=[2, 3, 224, 224])
❌ Relative speed: 0.97 (= 40.8ms / 42.1ms)

OneFlow resnet50 time: 38.7ms (= 7732.8ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 37.1ms (= 7426.7ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.96 (= 37.1ms / 38.7ms)

OneFlow swin dataloader time: 0.253s (= 50.601s / 200, num_workers=1)
PyTorch swin dataloader time: 0.151s (= 30.174s / 200, num_workers=1)
Relative speed: 0.596 (= 0.151s / 0.253s)

OneFlow swin dataloader time: 0.065s (= 13.014s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.406s / 200, num_workers=4)
Relative speed: 0.646 (= 0.042s / 0.065s)

OneFlow swin dataloader time: 0.037s (= 7.379s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.434s / 200, num_workers=8)
Relative speed: 0.601 (= 0.022s / 0.037s)

❌ OneFlow resnet50 time: 145.4ms (= 14536.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 166.0ms (= 16597.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.14 (= 166.0ms / 145.4ms)

OneFlow resnet50 time: 97.1ms (= 9707.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 113.1ms (= 11313.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 113.1ms / 97.1ms)

OneFlow resnet50 time: 76.3ms (= 15257.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.6ms (= 17718.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.16 (= 88.6ms / 76.3ms)

OneFlow resnet50 time: 63.8ms (= 12752.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 87.1ms (= 17418.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.37 (= 87.1ms / 63.8ms)

OneFlow resnet50 time: 56.8ms (= 11367.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 79.2ms (= 15830.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.39 (= 79.2ms / 56.8ms)