Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RDMA] 启用RDMA后client侧解析响应失败 #2814

Open
SimonCqk opened this issue Nov 4, 2024 · 1 comment
Open

[RDMA] 启用RDMA后client侧解析响应失败 #2814

SimonCqk opened this issue Nov 4, 2024 · 1 comment

Comments

@SimonCqk
Copy link

SimonCqk commented Nov 4, 2024

Describe the bug (描述bug)

应用场景可以简化为client发起rpc请求,server收到请求后取回数据,并通过response attachment返回,追踪日志看:

  1. fuse->rpc->worker 完成
  2. worker收到响应->take data-> append to attachment完成
  3. fuse侧hang住,直到报错
E1031 07:32:53.252275 44 input_messenger.cpp:123] Fail to parse response from 33.51.173.89:19893 by baidu_std at client-side
W1031 07:32:53.252285 44 input_messenger.cpp:249] Close Socket{id=7 fd=1053 addr=33.51.173.89:19893:33384} (0x7f1d1765c780): absolutely wrong message
I1031 07:32:53.353038 43 socket.cpp:2606] Checking Socket{id=7 addr=33.51.173.89:19893} (0x7f1d1765c780)
I1031 07:32:53.353356 44 socket.cpp:2666] Revived Socket{id=7 addr=33.51.173.89:19893} (0x7f1d1765c780) (Connectable)

参考#2265 对brpc收发方向的消息都打了日志,可见主要是正常的消息的和server所在进程,send方向的空数据:
image

worker响应的代码很简单,blockletBuffer打印出之后确实是有数据的,没有进一步的排查头绪了。

_cntl->response_attachment().append(blockletBuffer->data(),  blockletBuffer->size());

To Reproduce (复现方法)

Expected behavior (期望行为)

Versions (各种版本)
OS:
Compiler:
brpc: 1.11.0
protobuf:

Additional context/screenshots (更多上下文/截图)

@yanglimingcn
Copy link
Contributor

使用rdma_performance来测试你这个协议的请求和应答试试呢?这样能缩小问题的排查范围,更好的定位问题点。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants