Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream load 用json导入时发生内存泄漏 #4962

Open
PeterGZW opened this issue Nov 26, 2020 · 4 comments
Open

stream load 用json导入时发生内存泄漏 #4962

PeterGZW opened this issue Nov 26, 2020 · 4 comments
Labels
area/load Issues or PRs related to all kinds of load Memory Leak

Comments

@PeterGZW
Copy link

服务器配置 1fe3be , 8核16G ,内存都是默认参数

1.每次查询mysql 1w条记录 转成json文件
2.通过curl 导入进doris。平均doris耗时1s消费一批json。 (单个json文件大小在20-40M。)
3.稳定跑1h左右,3台内存有一台会达到90%以上,然后突然无响应。过一段时间后进程被系统kill,发生了OOM。

版本doris 0.13.11-release

@Dam1029
Copy link
Contributor

Dam1029 commented Dec 9, 2020

顶一下,这边也遇到了类似的问题,我们使用flink封装了stream load,并将json文件实时发送到doris后,be的内存值doris_be_memory_allocated_bytes一直上涨到OOM,即便任务停止,内存监控值也下不来(没有释放),重启才能恢复

@imay imay added area/load Issues or PRs related to all kinds of load Memory Leak labels Dec 10, 2020
@stdpain
Copy link
Contributor

stdpain commented Dec 12, 2020

issues #5069
pull request #5073

@LKK1535977706
Copy link

@Dam1029 请问这个问题解决了吗,我这边也遇到同样的问题了

@dataroaring
Copy link
Contributor

Which version do you use?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/load Issues or PRs related to all kinds of load Memory Leak
Projects
None yet
Development

No branches or pull requests

6 participants