[Task] hertzbeat metrics collector cluster design

[tomsun28]

Description

why collector cluster?

• cluster means more powerful performance and can support monitoring more
• unified monitoring between different isolated networks, one hertzbeat can monitoring multi isolated networks resource.
• avoid single point of failure

desgin discuss

Task List

No response

[Ceilzcx]

讨论存疑：

1. 采集后数据的存储位置？
   主hz（collector每次采集完需要将数据同步到主hz，需要处理各种丢包等问题，带宽使用高）
   collector模块自己存储（无法实现内网的collector，通过外网查询）
2. hz如何配置collector模块的地址？
   配置文件
   数据库
3. 心跳机制，collector向hz发送心跳还是hz向collector发送心跳？
4. hz如何响应collector模块的上线，注销，连接失败，重连接等问题？

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Discussion in doubt:

1. Where is the collected data stored?
   Main hz (the collector needs to synchronize the data to the main hz after each collection, and needs to deal with various packet loss and other problems, and the bandwidth usage is high)
   The collector module stores itself (the collector of the intranet cannot be realized, and it can be queried through the external network)
2. How does hz configure the address of the collector module?
   configuration file
   database
3. Heartbeat mechanism, does the collector send a heartbeat to hz or does hz send a heartbeat to the collector?
4. How does hz respond to problems such as online, logout, connection failure, and reconnection of the collector module?

[zqr10159]

在监控量不大的情况下，是否保留原来的模式，以简化部署难度？当初就是看着hz不用额外部署agent才用的

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the case of a small amount of monitoring, should the original mode be retained to simplify the difficulty of deployment? At the beginning, it was only used when looking at hz without additional deployment of agent

[tomsun28]

在监控量不大的情况下，是否保留原来的模式，以简化部署难度？当初就是看着hz不用额外部署agent才用的

是的，默认还是会单体 all in one. 设计时也会尽量减少外部组件依赖

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the case of a small amount of monitoring, is the original mode retained to simplify the difficulty of deployment? At the beginning, it was only used when looking at hz without additional deployment of agent

Yes, the default is still all in one. The design will also minimize external component dependencies

[l646505418]

在监控量不大的情况下，是否保留原来的模式，以简化部署难度？当初就是看着hz不用额外部署agent才用的

是的，默认还是会单体 all in one. 设计时也会尽量减少外部组件依赖

如果在主 hz中提供webhook來接收其他collector的信息会不会减少对kafka的依赖

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the case of a small amount of monitoring, is the original mode retained to simplify the difficulty of deployment? At the beginning, it was only used when looking at hz without additional deployment of agent

Yes, the default is still all in one. The design will also minimize external component dependencies

If webhook is provided in the main hz to receive information from other collectors, will it reduce the dependence on Kafka?

[tomsun28]

在监控量不大的情况下，是否保留原来的模式，以简化部署难度？当初就是看着hz不用额外部署agent才用的

是的，默认还是会单体 all in one. 设计时也会尽量减少外部组件依赖

如果在主 hz中提供webhook來接收其他collector的信息会不会减少对kafka的依赖

可以有 把它做成可配置的形式 默认webhook

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

In the case of a small amount of monitoring, is the original mode retained to simplify the difficulty of deployment? At the beginning, it was only used when looking at hz without additional deployment of agent

Yes, the default is still all in one. The design will also minimize external component dependencies

If webhook is provided in the main hz to receive information from other collectors, will it reduce the dependence on kafka

It can be made into a configurable form, the default webhook

[l646505418]

感觉这个更像是提供一个控制中心呢😂

[hertzbeat]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

It feels more like providing a control center😂