-
Notifications
You must be signed in to change notification settings - Fork 11.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RIP-68] RocketMQ ACL 2.0 #7560
Comments
Wow~ looking forward to it. |
Regarding the ACL storage component, i have a little question, especially as the Controller and Dledger modes are increasingly becoming the top choices. Why not place ACL information in Namesrv? This could potentially simplify Broker scaling, and proxy will get the metadata(include acl) from namesrv. |
@echooymxq |
@dingshuangxi888 The namesrv not stateless actually. such as in controller mode, maybe the controller mode is special case. but there's some kvConfig.json about orderConf, and rocketmq-mqtt also rely on it. |
Interesting, looking forward to contribute! |
@dingshuangxi888 can you assign somthing to me to contribute on this feature? |
when this feature release? |
If you use the proxy cluster mode and enable authentication, each request is authenticated twice (proxy and broker)?@dingshuangxi888 |
The configuration for the proxy and the broker is different; only the proxy will be enabled if it's in local mode. |
Are there any plans to reverse this ability into RocketMQ 4.x? |
Status
Background & Motivation
What do we need to do
Why should we do that
Are there any problems with our current project?
Currently, the RocketMQ ACL design has the following limits:
当前RocketMQ ACL的设计存在以下一些限制:
What can we benefit from proposed changes?
Goals
To address the aforementioned issues, the key objectives of RocketMQ ACL 2.0 optimization are as follows:
为了解决以上问题,RocketMQ ACL 2.0的主要目标如下进行优化:
Non-Goals
Changes
Concepts
RocketMQ ACL 2.0 primarily adopts the Attribute-Based Access Control (ABAC) model in its permission system. The main concepts are as follows:
关于RocketMQ ACL 2.0主要采用权限体系中的ABAC模型,主要的概念如下:
Architecture
Original
In the original architecture of RocketMQ, the Broker is responsible for managing the storage of users and permissions, as well as implementing the logic for authentication and authorization. Whether it is handling interfaces such as message sending and receiving, cluster management, or data querying, all must go through ACL's authentication and authorization logic to ensure the legitimacy of all requests.
在RocketMQ的原始架构中,Broker负责管理用户和权限的存储,并实现认证和授权的逻辑。无论是处理消息的发送和接收、集群管理还是数据查询等接口,都必须经过ACL的认证和授权逻辑,以确保所有请求的合法性。
With Proxy
In the RocketMQ architecture with Proxy, the sending and consumption of messages are delegated to the Proxy component, hence authentication and authorization are carried out within the Proxy component. However, the cluster management-related interfaces are still handled by the Broker, requiring authentication and authorization from the Broker. As for the storage of user and permission information, due to the fact that only the Broker is a stateful application, it is still stored on the Broker, and the Broker provides corresponding interfaces for the Proxy to query and perform subsequent authentication logic.
在具备Proxy的RocketMQ架构中,消息的发送和消费由Proxy组件进行代理,因此在Proxy组件中进行认证和授权。然而,集群管理的相关接口仍由Broker负责处理,因此这些接口需要经过Broker的认证和授权。关于用户和权限的存储方面,由于只有Broker是有状态应用,因此仍将其存储在Broker上,并由Broker提供相关接口供Proxy查询,以执行后续的鉴权逻辑。
Domain Model
Date Type
Principal
Resource
Action
Interface Design/Change
The following are the API definitions for RocketMQ ACL 2.0:
以下是RocketMQ ACL 2.0的API定义:
User management
Permission management
Implementation
Mapping of API, Resources, and Actions
为了方便获取每个API请求对应的操作和资源信息进行鉴权,需要进行以下改造:
To facilitate the authorization process by obtaining information about the operation and resource corresponding to each API request, the following modifications need to be made:
Remoting API
For Remoting-related interfaces, as there are a large number of interfaces and each interface has a different resource name definition, to facilitate obtaining resource definitions, annotations can be used to label each API's RequestHeader, indicating which fields are resources and their IDs. This way, it will be easier to retrieve relevant information during the authorization process. The specific steps are as follows:
![image](https://private-user-images.githubusercontent.com/5010921/283077766-1869726c-b0bf-4a43-b648-285b9c58e2e2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ0NzIyNzQsIm5iZiI6MTcyNDQ3MTk3NCwicGF0aCI6Ii81MDEwOTIxLzI4MzA3Nzc2Ni0xODY5NzI2Yy1iMGJmLTRhNDMtYjY0OC0yODViOWM1OGUyZTIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDgyNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA4MjRUMDM1OTM0WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NGFkNjU2Y2E4NWQ3ZDM2MzU5YjBhNTg0YjVlY2M0ZDMzMTE0ZDJlMDU5NmM1MDY5YzBhZmVlY2ZhYzgzN2RiOCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.iqI2FraGek-BOhOqc7LCCHVG_TJeLIuzblRK2Fv0YcU)
对于Remoting相关接口,由于接口数量较多且每个接口定义的资源名称不同,为了方便获取资源定义,可以通过注解的方式为每个API的RequestHeader进行打标,标识字段中哪些是资源以及资源的ID。这样可以在鉴权过程中更方便地获取相关信息。具体如下:
Considering the frequent invocation and high performance requirements of the message sending and receiving interfaces, a hard-coded implementation approach is still used to ensure performance.
考虑到消息收发相关接口调用频繁且对性能要求较高,仍然采用硬编码的方式实现消息收发接口,以保证性能。
gRPC API
For gRPC-related interfaces, it is difficult to implement them using abstract classes and annotations because the protocol is defined and generated through proto files. Currently, it mainly involves PUB and SUB types of interfaces, so it can be implemented through hardcoding. Please refer to:
对于gRPC相关的接口,由于协议是通过proto文件进行定义和生成的,难以通过抽象类和注解的方式实现。目前主要涉及PUB和SUB类型的接口,因此可以先通过硬编码的方式实现,具体参考:
org.apache.rocketmq.acl.plain.PlainAccessResource#parse(com.google.protobuf.GeneratedMessageV3, org.apache.rocketmq.acl.common.AuthenticationHeader)
Storage model
The storage of ACL is primarily kept locally in the Broker, and there are two main options: file storage and RocksDB-based storage. In this case, RocksDB is used for storage, while file storage is provided as an example for reference purposes.
关于ACL的存储主要在Broker本地进行,有两种主要方案可供选择,一种是文件存储,另一种是基于RocksDB的存储。本次存储采用RocksDB,而文件存储仅作为示例供参考。
4.2.1 File storage
User file storage
Permissions file storage
RocksDB-based storage
In RocksDB, the data storage method is similar to file storage, but it uses the principal as the primary key for storage. Compared to file storage, using RocksDB may be simpler in terms of management.
在RocksDB中,存储的数据方式与文件存储相似,但是使用主体(principal)作为主键进行存储。相对于文件存储,使用RocksDB的方式可能在管理层面更加简单。
ACL evaluation
Compatibility, Deprecation, and Migration Plan
Are backward and forward compatibility taken into consideration?
Yes. In order to facilitate user migration, relevant compatibility handling needs to be done. Here are several modifications for compatibility:
是的。为了方便用户进行迁移,需要进行相关的兼容性处理。以下是针对兼容的几点改造:
Are there deprecated APIs?
Yes, the APIs related to ACL 1.0 will be removed. If you want to modify the permissions of ACL 1.0, you can achieve it by directly modifying the file.
是的,ACL1.0相关的API会被移除。如果你想要修改ACL1.0的权限,你可以通过直接修改文件的方式来实现。
How do we do migration?
Implementation Outline
The feature is currently under development, and I will complete it as soon as possible and submit it to the community.
该功能正在研发中,我会尽快完成并提交到社区。
The text was updated successfully, but these errors were encountered: