-
Notifications
You must be signed in to change notification settings - Fork 454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: cluster key rate limit #1002
Conversation
148a8cb
to
83cb0ac
Compare
156e1db
to
6c07b58
Compare
@johnlanni 代码已调整,麻烦再帮忙review下 |
|
||
| 配置项 | 类型 | 必填 | 默认值 | 说明 | | ||
| -------------- | ------ | ---- | ------ | ----------------------------------------------- | | ||
| service_source | string | 必填 | - | 类型为固定ip或者DNS,输入redis服务的注册来源 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-cache/main.go
麻烦参考下这里的redis配置解析和client初始化逻辑,使用FQDNCluster,简化用户的配置方式
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/alibaba/higress/blob/main/plugins/wasm-go/extensions/ai-cache/main.go
麻烦参考下这里的redis配置解析和client初始化逻辑,使用FQDNCluster,简化用户的配置方式
好的,这里我再调整下
| service_name | string | 必填 | - | 输入redis服务的注册名称 | | ||
| service_port | int | 必填 | - | 输入redis服务的服务端口 | | ||
| service_host | string | 必填 | - | 当类型为固定ip时必须填写,输入redis服务的主机名 | | ||
| service_domain | string | 必填 | - | 当类型为DNS时必须填写,输入redis服务的domain | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
其实redis 不需要设置domain
rule_name: limit_by_per_ip_from-header-x-forwarded-for | ||
limit_by_per_ip: from-header-x-forwarded-for | ||
limit_keys: | ||
# 精确ip |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里有一段空白符
| 配置项 | 类型 | 必填 | 默认值 | 说明 | | ||
| ----------------------- | ------ | ---- | ------ | ---- | | ||
| rule_name | string | 是 | - | 限流规则名称,根据限流规则名称和限流的客户端IP段来拼装redis key | | ||
| limit_by_header | string | 否,`limit_by_header`,`limit_by_param`,`limit_by_per_ip` 中选填一项 | - | 配置获取限流键值的来源 http 请求头名称 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字
这个应该是要配合认证的插件一起使用对吧
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
是的,如果用户开了 key-auth 等认证插件,就可以配合基于 consumer 做流控了,在openapi场景下挺有用的
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
是的,如果用户开了 key-auth 等认证插件,就可以配合基于 consumer 做流控了,在openapi场景下挺有用的
好的
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以考虑加一个 limit_by_consumer, 现在通过 x-mse-consumer 这个request header可以取出 consumer 的名字
这个目前根据请求头限流的应该可以覆盖这个场景了,limit_by_header直接配置成x-mse-consumer就可以了,如果单独再实现一个limit_by_consumer会有点奇怪,在limit_by_header使用文档中添加下对x-mse-consumer的使用场景的说明会更好些
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
limit_by_consumer 更好一些,目前通过header传递consumer信息的方式后续可能会调整,有了limit_by_consumer,用户就不感知这个调整
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
limit_by_consumer 更好一些,目前通过header传递consumer信息的方式后续可能会调整,有了limit_by_consumer,用户就不感知这个调整
好的,limit_by_consumer实现的时候也会和limit_by_per_header、limit_by_per_param一样支持配置一个正则表达式
| show_limit_quota_header | bool | 否 | false | 响应头中是否显示`X-RateLimit-Limit`(限制的总请求数)和`X-RateLimit-Remaining`(剩余还可以发送的请求数) | | ||
| rejected_code | int | 否 | 429 | 请求被限流时,返回的HTTP状态码 | | ||
| rejected_msg | string | 否 | Too many requests | 请求被限流时,返回的响应体 | | ||
| redis | object | 是 | - | redis相关配置 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以进一步扩展一下,类似 per ip 这种,用于不可枚举场景的限流:
limit_by_per_header
limit_by_per_param
key 可以支持配置一个正则表达式,符合正则匹配的进行限制
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可以进一步扩展一下,类似 per ip 这种,用于不可枚举场景的限流: limit_by_per_header limit_by_per_param
key 可以支持配置一个正则表达式,符合正则匹配的进行限制
key以regexp-开头的标识根据正则表达式匹配,示例如下:
rule_name: limit_by_param_x-ca-key
limit_by_header: x-ca-key
limit_keys:
- key: 102234
query_per_second: 10
- key: regexp-\d+
query_per_hour: 10
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
regexp:\d+
用冒号分隔看上去自然一些
} | ||
|
||
// 构建redis限流key和参数 | ||
limitKey := fmt.Sprintf(ClusterRateLimitFormat, config.ruleName, key) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
redis key add ruleType
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Ⅰ. Describe what this PR did
cluster key rate limit
Ⅱ. Does this pull request fix one issue?
fixes #851
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
1)、识别请求参数 apikey,进行区别限流
wasmplugin.yam:
1)根据第一个apikey进行限流
一分钟内请求三次:
响应头中x-ratelimit-limit为10(限制的总请求数),x-ratelimit-remaining为7(剩余还可以发送的请求数)
触发限流后:
2)根据第二个apikey限流
请求三次:
redis中数据:
2)、识别请求头 x-ca-key,进行区别限流
wasmplugin.yam:
1)根据第一个x-ca-key限流
一分钟内请求三次:
响应头中x-ratelimit-limit为10(限制的总请求数),x-ratelimit-remaining为7(剩余还可以发送的请求数)
触发限流后:
3)、根据请求头 x-forwarded-for 获取对端IP,进行区别限流
1)命中1.1.1.1
请求三次:
2)命中1.1.1.0/24
请求四次:
redis中数据(key名称包含从x-forwarded-for中获取的IP 1.1.1.2):
Ⅴ. Special notes for reviews