Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optimize transformer go wasm plugin #712

Merged

Conversation

Uncle-Justice
Copy link
Contributor

@Uncle-Justice Uncle-Justice commented Dec 19, 2023

Ⅰ. Describe what this PR did

  1. 支持双向转换
  2. map支持跨类型映射
  3. kvt操作顺序自定义
  4. 操作内部的字段命名优化

Ⅱ. Does this pull request fix one issue?

fix #587

Ⅲ. Why don't you add test cases (unit test/integration test)?

已附带相应e2e测试

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

map支持跨类型映射

目前的代码结构决定了,map操作的顺序是headers->querys->body,即如果我要做headers的map操作,映射来源是querys时,实际参考的数据是未被transform过的旧query,以上逻辑应该是合理的,不过或许需要在readme中追加说明。

@codecov-commenter
Copy link

codecov-commenter commented Dec 19, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (0bb9e6d) 37.99% compared to head (b157aec) 38.10%.
Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #712      +/-   ##
==========================================
+ Coverage   37.99%   38.10%   +0.11%     
==========================================
  Files          61       61              
  Lines       10322    10336      +14     
==========================================
+ Hits         3922     3939      +17     
+ Misses       6099     6098       -1     
+ Partials      301      299       -2     

see 4 files with indirect coverage changes

@johnlanni
Copy link
Collaborator

@WeixinX 看下

Copy link
Collaborator

@WeixinX WeixinX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

除了以下修改意见外,也请麻烦更新 README,另外由于 E2E 还未支持 body 的校验,在本地测试时也请多关注一下 ~

plugins/wasm-go/extensions/transformer/main.go Outdated Show resolved Hide resolved
test/e2e/conformance/tests/go-wasm-transformer.yaml Outdated Show resolved Hide resolved
)

type Param struct {
paramType ParamType
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

与父级的 operate 字段重合,个人认为可以去掉 ParamType

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已将Param.paramType类型修改为string,直接根据operate值进行初始化;删除ParamType类型声明

mapSource := getMapSourceFromRule(config.reqRules)

// TODO: not support mapping headers or querys from body in requestTransformer before #582 is fixed
if mapSource == "body" && (isHeaderChange || isQueryChange) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isHeaderChange 似乎并不能说明一定存在 header map 操作,isQueryChange 同理,它们只说明了 headers 或 querys 中存在某种转换操作。尽管 mapSource 被配置能说明 headers, querys 中存在 map 操作,但不排除用户错误配置的可能?807 行同理。

Copy link
Contributor Author

@Uncle-Justice Uncle-Justice Dec 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已将不支持map的条件更加细化。比如此处,必须在headers或者query中检测出map要求,才不支持相应map操作

switch config.reqTrans.GetMapSource() {
case "headers":
{
headers, err := proxywasm.GetHttpRequestHeaders()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetHttpRequestHeaders is used for retrieving HTTP request headers. Only available during types.HttpContext.OnHttpRequestHeaders and types.HttpContext.OnHttpStreamDone.

有一种方案是在 onHttpRequestHeaders 时获取 headers,并通过 ctx 传递。586 行同理。

也请添加 map from header 的测试用例 ~

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. 已在onHttpReq/RespBody中对headers和query的获取方式改为通过ctx
  2. 已增加map from header to query的测试用例,发现因为在之前的代码读取headers时会将其全部改为小写(func convertHeaders(hs [][2]string) map[string][]string),而qquery并未做这样的操作, 导致如果headers中的某个fromKey含大写字母,query匹配不上,只能匹配小写版的fromKey,请问这个算预期内行为吗?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Uncle-Justice @johnlanni 可以考虑在匹配时将 query 转为小写进行匹配?或是保留 header 原本的状态?(不确定 proxywasm.GetHttpRequestHeaders() 获取到的 header 是否有进行小写转换)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改为:当映射来源为headers时,fromKey以小写形式去做匹配

@Uncle-Justice
Copy link
Contributor Author

@WeixinX 谢谢你的指导和建议,以上修改意见均已做相应改动,README也已经更新。关于body的测试,在#733 中,我已经写了一点测试用例,后续等#733 合并后我会追加更完整的测试用例

麻烦recheck


type Param struct {
paramType ParamType
paramType string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

目前的逻辑是在使用到 paramType 的地方(constructParam)会将 operate 复制给 paramType,所以我认为可以去掉该字段,用 operate 字段做相应的判断即可

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你说得对,这个后面也没有使用,该字段现已删除

switch config.reqTrans.GetMapSource() {
case "headers":
{
headers, err := proxywasm.GetHttpRequestHeaders()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Uncle-Justice @johnlanni 可以考虑在匹配时将 query 转为小写进行匹配?或是保留 header 原本的状态?(不确定 proxywasm.GetHttpRequestHeaders() 获取到的 header 是否有进行小写转换)

plugins/wasm-go/extensions/transformer/main.go Outdated Show resolved Hide resolved
plugins/wasm-go/extensions/transformer/main.go Outdated Show resolved Hide resolved
@johnlanni
Copy link
Collaborator

johnlanni commented Dec 28, 2023

@WeixinX proxywasm.GetHttpRequestHeaders() 获取到的header key都是小写的

Copy link
Collaborator

@WeixinX WeixinX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@johnlanni
Copy link
Collaborator

@Uncle-Justice Please resolve the conflicts.

@Uncle-Justice
Copy link
Contributor Author

@johnlanni Roger that

Copy link
Collaborator

@johnlanni johnlanni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@johnlanni johnlanni merged commit 879192c into alibaba:main Jan 22, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wasm plugin optimization plan for request/response Transformer
4 participants