Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YamlUtils工具类 ymlToObj函数 Properties baseProperties参数问题 #4355

Closed
2 tasks
chen-collab opened this issue Aug 14, 2022 · 4 comments
Closed
2 tasks
Milestone

Comments

@chen-collab
Copy link

chen-collab commented Aug 14, 2022

  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I have checked the FAQ of this repository and believe that this is not a duplicate.

environment

  • canal 1.1.7
  • mysql 5.7.34

Issue Description

com.alibaba.otter.canal.client.adapter.support 包下YamlUtils工具类ymlToObj

java 基本数据类型作为参数传递时,传递的是值的拷贝,无论怎么改变这个拷贝,原值是不会改变的,
引用对象参数传递时,传递的是指向真实对象的地址,而函数中的形参 baseProperties 拿到同样的地址时,通过 baseProperties.putAll(properties);,会通过地址找到真实的对象进行操作,会改变属性数据,进而导致获取的配置对象错乱

Steps to reproduce

Expected behaviour

Actual behaviour

If there is an exception, please attach the exception trace:

Just put your stack trace here!
@xesygao
Copy link

xesygao commented Dec 6, 2022

同一个adapter下,如果有多个配置文件,那么multiMapping部分就会错乱,如果有相同的配置项并且为集合类型,后面的配置会包含前面的集合配置。定位到这部分内容,粗略看了下baseProperties.putAll(properties);这一步似乎没有使用到的地方。为了最小限度降低对现有代码的影响,我选择注释掉这句,实测配置可以正常读取。
image

@chinahuhq
Copy link

同样发现了这个问题,
场景描述:由于源库是一个采买项目,里面表结构定义比较混乱,存在不同的主键字段,id,ID,SEQORDER 等等。
在rdb文件夹下配置多个yml时,targetPk 写的是各个表自己的主键,但是在代码位置
com.alibaba.otter.canal.client.adapter.support.YamlUtils#ymlToObj(java.lang.String, java.lang.String, java.lang.Class, java.lang.String, java.util.Properties)
image
中,会导致第二次配置加载时读取了第一次配置的targetPk,越晚加载的拥有的targetPk越多。
最终在实际同步时,拼接主键逻辑com.alibaba.otter.canal.client.adapter.rdb.service.RdbSyncService#appendCondition(com.alibaba.otter.canal.client.adapter.rdb.config.MappingConfig.DbMapping, java.lang.StringBuilder, java.util.Map<java.lang.String,java.lang.Integer>, java.util.List<java.util.Map<java.lang.String,?>>, java.util.Map<java.lang.String,java.lang.Object>, java.util.Map<java.lang.String,java.lang.Object>)
image
进入type==null 判断后报错。

临时方案是注释掉部分代码,暂时没有出现问题。
希望有完整的修复方案,感谢。

@weirhp
Copy link
Contributor

weirhp commented Jan 7, 2023

在不改原逻辑的情况下建议这样修改。
微信截图_20230107084311

@weirhp
Copy link
Contributor

weirhp commented Jan 7, 2023

#4560 修改后的代码参考这里 #

@agapple agapple added this to the v1.1.7 milestone Sep 4, 2023
@agapple agapple closed this as completed Sep 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants