增加表字段过滤功能

[cjk87927]

instance增加配置

canal.instance.filter.field

• 格式: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2
• 示例: test1.t_product:id/subject/keywords,test2.t_company:id/name/contact/ch

canal.instance.filter.black.field

• 格式: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2
• 示例: test1.t_product:subject/product_image,test2.t_company:id/name/contact/ch

冲突时以白名单为准

spring/xxx-instance增加

<property name="fieldFilter" value="${canal.instance.filter.field}" />
<property name="fieldBlackFilter" value="${canal.instance.filter.black.field}" />

实现

com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert行解析中对字段名进行判断,不符合条件的不加入rowDataBuilder

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@adccc0e). Click here to learn what that means.
The diff coverage is 0%.

@@           Coverage Diff            @@
##             master   #1914   +/-   ##
========================================
  Coverage          ?   5.31%           
  Complexity        ?     126           
========================================
  Files             ?     265           
  Lines             ?   23944           
  Branches          ?    3510           
========================================
  Hits              ?    1272           
  Misses            ?   22460           
  Partials          ?     212

Impacted Files	Coverage Δ	Complexity Δ
...al/parse/inbound/mysql/tsdb/DatabaseTableMeta.java	0% <0%> (ø)	0 <0> (?)
...otter/canal/parse/inbound/AbstractEventParser.java	0% <0%> (ø)	0 <0> (?)
.../parse/inbound/mysql/AbstractMysqlEventParser.java	0% <0%> (ø)	0 <0> (?)
...al/parse/inbound/mysql/dbsync/LogEventConvert.java	0% <0%> (ø)	0 <0> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update adccc0e...1826169. Read the comment docs.

[agapple]

需要字段过滤的主要业务场景是啥？想了解一下

[cjk87927]

比如我要同步新闻表
canal -> MQ -> Java处理 -> Solr
JAVA处理中是会取出新闻数据的,MQ过来的其实只要一个新闻ID就可以,但如果所有数据都扔到MQ,很有可能超出消息大小(新闻内容过多,图片用了base64等等)

也就是说 我在乎的是某一条数据的变化,但不在乎他具体变成什么样了,因为后面还有业务逻辑来处理

[agapple]

canal.instance.filter.field=test1.t_product=id/subject/keywords,test2.t_company=id/name/contact/ch

能按照冒号来分隔表和列的关系，例子:
canal.instance.filter.field=test1.t_product:id/subject/keywords

还有类似黑名单列的例子：
canal.instance.filter.black.field=test1.t_product:subject

可以按照完整功能的方式来做支持

[agapple]

tks