Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TableMeta Error #1168

Closed
wongminbin opened this issue Nov 21, 2018 · 5 comments
Closed

TableMeta Error #1168

wongminbin opened this issue Nov 21, 2018 · 5 comments
Assignees
Labels
Milestone

Comments

@wongminbin
Copy link

issues #980 描述同样的问题,得到的结果是已经在v1.1.1版本解决该Bug。
而我当前的版本就是v1.1.1,依然出现该问题

2018-11-20 17:25:19.708 [[scheduler-table-meta-snapshot]] ERROR c.a.o.canal.parse.inbound.mysql.tsdb.DatabaseTableMeta - pls submit github issue, show create table ddl:CREATE TABLE vs_order_goods_hold (
pk_id int(11) NOT NULL AUTO_INCREMENT COMMENT '主键',
fk_order_id int(11) NOT NULL COMMENT '订单ID(表:vs_order)',
fk_store_id int(11) NOT NULL COMMENT '店铺ID(表:vs_store)',
fk_goods_id int(11) NOT NULL COMMENT '商品ID(表:vs_store_goods)',
fk_specgds_id int(11) NOT NULL COMMENT '规格ID(表:esl_gds_specgds)',
hold_num int(10) NOT NULL COMMENT '订单规格商品数量',
is_archiving int(1) NOT NULL COMMENT '是否归档;0:false ,1:true',
collage_order int(1) DEFAULT NULL COMMENT '是否拼团订单商品(0:否,1:是)',
fk_collage_act_id int(11) DEFAULT NULL COMMENT '拼团活动ID(表:esl_sale_activity)',
remark varchar(50) DEFAULT NULL COMMENT '备注',
create_user int(11) NOT NULL COMMENT '创建者用户ID(表:vs_member)',
create_time datetime NOT NULL COMMENT '创建时间',
update_user int(11) DEFAULT NULL COMMENT '更新者用户ID(表:vs_member)',
update_time datetime DEFAULT NULL COMMENT '更新时间',
PRIMARY KEY (pk_id)
) ENGINE=InnoDB AUTO_INCREMENT=3540 DEFAULT CHARSET=utf8 COMMENT='订单占有商品库存' , compare failed .
db : TableMeta [schema=phs_erp_test, table=vs_order_goods_hold, fileds=
FieldMeta [columnName=pk_id, columnType=int(11), nullable=false, key=true, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_order_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_store_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_goods_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_specgds_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=hold_num, columnType=int(10), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=is_archiving, columnType=int(1), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=collage_order, columnType=int(1), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_collage_act_id, columnType=int(11), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=remark, columnType=varchar(50), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=create_user, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=create_time, columnType=datetime, nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=update_user, columnType=int(11), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=update_time, columnType=datetime, nullable=true, key=false, defaultValue=null, extra=null, unique=false]
]
mem : TableMeta [schema=phs_erp_test, table=vs_order_goods_hold, fileds=
FieldMeta [columnName=pk_id, columnType=int(11), nullable=false, key=true, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_order_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_store_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_goods_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=fk_specgds_id, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=hold_num, columnType=int(10), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=is_archiving, columnType=int(1), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=remark, columnType=varchar(50), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=create_user, columnType=int(11), nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=create_time, columnType=datetime, nullable=false, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=update_user, columnType=int(11), nullable=true, key=false, defaultValue=null, extra=null, unique=false]
FieldMeta [columnName=update_time, columnType=datetime, nullable=true, key=false, defaultValue=null, extra=null, unique=false]
]

@agapple
Copy link
Member

agapple commented Nov 21, 2018

collage_order, fk_collage_act_id 少了这两个列,得找找DDL的变更记录

@wongminbin
Copy link
Author

我在h2文件中找到DDL变更记录,但只有对其中一个SCHEMA操作记录
image

@agapple
Copy link
Member

agapple commented Nov 21, 2018

@wongminbin 从SQL的解析来看,是支持你这个ALTER TABLE的sql的,理论上可以得到内存里正确的表结构,有条件的可以debug试试DatabaseTableMeta.rollback方法,看看重新定位之后拿到的tableMeta是啥?

@wongminbin
Copy link
Author

@agapple
我想我已经试验并重现Bug的流程
1、一开始Canal Instance、Canal Client的filter.regex=.\..
2、DatabaseTableMeta的MemoryTableMeta记录所有schema下的信息
3、中途改变Canal Client的filter.regex=schema_a\..,然后覆盖了Canal Instance的filter.regex=schema_a\..
4、对主库schema_b下table_a操作了DDL命令,因为过滤规则,Canal Instance并没有执行该DDL,所以schema_b的MemoryTableMeta还是旧数据
5、DatabaseTableMeta中有一个24h的定时任务,生成主库的一份snapshot。因为以本地MemoryTableMeta为基准和主库schema信息作比较,所以异常出现在compareTableMetaDbAndMemory

image

@agapple
Copy link
Member

agapple commented Nov 22, 2018

也就是过滤条件发生了变化之后导致的咯?

@agapple agapple closed this as completed Nov 25, 2018
@agapple agapple added this to the v1.1.2 milestone Nov 25, 2018
@agapple agapple added the bug label Nov 25, 2018
@agapple agapple self-assigned this Nov 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants