Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

canal client need do connector.rollback() when catching exception #2756

Closed
2 tasks done
hxy91819 opened this issue May 21, 2020 · 1 comment
Closed
2 tasks done

canal client need do connector.rollback() when catching exception #2756

hxy91819 opened this issue May 21, 2020 · 1 comment

Comments

@hxy91819
Copy link

hxy91819 commented May 21, 2020

  • I have searched the issues of this repository and believe that this is not a duplicate.
  • I have checked the FAQ of this repository and believe that this is not a duplicate.

environment

  • canal version = 1.1.4
  • mysql version = 5.7

Issue Description

I have been continuing suffering a data loss when using canal. I found there always been some exceptions (like connect problems) happened with data loss.

So I check the client code in AbstractCanalClientTest.java

In line 79, we got // connector.rollback(batchId); // 处理失败, 回滚数据

I think it will be good if we add connector.rollback(batchId) when we got an exception.

Such as:

while (running) {
    Message message = connector.getWithoutAck(batchSize); // 获取指定数量的数据
    long batchId = message.getId();
    try {
        int size = message.getEntries().size();
        if (batchId == -1 || size == 0) {
            // 无效信息,可以考虑休眠
        } else {
            printSummary(message, batchId, size);
            printEntry(message.getEntries());
            if (handleEntry != null) {
                handleEntry.invoke(message.getEntries());
            }
        }

        if (batchId != -1) {
            connector.ack(batchId); // 提交确认
        }
    } catch (Exception e) {
        try {
            Thread.sleep(1000L);
        } catch (InterruptedException e1) {
            Thread.currentThread().interrupt();
        }
        if (batchId != -1) {
            connector.rollback(batchId); // 处理失败, 回滚数据
        }
    }
}

The question is : why you guys didn't do this in the code of examples? Is this irregular?

Steps to reproduce

Expected behaviour

Actual behaviour

If there is an exception, please attach the exception trace:

Just put your stack trace here!
@hxy91819 hxy91819 changed the title canal client need rollback when meeting exception canal client need do connector.rollback() when catching exception May 21, 2020
@agapple
Copy link
Member

agapple commented Aug 21, 2020

默认rollbackOnConnect=true,会在下一轮自动rollback,不过代码的确是可以优化一下

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants