Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hotfix][Zeta] Fix job crash when schema changing on multiple parallel readers #6977

Merged
merged 2 commits into from
Jun 13, 2024

Conversation

dailai
Copy link
Contributor

@dailai dailai commented Jun 13, 2024

Purpose of this pull request

Fix the close task exception during Full incremental switching.

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

@dailai
Copy link
Contributor Author

dailai commented Jun 13, 2024

When I used this pr(#6929) to test the schema change using multi-parallel tasks, I found that the schema change during the full incremental switch of cdc tasks would result in the following exceptions. Link #6526 #6389
截屏2024-06-13 10 58 00

@dailai
Copy link
Contributor Author

dailai commented Jun 13, 2024

After the modification, the tests are normal for many times. You can also see the effect in logs.

截屏2024-06-13 11 00 21

@hailin0
Copy link
Member

hailin0 commented Jun 13, 2024

The PR that caused this issue
#6389

Fix suggestion:remove this IllegalStateException and add logs
image

@dailai dailai force-pushed the hotfix-for-sourceflow branch from 6812e03 to 58b468f Compare June 13, 2024 05:57
@dailai dailai changed the title [Hotfix][Seatunnel-engine] Fix the close task exception during Full incremental switching [Hotfix][Seatunnel-engine] Fix the close task exception during Full incremental switching with schema evolution Jun 13, 2024
@hailin0 hailin0 changed the title [Hotfix][Seatunnel-engine] Fix the close task exception during Full incremental switching with schema evolution [Hotfix][Zeta] Fix job crash when schema changing on multiple parallel readers Jun 13, 2024
Copy link
Member

@hailin0 hailin0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hailin0 hailin0 merged commit 1f33145 into apache:dev Jun 13, 2024
3 of 4 checks passed
@dailai dailai deleted the hotfix-for-sourceflow branch June 13, 2024 08:10
chaorongzhi pushed a commit to chaorongzhi/seatunnel that referenced this pull request Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants