Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try detecting timer and activity resurrection #4375

Merged
merged 4 commits into from
Aug 24, 2021

Conversation

yycptt
Copy link
Contributor

@yycptt yycptt commented Aug 16, 2021

What changed?
When processing user timer or activity timeout task, if a timer is overdue for a long time (by default, 24 hours), scan workflow history to see if that timer has already been fired or if that activity has ready finished.

Why?
For detecting timer/activity resurrection and prevent duplicated timer fired or activity completion.

How did you test it?

  • Unit test.

Potential risks
N/A in the worst case, we can increase the threshold for detecting resurrection to be a large number to turn the feature off.

Release notes

Documentation Changes

@yycptt yycptt requested review from yux0 and a team August 16, 2021 17:03
@coveralls
Copy link

coveralls commented Aug 16, 2021

Pull Request Test Coverage Report for Build ce964594-29fe-49e4-a9df-52f9658ba3c5

  • 162 of 190 (85.26%) changed or added relevant lines in 4 files are covered.
  • 154 unchanged lines in 15 files lost coverage.
  • Overall coverage decreased (-0.02%) to 56.478%

Changes Missing Coverage Covered Lines Changed/Added Lines %
service/history/task/timer_active_task_executor.go 150 178 84.27%
Files with Coverage Reduction New Missed Lines %
common/membership/rpServiceResolver.go 2 73.58%
common/task/weightedRoundRobinTaskScheduler.go 2 88.6%
service/history/execution/mutable_state_builder.go 2 69.86%
service/matching/taskListManager.go 2 74.09%
common/persistence/nosql/nosqlplugin/cassandra/workflow.go 3 50.23%
service/history/queue/timer_gate.go 3 95.83%
common/persistence/executionManager.go 4 75.5%
common/persistence/statsComputer.go 4 95.71%
common/persistence/nosql/nosqlplugin/cassandra/workflowUtils.go 11 77.83%
service/history/queue/transfer_queue_processor.go 12 57.64%
Totals Coverage Status
Change from base Build 4a34a039-15ff-46ba-b5f7-0f167c6f8e52: -0.02%
Covered Lines: 78896
Relevant Lines: 139693

💛 - Coveralls

Copy link
Contributor

@yux0 yux0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great changes. I think we can also have a similar thing to detect the mutable state and history not sync issue.

@yycptt yycptt merged commit 170deed into cadence-workflow:master Aug 24, 2021
@yycptt yycptt deleted the detect-resurrection branch August 24, 2021 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants