-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Auto-Disable Failing Connections feature #11099
Conversation
This is removing the attemptId from the create attempt activity to replace it with the attemptNumber. This will be modified in the workflow in a later commit.
final String JobStatusSelect = "SELECT status FROM jobs "; | ||
return jobDatabase.query(ctx -> ctx | ||
.fetch(JobStatusSelect + "WHERE " + | ||
"CAST(scope AS VARCHAR) = ? AND " + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to cast as a VARCHAR? scope = '?'
should be working (same with the config type).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought that was the convention since most of the other queries casted the scope, but just ran the unit tests without it and had no problems, so i'll remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I misspoke earlier, looks like the convention is not to cast the scope. But ConfigType
is normally casted.
* connection to prevent wasting resources | ||
*/ | ||
@ActivityMethod | ||
boolean autoDisableFailingConnection(AutoDisableConnectionActivityInput input); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Temporal recommends to use a POJO for the input/output. will help with adding element in the input or the output. This shouldn't require versioning if if we add something in the POJO. Even if it is likely that we will use one it might help to avoid potential issues if we forget it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done, added an output object
What
Adds feature to automatically disable failing connections. This addresses issue 9715. When this feature is turned on, it will disable a connection after
MAX_FAILED_JOBS_IN_A_ROW_BEFORE_CONNECTION_DISABLE
consecutive failures (currently defaults to 100 jobs) or afterMAX_DAYS_OF_ONLY_FAILED_JOBS_BEFORE_CONNECTION_DISABLE
days of only failed jobs (currently defaults to 14 days).todo:
Notify user when connection has been auto-disable.Will be followed up through this issue.listJobStatusWithConnection
is not filtering by timestamp in unit testsHow
When the
AUTO_DISABLE_FAILING_CONNECTIONS
feature flag is turned on, after a job failure, theDisableActivity
will check if either of the above criteria has been met. If so, it will set the connection status toINACTIVE
.Recommended reading order
airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
airbyte-workers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/AutoDisableConnectionActivityImpl.java