Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪟🐛 Keep refetching debug infos while job is running #14341

Merged
merged 5 commits into from
Jul 12, 2022

Conversation

timroes
Copy link
Collaborator

@timroes timroes commented Jul 1, 2022

What

Fixes #14244

Before changing the querying infrastructure we used the logs from the useGetJob call which auto refreshed after 2.5 seconds. Since the switch to the auto generated client, we use get_debug_info for the logs (since the useGetJob returns different types). This call wasn't auto refreshing so far.

This PR now makes this call to auto refresh while the job it's fetching debug infos for hasn't been finished less than 2 minutes ago. We need some buffer time, since logs come in for some time even after the job is marked as done.

@github-actions github-actions bot added area/platform issues related to the platform area/frontend Related to the Airbyte webapp labels Jul 1, 2022
@timroes timroes marked this pull request as ready for review July 1, 2022 17:21
@timroes timroes requested a review from a team as a code owner July 1, 2022 17:22
const lastAttemptEndTimestamp = data?.attempts[data.attempts.length - 1].attempt.endedAt;
// While no attempt ended timestamp exists yet (i.e. the job is still running) or it hasn't ended
// more than 2 minutes (2 * 60 * 1000ms) ago, keep refetching
return lastAttemptEndTimestamp && Date.now() - lastAttemptEndTimestamp * 1000 > 2 * 60 * 1000 ? false : 2500;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description says 6 minutes, but this looks like 2. Is one of them incorrect?

Copy link
Collaborator Author

@timroes timroes Jul 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, forgot to update that :) I created this originally with 6 minutes, but Davin clarified for me, that the log is supposed to be max 1 minute late based on how our log fetching implementation worked, so I felt 2 minutes might be a long enough grace period to make sure. Updated the PR description now as well.

@timroes timroes requested a review from teallarson July 11, 2022 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/frontend Related to the Airbyte webapp area/platform issues related to the platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Log view doesn't auto refresh for running jobs anymore
2 participants