Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rill Developer: observability for incremental & split Models #5720

Merged
merged 38 commits into from
Sep 24, 2024

Conversation

ericpgreen2
Copy link
Contributor

@ericpgreen2 ericpgreen2 commented Sep 17, 2024

This PR adds observability for incremental and split models in the Rill Developer Model workspace. Closes #5579.

Model Refresh Button

Previously in Rill Developer, models were only executed after an edit & save. Now, a refresh button can be used to rerun a model at any time.

For incremental models, the button provides two options:

  • Incremental refresh: Rerun the model with the current state.
  • Full refresh: Reset any incremental state and rerun the model.
image

Inspector: Incremental Processing

Incremental models now include an "Incremental processing" section in the inspector, showing any data held in the state.

image

If the model is incremental but does not leverage any state, an informational message will be displayed.

image

Inspector: Model Splits

Split models now have a "Model splits" section in the inspector. Clicking "View splits" will open a dialog with:

  • A table displaying splits, execution timestamps, watermarks, and errors.
  • Filtering for pending and errored splits.
  • The ability to refresh specific splits.
image

@ericpgreen2 ericpgreen2 self-assigned this Sep 17, 2024
@ericpgreen2 ericpgreen2 changed the title Rill Developer: Model splits UI Rill Developer: observability for incremental & split Models Sep 20, 2024
@ericpgreen2 ericpgreen2 marked this pull request as ready for review September 20, 2024 21:00
Copy link
Contributor

@lovincyrus lovincyrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SplitsTable works as expected from the screenshots.

On a side note, we should make the header sticky.
CleanShot 2024-09-23 at 00 20 17@2x

Copy link
Contributor

@lovincyrus lovincyrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very close! I noticed that the scroller continues to scroll to the bottom of the table even when there are no more rows available. Check splits_days_incremental.yaml and splits_glob.yaml

CleanShot.2024-09-24.at.08.47.38.mp4

Copy link
Contributor

@lovincyrus lovincyrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@ericpgreen2 ericpgreen2 merged commit 9a785a5 into main Sep 24, 2024
7 checks passed
@ericpgreen2 ericpgreen2 deleted the model-splits-ui branch September 24, 2024 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rill Developer: show a Model's incremental state in the Inspector
3 participants