Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to force processing of specific entities #248

Open
andebor opened this issue Jun 26, 2023 · 0 comments
Open

Add option to force processing of specific entities #248

andebor opened this issue Jun 26, 2023 · 0 comments
Labels
enhancement New feature or request Priority:High

Comments

@andebor
Copy link
Contributor

andebor commented Jun 26, 2023

I would like a feature to mark an entity in a dataset as updated even if it hasn't been changed. Basically just a touch.
When fixing an issue related to an edge case entity, we usually need to run a full sync to reprocess the affected entities. This can sometimes lead to some unwanted effects (and potentially take several hours) and it would help a lot if we could specifically target the relevant entities instead.

Ideal Process

  1. Discover one or more entities than needs special handling in the transform
  2. Do changes to transform
  3. Deploy new version
  4. Touch list of affected entities in source dataset resulting in updated versions in sink dataset
@andebor andebor added the enhancement New feature or request label Jun 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Priority:High
Projects
None yet
Development

No branches or pull requests

1 participant