Feature: dependent assertions as dependencies #1720
Conversation
There was a problem hiding this comment.
There's quite a few outstanding comments in the design doc, and we haven't done a review session to close them, which we should do before this is merged!
Why is it just operations and tables that can have dependOnDependencyAssertions? Shouldn't all action types (excluding declarations) be able to require directly downstream assertions to have completed in order to run?
We have 5 types of actions Assertions, Declaration, Notebook, Operations, and Tables.
that leave only Tables and Operations. |
| @@ -901,7 +901,7 @@ suite("@dataform/core", () => { | |||
| expect( | |||
| cGraph.graphErrors.compilationErrors.filter(item => | |||
| item.message.match( | |||
| /Ambiguous Action name: {\"name\":\"a\"}. Did you mean one of: foo.a, bar.a./ | |||
| /Ambiguous Action name: {\"name\":\"a\",\"includeDependentAssertions\":false}. Did you mean one of: foo.a, bar.a./ | |||
There was a problem hiding this comment.
Note to self: this is kind of gross, we should fix it by not using JSON stringify like this https://github.com/dataform-co/dataform/blob/754d582f4f588b3c812c40a38ec839607ec3d4f0/core/utils.ts#L129C42-L129C53
There was a problem hiding this comment.
The version should be increased here, for publishing the changes https://github.com/dataform-co/dataform/blob/main/version.bzl
|
I've got most of the cloud build tests working in https://github.com/dataform-co/dataform, I think next time you push they should pop up! If not, we can just run |
* Update machine type and references to non dataform-open-source * Disable remote cache * Temporarily disable tests that require working BQ credentials * Make CLI test disablement only for run * Nit tidy
* Feature: dependent assertions as dependencies * Action.yaml support for dependent assertions * Review updates * spell check and ActionMap * dependency duplication check O(n) * Formating * includeAssertionsForDependencyMap changed to includeAssertionsForDependency * review fixes * Lint fixes * fix shadow beforeEach call
Feature: Dependent assertions as dependencies.
Summary:
This PR introduces changes that will allow users to add dependent assertions as dependencies.
This required a new
ActionAssertionMapthat holds information regarding assertions that depend on each action. This map is updated while create assertions.New flags for Table and Operation to allow managing dependent assertions. These flags are updated for each dependency when its added in the Action either through config or ref(both happens in same place).
Lastly, when dependencies are qualified for compiledGraph, we add dependent assertions as dependencies using the ActionAssertionMap and the flags.
This PR also introduces new tests to test this feature and minor updates to some old test to adjust to new flags being introduces.
#core #feature #dependentAssertionsAsDependencies