Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metric nodes depends_on will include semantic_models based on measures, but not based on filters #3682

Closed
jtcohen6 opened this issue Jul 6, 2023 · 3 comments · Fixed by #3733
Assignees
Labels
dbt-core v1.6 Docs impact for the v1.6 release (July 2023) semantic-layer-beta Label for new semantic layer beta launch

Comments

@jtcohen6
Copy link
Collaborator

jtcohen6 commented Jul 6, 2023

Somewhere in these docs:

Let's capture that metric nodes will reflect dependencies on semantic models for their measures, but will not reflect dependencies based on filters, for the purposes of:

  • dbt selection syntax
  • visualizing the DAG (in dbt-docs & IDE)

Context

(from Quigley)

There are three-ish places that metrics need to source the nodes for it's depends_on attribute

  1. Metric.type_params.input_metrics for RATIO and DERIVED type metrics
  2. Metric.type_params.measure for SIMPLE and CUMULATIVE type metrics
  3. From filters
    a. Metric.filter
    b. Metric.type_params.measure.filter
    c. Metric.type_params.input_metrics[x].filter

The work for 1 and 2 exist currently on dbt-labs/dbt-core#8015.

Number 3 is a bit harder though. This is because
A. The spec for filters actually isn't finalized yet. The MetricFlow team is working on support for dunders
B. Which semantic model a dimension comes from in a filter is determined at query time (because a dimension can exist on multiple semantic models and the optimal join path is calculated).

Thus this issue is just to resolve 1 & 2, whereas 3 will be pushed off and become a separate core ticket/issue

Originally posted by @QMalcolm in dbt-labs/dbt-core#7854 (comment)

@jtcohen6 jtcohen6 added this to the dbt Core v1.6 Documentation milestone Jul 6, 2023
@jtcohen6 jtcohen6 added idea Proposes an idea for new content semantic-layer-beta Label for new semantic layer beta launch dbt-core v1.6 Docs impact for the v1.6 release (July 2023) labels Jul 6, 2023
@jtcohen6
Copy link
Collaborator Author

jtcohen6 commented Jul 13, 2023

dbt list --select my_semantic_model+

will include metrics defined on that semantic model

given that we only just merged the PR for this, let's confirm it actually works

if (and only if) the metrics are defined to use one of that semantic model's measures (or derived metrics downstream of those metrics)

but if a metric only references a semantic model by specifying one of its dimensions in its filters - it will not be considered downstream of that semantic model

@mirnawong1 mirnawong1 self-assigned this Jul 13, 2023
@jtcohen6
Copy link
Collaborator Author

jtcohen6 commented Jul 13, 2023

@mirnawong1 Just caught that you self-assigned this. I also asked @QMalcolm to help out here, so he can clarify / answer any questions / pitch in as well!

@mirnawong1
Copy link
Contributor

Dependencies

Metric nodes will reflect dependencies on semantic models for their measures. However, dependencies based on filters should not be reflected in:

This is because metrics need to source nodes for their depends_on attribute from three different places:

  • RATIO and DERIVED type metrics should reference Metric.type_params.input_metrics.
  • SIMPLE type metrics should reference Metric.type_params.measure.

dbeatty10 added a commit that referenced this issue Jul 15, 2023
adding language that metric nodes include dependencies on semantic
models based on measures but not filters in dag and dbt selection.

Resolves #3682

---------

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
@runleonarun runleonarun removed the idea Proposes an idea for new content label Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt-core v1.6 Docs impact for the v1.6 release (July 2023) semantic-layer-beta Label for new semantic layer beta launch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants