[CT-3123] Make seed inputs optional for unit tests

[graciegoheen]

Description

Our proposed unit testing spec, doesn't yet take into account if one of the inputs to the model you want to add a unit test to is a seed.

Current behavior: Let's say model_a depends on my_seed. If you want to add a unit test to model_a - you need to define an input for my_seed. That feels redundant since seeds don't often change and are already small data inputs.

Proposed solution:

Let's say you're unit testing my_favorite_model that depends on:

• my_favorite_source (a source)
• my_second_favorite_model (a model)
• and my_special_seed (a seed)

You must supply mock input data for all of the direct parent models and sources.

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      - input: ref('my_favorite_source')
        format: csv
        fixture: my_favorite_source_fixture
      - input: ref('my_second_favorite_model')
        format: csv
        fixture: my_second_favorite_model_fixture

In this case, we just use your seed for my_special_seed input.

You may supply mock input data for direct parent seeds, if desired.

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      - input: ref('my_favorite_source')
        format: csv
        fixture: my_favorite_source_fixture
      - input: ref('my_second_favorite_model')
        format: csv
        fixture: my_second_favorite_model_fixture
      - input: ref('my_special_seed')
        format: csv
        fixture: my_special_seed_fixture

In this case, we use your fixture my_special_seed_fixture for my_special_seed input.

Acceptance Criteria

• If no input is provided for a seed, we will use your seed (ignore pre and post hooks)
• If an input is provided for a seed, we will use the input

Impact to other teams

None

Will backports be required?

No

Context

[dbeatty10]

Maybe it "just works"?

Let's suppose my_upstream_model_a is a model (rather than a seed) and this is how we configure a fixture for it:

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      - input: ref('my_upstream_model_a')
        format: csv
        fixture: my_upstream_model_a_fixture

Then we can imagine that the above should "just work" even if my_upstream_model_a were a seed instead -- the seed's CSV rows would just be replaced by the CSV rows within my_upstream_model_a_fixture, right?

Can we think of reasons why it wouldn't "just work"?

An additional thought

For the special case of my_upstream_model_a_fixture being a seed, what about the idea of being able to make format and fixture optional:

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      - input: ref('my_upstream_model_a')

Or maybe even going the whole way and making the input: optional for upstream seeds:

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      # presumably other `input`s are explicitly defined here, but at least `my_upstream_model_a` is implicitly defined -- since it's a seed, we already have all the data we need to mock it

[graciegoheen]

Updated the issue!! Thanks @dbeatty10 for your great ideas :)

---

Additional thought...

What would happen if you didn't care to supply mock data for one of the direct parent models (because it's not relevant for the logic that you're trying to test), where you want to instead use an "empty" mock data set or a one-row, all NULL data set?

• would you not have to supply the input at all?

unit-tests:
  - name: test_my_model
    model: my_model
    given:

• or would we require that you explicitly specify that behavior?

unit-tests:
  - name: test_my_model
    model: my_model
    given:
      - input: ref('my_upstream_model_a')
        rows: {}

The second feels less magical, but I like that it's an intentional decision rather than the risk of "oops, I forgot to add mock data for my_upstream_model_a but didn't get any error"!

Food for thought... probably for a separate issue. cc: @MichelleArk

[graciegoheen]

Notes from refinement:

• we should have a consistent limit for how large a seed & fixture can be (maybe if you seed file is larger than the size limit for fixture, you must define a fixture for your seed) -> let's open up a new issue for this!

[graciegoheen]

Notes from estimation:

• seeds can have pre-hooks & post-hooks; can we just ignore these for unit testing? that's what we're doing for models

[aranke]

Hey @graciegoheen, can you please clarify how this scenario can be reached during unit testing?

seeds can have pre-hooks & post-hooks; can we just ignore these for unit testing? that's what we're doing for models

From what I can see, these only get triggered during dbt seed and not during dbt unit-test.

Furthermore, the spec labels this as an anti-goal:

Unit testing seeds, snapshots, analyses, or macros directly (they can be tested by making a passthrough model though!)

Given this, is it fair to cut pre- / post-hooks for seeds out of scope?

[graciegoheen]

Yes, unit testing seeds directly is an anti-goal - so we can cut pre- / post- hooks for seeds out of scope!