Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref uses most recently referenced project rather than current #1504

Closed
tbescherer opened this issue Jun 4, 2019 · 1 comment · Fixed by #1515
Closed

ref uses most recently referenced project rather than current #1504

tbescherer opened this issue Jun 4, 2019 · 1 comment · Fixed by #1515
Labels
bug Something isn't working good_first_issue Straightforward + self-contained changes, good for new contributors!

Comments

@tbescherer
Copy link
Contributor

tbescherer commented Jun 4, 2019

Issue

The expected behavior for a reference with no specified project like {{ref('model_one')}} is that it will refer to the model "model_one" in the current project. However this is not the case when a ref earlier in the model specifies a project of its own. For instance were I to define a model in a project named "A" with these contents:
SELECT * FROM {{ref('B', 'model_one')}}, {{ref('model_two')}}, and then run dbt docs generate both model_one and model_two will be searched for in project "B", and dbt will throw an error (in this case, model_two is in the "A" project).

I believe this is due to the variable target_model_package being assigned in the outer loop of method process_refs. https://github.com/fishtown-analytics/dbt/blob/master/core/dbt/parser/util.py#L163-L169 This causes each ref within a node to use the target_model_package of the previous ref as its default target_model_package, rather than None as I believe was intended.

While based on my reading of the docs and code I don't believe this is the intended behavior, if it is I would argue it should be changed. Having the order in which refs occur change the model they point at could for instance cause a stylistic change like refactoring a query to move part of it into a CTE to have a functional impact.

If this is indeed an issue I'd like to volunteer to make a PR for it!

Issue description

Profile type "redshift"

Results

(dbt-test-3)  ~/Test/proj_one  dbt docs generate
Running with dbt=0.13.1
Encountered an error:
Compilation Error in model model_one (models/example/model_one.sql)
  Model 'model.proj_one.model_one' depends on model 'model_two' in package 'proj_two' which was not found or is disabled

System information

The output of dbt --version:

installed version: 0.13.1
   latest version: 0.13.1

Up to date!

The operating system you're running on:
Mac OS Mojave 10.14
The python version you're using (probably the output of python --version)
Python 3.6.2

Steps to reproduce

  1. dbt init two fresh projects, in this case we're calling them proj_one and proj_two.
  2. change the "name" of these projects in the dbt_project.yml file to "proj_one" and "proj_two" respectively
    3.delete the default example models in the models folder
  3. create a file model_one.sql in the models folder of "proj_one" with this definition "select * from {{ref('proj_two', 'model_a')}}, {{ref('model_two')}}"
  4. create a second file in the models folder of "proj_one" named "model_two.sql" with the contents "Select 1"
  5. create a file in the models folder of "proj_two" with the name "model_a.sql" with the contents "select 1"
  6. in proj_one, define a packages.yml file with these contents """
    packages:
  • local: ../proj2
    """
  1. run dbt deps in proj_one to install proj_two as a package
  2. run dbt docs generate in proj_one. This will raise the error " Model 'model.proj_one.model_one' depends on model 'model_two' in package 'proj_two' which was not found or is disabled". This is incorrect, as the ref for model_two does not specify a project, so the current project should be used by default. Dbt instead appears to be using the project of the most recent ref.
@drewbanin
Copy link
Contributor

drewbanin commented Jun 5, 2019

Hey @tbescherer - thanks for the very comprehensive report! I think you're 100% right - we should be resetting the target_model_package inside of the loop over node.refs. I'd definitely welcome a PR for this change - don't hesitate to let me know how I can help!

@drewbanin drewbanin added the bug Something isn't working label Jun 5, 2019
@drewbanin drewbanin added this to the Louisa May Alcott milestone Jun 5, 2019
@drewbanin drewbanin added the good_first_issue Straightforward + self-contained changes, good for new contributors! label Jun 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good_first_issue Straightforward + self-contained changes, good for new contributors!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants