Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error deploying project because dbt model names in a manifest.json are not always unique #8066

Closed
owlas opened this issue Nov 23, 2023 · 4 comments · Fixed by #8390
Closed
Assignees
Labels
⚙️ backend This issue requires changes to the backend 🐛 bug Something isn't working as expected. This can be an error or an unexpected user experience released

Comments

@owlas
Copy link
Collaborator

owlas commented Nov 23, 2023

Description

Lightdash assumes that dbt model names are unique. This was true until this PR was merged: dbt-labs/dbt-core#7374 - with this change, Project A can have model users and also have a dependency on Project B with a model called users

The resulting manifest will have two models with the name users but they'll have different unique ids model.project_a.users and model.project_b.users

When this happens, Lightdash throws an error when deploying a project project.

Screenshot 2023-12-15 at 16 01 45
@owlas owlas added the 🐛 bug Something isn't working as expected. This can be an error or an unexpected user experience label Nov 23, 2023
@owlas owlas changed the title dbt model names are not always unique dbt model names in a manifest.json are not always unique Nov 23, 2023
@ZeRego ZeRego added the ⚙️ backend This issue requires changes to the backend label Dec 15, 2023
@ZeRego ZeRego changed the title dbt model names in a manifest.json are not always unique Error deploying project because dbt model names in a manifest.json are not always unique Dec 15, 2023
@TuringLovesDeathMetal
Copy link
Contributor

There is also versioning models in dbt which causes this issue! https://docs.getdbt.com/docs/collaborate/govern/model-versions#how-to-create-a-new-version-of-a-model

@TuringLovesDeathMetal
Copy link
Contributor

We should potentially make the unique ID a different thing than the name (before, teh name used to be unique, but now it's not anymore - we should update the model's unique ID to be something other than the name. E.g. project.model_name)

@ZeRego ZeRego self-assigned this Dec 22, 2023
@ZeRego ZeRego moved this from 📝 To-Do to 🏗 In progress in Selected for Development Dec 22, 2023
@ZeRego
Copy link
Contributor

ZeRego commented Dec 22, 2023

In this PR I'm avoiding this error by discarding/ignoring models with duplicate names. It will only use the first model for each name.

This fix is focused on not throwing an error to unblock users that deploy their project via the UI which deploys all their models. We can open a new ticket to support multiple models with the same name.

@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Selected for Development Jan 2, 2024
@lightdash-bot
Copy link
Collaborator

🎉 This issue has been resolved in version 0.921.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⚙️ backend This issue requires changes to the backend 🐛 bug Something isn't working as expected. This can be an error or an unexpected user experience released
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants