Fix issue 'No such file or directory' #1097
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Two Astronomer customers reported issues while trying to upgrade to Cosmos 1.5: ``` No such file or directory "/usr/local/aiflow/dags/dbt/dbt_venv/bin/python" ``` The issue is that their dbt project folder had a symbolic link to a file that no longer existed. When Cosmos attempted to iterate over that directory to create a hash representing the cache version, it could not open the file referenced by the symbolic link. This PR reproduces the issue and fixes the behaviour. Users are also advised to check if there are any broken sym links on their dbt folder and delete them. Closes #1096
dosubot
bot
added
the
size:M
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1097 +/- ##
=======================================
Coverage 96.23% 96.24%
=======================================
Files 63 63
Lines 3270 3273 +3
=======================================
+ Hits 3147 3150 +3
Misses 123 123 ☔ View full report in Codecov by Sentry. |
pankajastro
approved these changes
Jul 17, 2024
pankajkoti
reviewed
Jul 17, 2024
pankajkoti
approved these changes
Jul 17, 2024
tatiana
added a commit
that referenced
this pull request
Jul 17, 2024
Two Astronomer customers reported issues while trying to upgrade to Cosmos 1.5: ``` No such file or directory "/usr/local/aiflow/dags/dbt/dbt_venv/bin/python" ``` The issue is that their dbt project folder had a symbolic link to a file that no longer existed. When Cosmos attempted to iterate over that directory to create a hash representing the cache version, it could not open the file referenced by the symbolic link. This PR reproduces the issue and fixes the behavior. Users are also advised to check if there are any broken symlinks on their dbt folder and delete them. We also discourage having Python virtual environments inside the dbt project folder and/or the Airflow DAGs folder. Closes: #1096 (cherry picked from commit 0bafe88)
Merged
tatiana
added a commit
that referenced
this pull request
Jul 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Two Astronomer customers reported issues while trying to upgrade to Cosmos 1.5:
The issue is that their dbt project folder had a symbolic link to a file that no longer existed. When Cosmos attempted to iterate over that directory to create a hash representing the cache version, it could not open the file referenced by the symbolic link.
This PR reproduces the issue and fixes the behavior. Users are also advised to check if there are any broken symlinks on their dbt folder and delete them. We also discourage having Python virtual environments inside the dbt project folder and/or the Airflow DAGs folder.
Closes: #1096