Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: PythonVirtualenvOperator crashes if any python_callable function is defined in the same source as DAG #37165

Merged
merged 9 commits into from
Feb 7, 2024
Merged

fix: PythonVirtualenvOperator crashes if any python_callable function is defined in the same source as DAG #37165

merged 9 commits into from
Feb 7, 2024

Conversation

rawwar
Copy link
Collaborator

@rawwar rawwar commented Feb 4, 2024

closes: #35529

Added a check to see if the python_callable's module name is same as the modified unique path. This helps us to identify that the python callable is in the same module as the dag.

@rawwar
add checks
f8a1ea9 
Signed-off-by: kalyanr <kalyan.ben10@live.com>
@boring-cyborg boring-cyborg bot added the area:core-operators Operators, Sensors and hooks within Core Airflow label Feb 4, 2024
rawwar
rawwar commented Feb 4, 2024
View reviewed changes
airflow/utils/file.py Outdated Show resolved Hide resolved
rawwar
rawwar commented Feb 4, 2024
View reviewed changes
airflow/operators/python.py Outdated Show resolved Hide resolved
@rawwar
exception handling
69c2e17 
Signed-off-by: kalyanr <kalyan.ben10@live.com>
@rawwar rawwar marked this pull request as draft February 4, 2024 15:01
@rawwar rawwar mentioned this pull request Feb 6, 2024
Closed
2 tasks
@potiuk
Copy link
Member

potiuk commented Feb 6, 2024

No specific comments for now, this is the direction that I think makes sense.

@rawwar rawwar marked this pull request as ready for review February 7, 2024 01:02
@rawwar
Copy link
Collaborator Author

rawwar commented Feb 7, 2024

@potiuk , Please consider reviewing the PR. Thanks

@eladkal eladkal added this to the Airflow 2.8.2 milestone Feb 7, 2024
@eladkal eladkal added the type:bug-fix Changelog: Bug Fixes label Feb 7, 2024
@potiuk
Copy link
Member

potiuk commented Feb 7, 2024

Nice

@potiuk potiuk merged commit e75522b into apache:main Feb 7, 2024
57 checks passed
jedcunningham pushed a commit that referenced this pull request Feb 9, 2024
@rawwar @jedcunningham
fix: PythonVirtualenvOperator crashes if any python_callable function…
f841e70 
… is defined in the same source as DAG (#37165)

---------

Signed-off-by: kalyanr <kalyan.ben10@live.com>
(cherry picked from commit e75522b)
potiuk pushed a commit that referenced this pull request Feb 13, 2024
@rawwar @potiuk
fix: PythonVirtualenvOperator crashes if any python_callable function…
03fbb26 
… is defined in the same source as DAG (#37165)

---------

Signed-off-by: kalyanr <kalyan.ben10@live.com>
(cherry picked from commit e75522b)
ephraimbuddy pushed a commit that referenced this pull request Feb 22, 2024
@rawwar @ephraimbuddy
fix: PythonVirtualenvOperator crashes if any python_callable function…
571fa5c 
… is defined in the same source as DAG (#37165)

---------

Signed-off-by: kalyanr <kalyan.ben10@live.com>
(cherry picked from commit e75522b)
@ephraimbuddy ephraimbuddy mentioned this pull request Feb 22, 2024
Closed
68 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk approved these changes

@kaxil kaxil Awaiting requested review from kaxil

@XD-DENG XD-DENG Awaiting requested review from XD-DENG XD-DENG is a code owner

@ashb ashb Awaiting requested review from ashb ashb is a code owner

Assignees
No one assigned
Labels
area:core-operators Operators, Sensors and hooks within Core Airflow type:bug-fix Changelog: Bug Fixes
Projects
None yet
Milestone
Airflow 2.8.2
Development

Successfully merging this pull request may close these issues.

PythonVirtualenvOperator crashes if any python_callable function is defined in the same source as DAG
3 participants
@rawwar @potiuk @eladkal