Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Add support to DbtSourceGcpCloudRunJob for on_warning_callback #1506

Open
1 task done
anai-s opened this issue Feb 4, 2025 · 1 comment
Open
1 task done
Labels
area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc bug Something isn't working triage-needed Items need to be reviewed / assigned to milestone

Comments

@anai-s
Copy link
Contributor

anai-s commented Feb 4, 2025

Astronomer Cosmos Version

1.8.0

dbt-core version

1.9.0

Versions of dbt adapters

I have the following message error recently:

Broken DAG (dags/cosmos_cloud_run_dag.py): Traceback (most recent call last): File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 499, in apply_defaults result = func(self, **kwargs, default_args=default_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/python3.11/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 947, in __init__ raise AirflowException( airflow.exceptions.AirflowException: Invalid arguments were passed to DbtSourceGcpCloudRunJobOperator (task_id: estat_video_streaming_events_source). Invalid arguments were: **kwargs: {'on_warning_callback': None

Should we had a on_warning_callback argument to the class DbtSourceGcpCloudRunJobOperator as you did to DbtTestGcpCloudRunJobOperator

Image

LoadMode

DBT_LS_MANIFEST

ExecutionMode

DOCKER

InvocationMode

None

airflow version

2.10.2

Operating System

windows 11

If a you think it's an UI issue, what browsers are you seeing the problem on?

No response

Deployment

Google Cloud Composer

Deployment details

No response

What happened?

A bug happened!

Relevant log output

How to reproduce

ExecutionMode.GCP_CLOUD_RUN_JOB
source_rendering_behavior=SourceRenderingBehavior.WITH_TESTS_OR_FRESHNESS
I don't have any callback function imported in the DAG.

Anything else :)?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Contact Details

No response

@anai-s anai-s added bug Something isn't working triage-needed Items need to be reviewed / assigned to milestone labels Feb 4, 2025
@dosubot dosubot bot added the area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc label Feb 4, 2025
@tatiana
Copy link
Collaborator

tatiana commented Feb 4, 2025

Thanks for raising this, @anai-s , we'd love contributions in this area.

Recently, @LuigiCerone made a similar contribution, but on the DbtSourceKubernetesOperator:
#1501

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc bug Something isn't working triage-needed Items need to be reviewed / assigned to milestone
Projects
None yet
Development

No branches or pull requests

2 participants