Webserver returns 500 for POST requests to api/dag/*/dagrun from anonymous user

[eliskovets]

Apache Airflow version

2.7.3

What happened

After upgrade to 2.7.3 I noticed that webserver returns 500 for POST requests with a note field. After some investigation it seems that None in user_id converted to str causing it:

[SQL: INSERT INTO dag_run_note (user_id, dag_run_id, content, created_at, updated_at) VALUES (%(user_id)s, %(dag_run_id)s, %(content)s, %(created_at)s, %(updated_at)s)]
[parameters: {'user_id': 'None', 'dag_run_id': 2944, 'content': 'test', 'created_at': datetime.datetime(2023, 12, 7, 17, 42, 15, 390085, tzinfo=Timezone('UTC')), 'updated_at': datetime.datetime(2023, 12, 7, 17, 42, 15, 390096, tzinfo=Timezone('UTC'))}]
rundag [07/Dec/2023:17:42:15 +0000] "POST /api/v1/dags/*/dagRuns HTTP/1.1" 500 1539 "-" "python-requests/2.31.0"

That's probably not an issue of Airflow itself, but SQLAlchemy 1.4. it's probably fixed in 2.0.

What you think should happen instead

Dagrun should be successfully registered by airflow and note should be saved into dag_run_note table.

How to reproduce

1. Disable Auth to allow unauthorized access to the API in webserver_config.py

AUTH_ROLE_PUBLIC = "Admin"

2. Try to schedule a run via DAGRun API

import requests
dag_run_url="https://airflow/api/v1/dags/test/dagRuns"
r = requests.post(
    url=dag_run_url, json={"dag_run_id": "test_me", "note":"note", "conf": {}}
)
assert r.status_code == 500

print(r.text)

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

Docker image: apache/airflow:2.7.3-python3.9
DB: Postgresql

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[pateash]

I can take this up if needed.
@potiuk @eladkal

[pateash]

Hi @eliskovets,
i am not able to repro it using
AUTH_ROLE_PUBLIC = "Admin"

as per my knowledge it will only affect UI access not API.
I had to disable API authentication in airflow.cfg using
auth_backends = airflow.api.auth.backend.default

But that's not the concern here.

[eliskovets]

Hi @pateash ,
Sorry, forgot to tell you, that I also have "AIRFLOW__API__AUTH_BACKENDS": "airflow.api.auth.backend.default" set.

[pateash]

RCA ( its reproducable in main branch as well ie. 2.9-dev )

user_id is coming as 'None':,
The error message indicates that user_id is being passed as the string 'None', which is invalid for an integer field in the database. This is happening because the user context is not properly set due to disabled authentication.

[pateash]

The issue here is , user_id is a foreign key from ab_user table,
so what should we add in here?
although we can insert add null.

Need your input @potiuk, @eladkal here.

I am interested in how it used to work before 2.7 @eliskovets

[pateash]

looks like we should be able to insert null here,
moving forward by inserting null, we can revisit if there is any objection.

[eliskovets]

@pateash i believe before 2.7 notes weren't a separate table, that's why it worked before.