Bugfix: dag_bag.get_dag should not raise exception

[ephraimbuddy]

get_dag raising exception is breaking many parts of the codebase.
The usage in code suggests that it should return None if a dag is not
found. There are about 30 usages expecting it to return None if a dag
is not found. A missing dag errors out in the UI instead of returning
a message that DAG is missing.

This PR returns None when a dag is not found in SerializedDagModel instead of raising exception

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

[ephraimbuddy]

Also, error for missing DAG now returns a REST API not found error:

{
  "detail": null,
  "status": 404,
  "title": "DAG 'dag_pod_operatorxco' not found in serialized_dag table",
  "type": "http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/latest/stable-rest-api-ref.html#section/Errors/NotFound"
}

I think this PR #18523 caused it

[uranusjr]

The API has been returning 404 for quite some time (and IMO it's the correct behaviour). #18523 only refactored the implementation to raise 404 in a different way.

[ephraimbuddy]

The API has been returning 404 for quite some time (and IMO it's the correct behaviour). #18523 only refactored the implementation to raise 404 in a different way.

I meant on the webserver. Sorry I didn't make that clear. If you change the URL for the dag on tree/graph view such that the dag Id is not in SerializedDagModel. You will get the above error which I think should be for REST API

[ephraimbuddy]

Another option would be to use try_except in all the places we called get_dag

[ashb]

Can you explain when this happens -- when is there a row in the DAG table not not in the SerilaizedDag table?

[ephraimbuddy]

When a DAG is deleted, trying to view the deleted DAG in the UI raises an exception. This is expected but because it breaks the UI instead of showing a simple error message that the DAG is missing, it looks like an error.

On check, I discovered that there are many other places get_dag was used expecting it to return None if there’s no dag and it could raise exceptions in those places. I tried using try/except to guard the usages but it was many and I then decided to remove it and return none instead of exception