Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create FAB provider and move FAB auth manager in it #35926

Merged
merged 49 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
9989fb1
Move files from core Airflow to FAB provider
vincbeck Nov 23, 2023
7437707
Move tests
vincbeck Nov 23, 2023
430c9a4
Docs
vincbeck Nov 23, 2023
5a1bb6d
Pre-install FAB provider
vincbeck Nov 24, 2023
a88ca36
Fix `provider.yaml` file
vincbeck Nov 28, 2023
280d658
Add title to doc
vincbeck Nov 28, 2023
d840c45
Fix circular dependency
vincbeck Nov 28, 2023
246a128
Fix test
vincbeck Nov 28, 2023
a9a2938
Fix tests
vincbeck Nov 28, 2023
e030a5b
Move tests
vincbeck Nov 28, 2023
d609e87
Add missing tests
vincbeck Nov 28, 2023
de9e2de
Merge branch 'main' into vincbeck/fab_provider
vincbeck Nov 28, 2023
589aabe
Mark test as DB test
vincbeck Nov 29, 2023
d873373
Fix documentation link
vincbeck Nov 29, 2023
e437097
Merge branch 'main' into vincbeck/fab_provider
vincbeck Nov 29, 2023
8f99406
Revert "Pre-install FAB provider"
vincbeck Nov 29, 2023
b5bd539
Fix `provider.yaml` file
vincbeck Nov 29, 2023
118752b
Update docs/apache-airflow-providers-fab/index.rst
vincbeck Nov 30, 2023
3e2b9b7
Update docs/apache-airflow-providers-fab/index.rst
vincbeck Nov 30, 2023
07aa863
Merge branch 'main' into vincbeck/fab_provider
vincbeck Nov 30, 2023
02cd32c
Add `google-re` as dependency of Fab
vincbeck Nov 30, 2023
d9fb908
Fix doc
vincbeck Nov 30, 2023
a1b1746
Fix doc
vincbeck Nov 30, 2023
396bf7b
Make FAB provider depends on Airflow 2.8.0
vincbeck Nov 30, 2023
25d972e
Update dependencies
vincbeck Nov 30, 2023
7a68ce7
Update dependencies
vincbeck Nov 30, 2023
7b30f8a
Fix dependencies
vincbeck Nov 30, 2023
d1919ca
Remove Fab provider from Airflow 2.5 compatibility check
vincbeck Nov 30, 2023
c958363
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 1, 2023
49cb6c9
Bump up flask-appbuilder to 4.3.10
vincbeck Dec 1, 2023
baf9f79
Update airflow/providers/fab/auth_manager/security_manager/override.py
vincbeck Dec 1, 2023
24848e1
Remove side effects from tests
vincbeck Dec 1, 2023
1b65d43
Fix doc
vincbeck Dec 1, 2023
71addbe
Revert "Revert "Pre-install FAB provider""
vincbeck Dec 4, 2023
b50ffa4
Add FAB as chicken-egg provider
vincbeck Dec 4, 2023
2382d1a
Revert "Revert "Revert "Pre-install FAB provider"""
vincbeck Dec 4, 2023
0b4a990
Add fab provider as installed provider
vincbeck Dec 4, 2023
55e883c
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 5, 2023
c45b7a7
Fix static check
vincbeck Dec 5, 2023
c45d51e
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 5, 2023
4524aab
Bump up FAB min version of Airflow to 2.9.0
vincbeck Dec 5, 2023
6003e31
Revert "Bump up FAB min version of Airflow to 2.9.0"
vincbeck Dec 5, 2023
b0934b1
Revert "Revert "Bump up FAB min version of Airflow to 2.9.0""
vincbeck Dec 5, 2023
2d707f4
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 6, 2023
80a3142
Revert "Revert "Revert "Bump up FAB min version of Airflow to 2.9.0"""
vincbeck Dec 6, 2023
6d13d3e
Revert "Revert "Revert "Revert "Bump up FAB min version of Airflow to…
vincbeck Dec 6, 2023
0641c57
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 8, 2023
8f1b089
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 11, 2023
83354b4
Merge branch 'main' into vincbeck/fab_provider
vincbeck Dec 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/airflow_providers_bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ body:
- docker
- elasticsearch
- exasol
- fab
- facebook
- ftp
- github
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -517,7 +517,6 @@ repos:
exclude: >
(?x)
^airflow/api_connexion/openapi/v1.yaml$|
^airflow/auth/managers/fab/security_manager/|
^airflow/cli/commands/webserver_command.py$|
^airflow/cli/commands/internal_api_command.py$|
^airflow/config_templates/|
Expand All @@ -532,6 +531,7 @@ repos:
^airflow/providers/apache/spark/hooks/|
^airflow/providers/apache/spark/operators/|
^airflow/providers/exasol/hooks/exasol.py$|
^airflow/providers/fab/auth_manager/security_manager/|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Non-blocking] Interesting. I wonder why so many are exempt, we should maybe work on getting this list trimmed down in another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is, very often it leads to breaking change (the method/function name needs to be changed). But still, you're right we could rename them and deprecate the others

^airflow/providers/google/cloud/hooks/bigquery.py$|
^airflow/providers/google/cloud/operators/cloud_build.py$|
^airflow/providers/google/cloud/operators/dataproc.py$|
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -713,7 +713,7 @@ apache.kylin, apache.livy, apache.pig, apache.pinot, apache.spark, apache.sqoop,
apprise, arangodb, asana, async, atlas, atlassian.jira, aws, azure, cassandra, celery, cgroups,
cloudant, cncf.kubernetes, cohere, common.io, common.sql, crypto, databricks, datadog, dbt.cloud,
deprecated_api, devel, devel_all, devel_ci, devel_hadoop, dingding, discord, doc, doc_gen, docker,
druid, elasticsearch, exasol, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
druid, elasticsearch, exasol, fab, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
google_auth, grpc, hashicorp, hdfs, hive, http, imap, influxdb, jdbc, jenkins, kerberos, kubernetes,
ldap, leveldb, microsoft.azure, microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql,
mysql, neo4j, odbc, openai, openfaas, openlineage, opensearch, opsgenie, oracle, otel, pagerduty,
Expand Down
2 changes: 1 addition & 1 deletion INSTALL
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ apache.kylin, apache.livy, apache.pig, apache.pinot, apache.spark, apache.sqoop,
apprise, arangodb, asana, async, atlas, atlassian.jira, aws, azure, cassandra, celery, cgroups,
cloudant, cncf.kubernetes, cohere, common.io, common.sql, crypto, databricks, datadog, dbt.cloud,
deprecated_api, devel, devel_all, devel_ci, devel_hadoop, dingding, discord, doc, doc_gen, docker,
druid, elasticsearch, exasol, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
druid, elasticsearch, exasol, fab, facebook, ftp, gcp, gcp_api, github, github_enterprise, google,
google_auth, grpc, hashicorp, hdfs, hive, http, imap, influxdb, jdbc, jenkins, kerberos, kubernetes,
ldap, leveldb, microsoft.azure, microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql,
mysql, neo4j, odbc, openai, openfaas, openlineage, opensearch, opsgenie, oracle, otel, pagerduty,
Expand Down
8 changes: 4 additions & 4 deletions airflow/api/auth/backend/basic_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,23 +17,23 @@
"""
This module is deprecated.

Please use :mod:`airflow.auth.managers.fab.api.auth.backend.basic_auth` instead.
Please use :mod:`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth` instead.
"""
from __future__ import annotations

import warnings
from typing import TYPE_CHECKING, Any, Callable

import airflow.auth.managers.fab.api.auth.backend.basic_auth as fab_basic_auth
import airflow.providers.fab.auth_manager.api.auth.backend.basic_auth as fab_basic_auth
from airflow.exceptions import RemovedInAirflow3Warning

if TYPE_CHECKING:
from airflow.auth.managers.fab.models import User
from airflow.providers.fab.auth_manager.models import User

CLIENT_AUTH: tuple[str, str] | Any | None = None

warnings.warn(
"This module is deprecated. Please use `airflow.auth.managers.fab.api.auth.backend.basic_auth` instead.",
"This module is deprecated. Please use `airflow.providers.fab.auth_manager.api.auth.backend.basic_auth` instead.",
RemovedInAirflow3Warning,
stacklevel=2,
)
Expand Down
2 changes: 1 addition & 1 deletion airflow/api/auth/backend/kerberos_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ def requires_authentication(function: T, find_user: Callable[[str], BaseUser] |
if not find_user:
warnings.warn(
"This module is deprecated. Please use "
"`airflow.auth.managers.fab.api.auth.backend.kerberos_auth` instead.",
"`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth` instead.",
RemovedInAirflow3Warning,
stacklevel=2,
)
Expand Down
4 changes: 2 additions & 2 deletions airflow/api_connexion/endpoints/forward_to_fab_endpoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from typing import TYPE_CHECKING

from airflow.api_connexion.exceptions import BadRequest
from airflow.auth.managers.fab.api_endpoints import role_and_permission_endpoint, user_endpoint
from airflow.providers.fab.auth_manager.api_endpoints import role_and_permission_endpoint, user_endpoint
from airflow.www.extensions.init_auth_manager import get_auth_manager

if TYPE_CHECKING:
Expand All @@ -37,7 +37,7 @@ def _require_fab(func: Callable) -> Callable:
"""

def inner(*args, **kwargs):
from airflow.auth.managers.fab.fab_auth_manager import FabAuthManager
from airflow.providers.fab.auth_manager.fab_auth_manager import FabAuthManager

auth_mgr = get_auth_manager()
if not isinstance(auth_mgr, FabAuthManager):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from marshmallow import Schema, fields
from marshmallow_sqlalchemy import SQLAlchemySchema, auto_field

from airflow.auth.managers.fab.models import Action, Permission, Resource, Role
from airflow.providers.fab.auth_manager.models import Action, Permission, Resource, Role


class ActionSchema(SQLAlchemySchema):
Expand Down
2 changes: 1 addition & 1 deletion airflow/api_connexion/schemas/user_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

from airflow.api_connexion.parameters import validate_istimezone
from airflow.api_connexion.schemas.role_and_permission_schema import RoleSchema
from airflow.auth.managers.fab.models import User
from airflow.providers.fab.auth_manager.models import User


class UserCollectionItemSchema(SQLAlchemySchema):
Expand Down
2 changes: 1 addition & 1 deletion airflow/api_connexion/security.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def requires_access(permissions: Sequence[tuple[str, str]] | None = None) -> Cal
RemovedInAirflow3Warning,
stacklevel=2,
)
from airflow.auth.managers.fab.decorators.auth import _requires_access_fab
from airflow.providers.fab.auth_manager.decorators.auth import _requires_access_fab

return _requires_access_fab(permissions)

Expand Down
55 changes: 19 additions & 36 deletions airflow/auth/managers/fab/api/auth/backend/basic_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,55 +14,38 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""Basic authentication backend."""
from __future__ import annotations
"""
This module is deprecated.

from functools import wraps
from typing import TYPE_CHECKING, Any, Callable, TypeVar, cast
Please use :mod:`airflow.providers.fab.auth_manager.api.auth.backend.basic_auth` instead.
"""
from __future__ import annotations

from flask import Response, request
from flask_appbuilder.const import AUTH_LDAP
from flask_login import login_user
import warnings
from typing import TYPE_CHECKING, Any, Callable

from airflow.utils.airflow_flask_app import get_airflow_app
import airflow.providers.fab.auth_manager.api.auth.backend.basic_auth as fab_basic_auth
from airflow.exceptions import RemovedInAirflow3Warning

if TYPE_CHECKING:
from airflow.auth.managers.fab.models import User
from airflow.providers.fab.auth_manager.models import User

CLIENT_AUTH: tuple[str, str] | Any | None = None

T = TypeVar("T", bound=Callable)
warnings.warn(
"This module is deprecated. Please use `airflow.providers.fab.auth_manager.api.auth.backend.basic_auth` instead.",
RemovedInAirflow3Warning,
stacklevel=2,
)


def init_app(_):
"""Initialize authentication backend."""
fab_basic_auth.init_app(_)


def auth_current_user() -> User | None:
"""Authenticate and set current user if Authorization header exists."""
auth = request.authorization
if auth is None or not auth.username or not auth.password:
return None

ab_security_manager = get_airflow_app().appbuilder.sm
user = None
if ab_security_manager.auth_type == AUTH_LDAP:
user = ab_security_manager.auth_user_ldap(auth.username, auth.password)
if user is None:
user = ab_security_manager.auth_user_db(auth.username, auth.password)
if user is not None:
login_user(user, remember=False)
return user


def requires_authentication(function: T):
"""Decorate functions that require authentication."""
return fab_basic_auth.auth_current_user()

@wraps(function)
def decorated(*args, **kwargs):
if auth_current_user() is not None:
return function(*args, **kwargs)
else:
return Response("Unauthorized", 401, {"WWW-Authenticate": "Basic"})

return cast(T, decorated)
def requires_authentication(function: Callable):
return fab_basic_auth.requires_authentication(function)
27 changes: 15 additions & 12 deletions airflow/auth/managers/fab/api/auth/backend/kerberos_auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,28 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
"""
This module is deprecated.

Please use :mod:`airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth` instead.
"""
from __future__ import annotations

import logging
from functools import partial
import warnings
from typing import Any

from requests_kerberos import HTTPKerberosAuth

from airflow.api.auth.backend.kerberos_auth import (
init_app as base_init_app,
requires_authentication as base_requires_authentication,
)
from airflow.utils.airflow_flask_app import get_airflow_app

log = logging.getLogger(__name__)
import airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth as fab_kerberos_auth
from airflow.exceptions import RemovedInAirflow3Warning

CLIENT_AUTH: tuple[str, str] | Any | None = HTTPKerberosAuth(service="airflow")

init_app = base_init_app
requires_authentication = partial(
base_requires_authentication, find_user=get_airflow_app().appbuilder.sm.find_user
warnings.warn(
"This module is deprecated. Please use `airflow.providers.fab.auth_manager.api.auth.backend.kerberos_auth` instead.",
RemovedInAirflow3Warning,
stacklevel=2,
)

init_app = fab_kerberos_auth.init_app
requires_authentication = fab_kerberos_auth.requires_authentication
Loading