Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(filter-set): Add filterset resource #14015

Merged
merged 73 commits into from
Sep 23, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
b7865ad
Add filterset resource
Apr 8, 2021
085a9f7
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Apr 8, 2021
60ab14f
fix: fix pre-commit
Apr 8, 2021
8fddfd0
add tests
Apr 11, 2021
68aa395
add tests and fixes based of failures
Apr 13, 2021
9ca86fb
Fix pre-commit errors
Apr 14, 2021
f292dea
chore init filterset resource under ff constraint
Apr 14, 2021
d7e4b7b
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Apr 18, 2021
2a782f9
Fix migration conflicts
Apr 18, 2021
8cd164e
Fix pylint and migrations issues
Apr 18, 2021
3ab9609
Fix pylint and migrations issues
Apr 18, 2021
6c2ed5c
Fix pylint and migrations issues
Apr 18, 2021
4c87d47
Fix pylint and migrations issues
Apr 18, 2021
94abd23
Fix pylint and migrations issues
Apr 18, 2021
4037097
Fix pylint and migrations issues
Apr 18, 2021
4b3fd59
Fix pylint and migrations issues
Apr 18, 2021
733a537
Fix pylint and migrations issues
Apr 18, 2021
eac8526
Fix pylint and migrations issues
Apr 18, 2021
89e8bee
Fix pylint and migrations issues
Apr 18, 2021
3a7f080
Fix pylint and migrations issues
Apr 18, 2021
8c367cc
add tests and fixes based of failures
Apr 27, 2021
28d28c8
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Apr 27, 2021
4aa4314
Fix missing license
Apr 28, 2021
ad27fca
Merge branch 'master' into feat/fileter_set
May 3, 2021
6d2d23e
fix down revision
May 3, 2021
ad54807
Merge branch 'master' into feat/fileter_set
May 25, 2021
ff6ca29
update down_revision
May 25, 2021
2452389
fix: update down_revision
May 25, 2021
2c7d2c4
chore: add description to migration
May 25, 2021
4aff6b1
fix: type
amitmiran137 Jun 6, 2021
145b07d
refactor: is_user_admin
amitmiran137 Jun 6, 2021
73ab92f
fix: use get_public_role
amitmiran137 Jun 6, 2021
af8e35f
fix: move import to the relevant location
Jun 6, 2021
48b84e8
chore: add openSpec api schema
Jun 6, 2021
f7526b8
chore: cover all openspec API
Jun 6, 2021
7ed5321
fix: pre-commit and lint
Jun 6, 2021
dbb0ae0
fix: put and post schemas
Jun 6, 2021
033734d
fix: undo superset_test_config.py
Jun 6, 2021
39f4d70
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Jun 6, 2021
ebb1405
fix: limit filterSetsApi to include_route_methods = {"get_list", "put…
Jun 7, 2021
16c8a18
renaming some params
Jun 8, 2021
a87172a
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Jun 8, 2021
94d8478
chore: add debug in test config
Jun 8, 2021
6b4b3d7
fix: rename database to different name
Jun 8, 2021
9078677
fix: try to make conftest.py harmless
Jun 8, 2021
e84dd26
fix: pre-commit
Jun 8, 2021
0465379
Merge branch 'master' into feat/fileter_set
Aug 24, 2021
1c28bf8
fix: new down_revision ref
Aug 24, 2021
a6b6c11
fix: bad ref
Aug 24, 2021
645e371
fix: bad ref 2
Aug 24, 2021
e2cb552
fix: bad ref 3
Aug 24, 2021
adfe513
fix: add api in initiatior
Aug 24, 2021
abef564
fix: open spec
Aug 25, 2021
930863b
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Aug 26, 2021
ee7c4e5
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Aug 29, 2021
d500272
fix: convert name to str to include int usecases
Aug 29, 2021
6762eb5
Merge remote-tracking branch 'upstream/master' into feat/fileter_set
Aug 30, 2021
1dfb029
fix: pylint
Aug 30, 2021
07792fa
fix: pylint
Aug 30, 2021
55dbfa4
Update superset/common/request_contexed_based.py
amitmiran137 Sep 7, 2021
a9ea304
Merge branch 'upstream_master' into feat/fileter_set
ofekisr Sep 13, 2021
568b3a5
chore: resolve PR comments
ofekisr Sep 13, 2021
6359817
chore: resolve PR comments
ofekisr Sep 13, 2021
4af6a6a
chore: resolve PR comments
ofekisr Sep 13, 2021
fd6deca
fix failed tests
ofekisr Sep 14, 2021
4b0a9ed
fix pylint
ofekisr Sep 14, 2021
d1a5968
Update conftest.py
ofekisr Sep 19, 2021
e2bf8d7
chore remove BaseCommand to remove abstraction
ofekisr Sep 19, 2021
2d63fe6
chore remove BaseCommand to remove abstraction
ofekisr Sep 19, 2021
2674d9a
chore remove BaseCommand to remove abstraction
ofekisr Sep 19, 2021
414b0c0
chore remove BaseCommand to remove abstraction
ofekisr Sep 19, 2021
9e7f948
Merge branch 'upstream_master' into feat/fileter_set
ofekisr Sep 23, 2021
51e2084
chore fix migration
ofekisr Sep 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions superset/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,9 @@ def init_views(self) -> None:
icon="fa-cog",
)
appbuilder.add_separator("Data")
from superset.dashboards.filter_sets.api import FilterSetRestApi
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved

appbuilder.add_api(FilterSetRestApi)

def init_app_in_ctx(self) -> None:
"""
Expand Down
22 changes: 21 additions & 1 deletion superset/commands/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from typing import Any, Dict, List
from typing import Any, Dict, List, Optional

from flask_babel import lazy_gettext as _
from marshmallow import ValidationError
Expand All @@ -31,6 +31,26 @@ def __repr__(self) -> str:
return repr(self)


class ObjectNotFoundError(CommandException):
status = 404
message_format = "{} {}not found."

def __init__(
self,
object_type: str,
object_id: str = None,
exception: Optional[Exception] = None,
) -> None:
super().__init__(
_(
self.message_format.format(
object_type, '"%s" ' % object_id if object_id else ""
)
),
ofekisr marked this conversation as resolved.
Show resolved Hide resolved
exception,
)


class CommandInvalidError(CommandException):
""" Common base class for Command Invalid errors. """

Expand Down
38 changes: 38 additions & 0 deletions superset/common/request_contexed_based.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from __future__ import annotations

from typing import List, TYPE_CHECKING

from flask import g

from superset import conf, security_manager

if TYPE_CHECKING:
from flask_appbuilder.security.sqla.models import Role


def get_user_roles() -> List[Role]:
if g.user.is_anonymous:
public_role = conf.get("AUTH_ROLE_PUBLIC")
return [security_manager.find_role(public_role)] if public_role else []
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
return g.user.roles


def is_user_admin() -> bool:
user_roles = [role.name.lower() for role in list(get_user_roles())]
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
return "admin" in user_roles
10 changes: 8 additions & 2 deletions superset/dashboards/commands/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
from typing import Optional

from flask_babel import lazy_gettext as _
from marshmallow.validate import ValidationError

Expand All @@ -24,6 +26,7 @@
DeleteFailedError,
ForbiddenError,
ImportFailedError,
ObjectNotFoundError,
UpdateFailedError,
)

Expand All @@ -41,8 +44,11 @@ class DashboardInvalidError(CommandInvalidError):
message = _("Dashboard parameters are invalid.")


class DashboardNotFoundError(CommandException):
message = _("Dashboard not found.")
class DashboardNotFoundError(ObjectNotFoundError):
def __init__(
self, dashboard_id: str = None, exception: Optional[Exception] = None
) -> None:
super().__init__("Dashboard", dashboard_id, exception)


class DashboardCreateFailedError(CreateFailedError):
Expand Down
16 changes: 16 additions & 0 deletions superset/dashboards/filter_sets/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
147 changes: 147 additions & 0 deletions superset/dashboards/filter_sets/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import logging
from flask import Response, request, g
from flask_appbuilder.api import expose, protect, safe, rison, permission_name, merge_response_func, \
get_list_schema, API_ORDER_COLUMNS_RIS_KEY, API_LABEL_COLUMNS_RIS_KEY, \
API_DESCRIPTION_COLUMNS_RIS_KEY, API_LIST_COLUMNS_RIS_KEY, API_LIST_TITLE_RIS_KEY, ModelRestApi
from flask_appbuilder.models.sqla.interface import SQLAInterface
from marshmallow import ValidationError
from superset import is_feature_enabled
from superset.commands.exceptions import ObjectNotFoundError
from superset.dashboards.commands.exceptions import DashboardNotFoundError
from superset.dashboards.filter_sets.commands.exceptions import FilterSetForbiddenError, FilterSetUpdateFailedError, FilterSetDeleteFailedError, FilterSetCreateFailedError, UserIsNotDashboardOwnerError
from superset.dashboards.filter_sets.commands.create import CreateFilterSetCommand
from superset.dashboards.filter_sets.commands.update import UpdateFilterSetCommand
from superset.dashboards.filter_sets.filters import FilterSetFilter
from superset.dashboards.filter_sets.schemas import FilterSetPostSchema, FilterSetPutSchema
from superset.extensions import event_logger
from superset.models.filter_set import FilterSet
from superset.views.base_api import BaseSupersetModelRestApi, statsd_metrics
from superset.dashboards.dao import DashboardDAO
from superset.dashboards.filter_sets.consts import OWNER_OBJECT_FIELD, DASHBOARD_FIELD, \
FILTER_SET_API_PERMISSIONS_NAME, NAME_FIELD, DESCRIPTION_FIELD, OWNER_TYPE_FIELD, OWNER_ID_FIELD, DASHBOARD_ID_FIELD, JSON_METADATA_FIELD

logger = logging.getLogger(__name__)


class FilterSetRestApi(BaseSupersetModelRestApi):
datamodel = SQLAInterface(FilterSet)
resource_name = "dashboard"
class_permission_name = FILTER_SET_API_PERMISSIONS_NAME
allow_browser_login = True
csrf_exempt = True
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
add_exclude_columns = ['id', OWNER_OBJECT_FIELD, DASHBOARD_FIELD]
add_model_schema = FilterSetPostSchema()
edit_model_schema = FilterSetPutSchema()
edit_exclude_columns = ['id', OWNER_OBJECT_FIELD, DASHBOARD_FIELD]
list_columns = ['created_on', 'changed_on', 'created_by_fk', 'changed_by_fk', NAME_FIELD,
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
DESCRIPTION_FIELD, OWNER_TYPE_FIELD, OWNER_ID_FIELD, DASHBOARD_ID_FIELD, JSON_METADATA_FIELD]
show_exclude_columns = [OWNER_OBJECT_FIELD, DASHBOARD_FIELD]
search_columns = ['id', NAME_FIELD, OWNER_ID_FIELD, DASHBOARD_ID_FIELD]
base_filters = [[OWNER_ID_FIELD, FilterSetFilter, '']]

def __init__(self) -> None:
self.datamodel.get_search_columns_list = lambda: []
if is_feature_enabled("THUMBNAILS"):
self.include_route_methods = self.include_route_methods | {"thumbnail"}
super().__init__()

def _init_properties(self) -> None:
super(BaseSupersetModelRestApi, self)._init_properties()

@expose("/<dashboard_id>/filtersets", methods=["GET"])
@protect()
@safe
@permission_name("get")
@rison(get_list_schema)
@merge_response_func(ModelRestApi.merge_order_columns, API_ORDER_COLUMNS_RIS_KEY)
amitmiran137 marked this conversation as resolved.
Show resolved Hide resolved
@merge_response_func(ModelRestApi.merge_list_label_columns, API_LABEL_COLUMNS_RIS_KEY)
@merge_response_func(ModelRestApi.merge_description_columns, API_DESCRIPTION_COLUMNS_RIS_KEY)
@merge_response_func(ModelRestApi.merge_list_columns, API_LIST_COLUMNS_RIS_KEY)
@merge_response_func(ModelRestApi.merge_list_title, API_LIST_TITLE_RIS_KEY)
def get_list(self, dashboard_id: int, **kwargs) -> Response:
if not DashboardDAO.find_by_id(dashboard_id):
return self.response(404, message="dashboard '%s' not found" % dashboard_id)
rison_data = kwargs.setdefault('rison', {})
rison_data.setdefault('filters', [])
rison_data['filters'].append({'col': 'dashboard_id', 'opr': 'eq', 'value': str(dashboard_id)})
return self.get_list_headless(**kwargs)

@expose("/<int:dashboard_id>/filtersets", methods=["POST"])
@protect()
@safe
@statsd_metrics
@event_logger.log_this_with_context(
action=lambda self, *args, **kwargs: f"{self.__class__.__name__}.post",
log_to_statsd=False,
)
def post(self, dashboard_id: int) -> Response:
if not request.is_json:
return self.response_400(message="Request is not JSON")
try:
item = self.add_model_schema.load(request.json)
new_model = CreateFilterSetCommand(g.user, dashboard_id, item).run()
return self.response(201, id=new_model.id, result=item)
except ValidationError as error:
return self.response_400(message=error.messages)
except UserIsNotDashboardOwnerError as error:
return self.response_403()
except FilterSetCreateFailedError as error:
return self.response_400(message=error.message)
except DashboardNotFoundError:
return self.response_404()

@expose("/<dashboard_id>/filtersets/<pk>", methods=["PUT"])
@protect()
@safe
@statsd_metrics
@event_logger.log_this_with_context(
action=lambda self, *args, **kwargs: f"{self.__class__.__name__}.put",
log_to_statsd=False,
)
def put(self, dashboard_id: int, pk: int) -> Response:
if not request.is_json:
return self.response_400(message="Request is not JSON")
try:
item = self.edit_model_schema.load(request.json)
changed_model = UpdateFilterSetCommand(g.user, dashboard_id, pk, item).run()
return self.response(200, id=changed_model.id, result=item)
except ValidationError as error:
return self.response_400(message=error.messages)
except (ObjectNotFoundError, FilterSetForbiddenError, FilterSetUpdateFailedError) as e:
logger.error(e)
return self.response(e.status)


@expose("/<dashboard_id>/filtersets/<pk>", methods=["DELETE"])
@protect()
@safe
@statsd_metrics
@event_logger.log_this_with_context(
action=lambda self, *args, **kwargs: f"{self.__class__.__name__}.delete",
log_to_statsd=False,
)
def delete(self, dashboard_id: int, pk: int) -> Response:
try:
changed_model = UpdateFilterSetCommand(g.user, dashboard_id, pk).run()
return self.response(200, id=changed_model.id)
except ValidationError as error:
return self.response_400(message=error.messages)
except (ObjectNotFoundError, FilterSetForbiddenError, FilterSetDeleteFailedError) as e:
logger.error(e)
return self.response(e.status)
16 changes: 16 additions & 0 deletions superset/dashboards/filter_sets/commands/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
65 changes: 65 additions & 0 deletions superset/dashboards/filter_sets/commands/base.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
import logging
from typing import Optional
from flask_appbuilder.models.sqla import Model
from flask_appbuilder.security.sqla.models import User
from superset.commands.base import BaseCommand
from superset.common.request_contexed_based import is_user_admin
from superset.dashboards.commands.exceptions import DashboardNotFoundError
from superset.dashboards.dao import DashboardDAO
from superset.dashboards.filter_sets.commands.exceptions import FilterSetNotFoundError, FilterSetForbiddenError
from superset.models.dashboard import Dashboard
from superset.models.filter_set import FilterSet
from superset.dashboards.filter_sets.consts import USER_OWNER_TYPE

logger = logging.getLogger(__name__)


class BaseFilterSetCommand(BaseCommand):
_dashboard: Dashboard
_filter_set_id: Optional[int]
_filter_set: Optional[FilterSet]

def __init__(self, user: User, dashboard_id: int):
self._actor = user
self._dashboard_id = dashboard_id

def run(self) -> Model:
pass

def validate(self) -> None:
self._dashboard = DashboardDAO.get_by_id_or_slug(str(self._dashboard_id))
if not self._dashboard:
raise DashboardNotFoundError()

def is_user_dashboard_owner(self) -> bool:
return is_user_admin() or self._dashboard.am_i_owner()

def validate_exist_filter_use_cases_set(self):
if self._filter_set_id:
self._filter_set = self._dashboard.filter_sets.get(self._filter_set_id, None)
if not self._filter_set:
raise FilterSetNotFoundError(str(self._filter_set_id))
self.check_ownership()

def check_ownership(self):
if self._filter_set.owner_type == USER_OWNER_TYPE:
if self._actor.id != self._filter_set.owner_id:
raise FilterSetForbiddenError(str(self._filter_set_id), "The user is not the owner of the filter_set")
elif not self.is_user_dashboard_owner():
raise FilterSetForbiddenError(str(self._filter_set_id), "The user is not an owner of the filter_set's dashboard")
Loading