Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new config to filter specific users from dropdown lists #21515

Conversation

dpgaspar
Copy link
Member

@dpgaspar dpgaspar commented Sep 19, 2022

SUMMARY

Introduces a new config key named EXCLUDE_USERS_FROM_LISTS with a list of usernames to exclude from all UI
dropdown user list, owners, created by filters etc.
This is useful for excluding service user's like an initial admin and/or the thumbnail user.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Sep 19, 2022

Codecov Report

Merging #21515 (0d62500) into master (7d2f07e) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master   #21515   +/-   ##
=======================================
  Coverage   66.67%   66.68%           
=======================================
  Files        1793     1793           
  Lines       68493    68513   +20     
  Branches     7275     7275           
=======================================
+ Hits        45665    45685   +20     
  Misses      20966    20966           
  Partials     1862     1862           
Flag Coverage Δ
hive 53.09% <80.00%> (+0.01%) ⬆️
mysql 78.21% <100.00%> (+0.01%) ⬆️
postgres 78.28% <100.00%> (+0.01%) ⬆️
presto 52.99% <80.00%> (+0.01%) ⬆️
python 81.42% <100.00%> (+0.01%) ⬆️
sqlite 76.78% <100.00%> (+0.01%) ⬆️
unit 51.07% <80.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/charts/api.py 86.03% <100.00%> (+0.05%) ⬆️
superset/config.py 91.61% <100.00%> (+0.02%) ⬆️
superset/dashboards/api.py 92.52% <100.00%> (+0.02%) ⬆️
superset/datasets/api.py 87.29% <100.00%> (ø)
superset/queries/api.py 100.00% <100.00%> (ø)
superset/reports/api.py 90.22% <100.00%> (ø)
superset/security/manager.py 95.75% <100.00%> (+0.01%) ⬆️
superset/views/filters.py 100.00% <100.00%> (ø)
superset/db_engine_specs/pinot.py 97.36% <0.00%> (ø)
superset/connectors/sqla/models.py 90.60% <0.00%> (ø)
... and 2 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but a slight nit regarding the naming of the config.

superset/config.py Outdated Show resolved Hide resolved
tests/integration_tests/base_api_tests.py Outdated Show resolved Hide resolved
@ktmud
Copy link
Member

ktmud commented Sep 19, 2022

Should this be an attribute for the User model instead, or potentially a config/method in SecurityManager (which users can already override)? We should really try our best to reduce the number of config keys added plus setting this in config would also mean new service accounts cannot be added without a new deployment of Superset.

@dpgaspar
Copy link
Member Author

Should this be an attribute for the User model instead, or potentially a config/method in SecurityManager (which users can already override)? We should really try our best to reduce the number of config keys added plus setting this in config would also mean new service accounts cannot be added without a new deployment of Superset.

Very good point, an extra attribute for the User model could be tricky. I've chose to do both, config key and a security manager method to override. Config key if not None will take precedence.

Copy link
Member

@ktmud ktmud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I'm not a big fan of filtering entities by hard-coded id/names as they are not very scalable. You cannot automate (e.g. syncing account list with an internal access control service) and any change would require code re-deployment.

It seems there is actually a not-very-much-used user_attribute table already exist in Superset that could be used to store such information (whether a user is service account or not). Do you think it'd worth looking into whether we can use this table for filtering instead?

:return: A list of usernames
"""
return []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a list of ids instead of usernames? Internal filtering should preferably use IDs as they are more determinant.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

username is unique so it has the same level of determinism has id, I can't internally identify service users that are dynamically created by id.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A username can be changed, but id cannot right? By definition that makes id more deterministic. It may not happen often in this case, but I've seen things broken in other systems because a username was changed.

Wouldn't you be able to get the user id by just going to the users CRUD views after they are created? The user id is in the /users/edit/:userId URL.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could, but why make an extra query to fetch an ID?
Service users normally have pre-defined usernames, does not seem common practice to me to identify users on a config key by user IDs. There are lot's of cases of pre-defined usernames for service users or special well known users: postgres, root, rdsadmin, rdsrepladmin, operator, backup, guest, anonymous, QSECOFR. We just assume these names will not change.

exclude_users = (
security_manager.get_exclude_users_from_lists()
if current_app.config["EXCLUDE_USERS_FROM_LISTS"] is None
else current_app.config["EXCLUDE_USERS_FROM_LISTS"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not sure if we need two config values. I understand setting a hard-coded EXCLUDE_USERS_FROM_LISTS is easy but so is setting it in a custom security manager.

Even if we do want to keep EXCLUDE_USERS_FROM_LISTS, you can use it in the default get_exclude_users_from_lists so if users set get_exclude_users_from_lists, EXCLUDE_USERS_FROM_LISTS will be ignored. This prevents them from changing both configs and bite themselves down the road.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overriding the method makes it possibly more dynamic and the existing config key makes it easier to discover the functionality itself.
I understand yet another config setting, but relying on configuration hidden behind overriding a method on security manager is harder to discover and forces users to override the security manager (they may have not done it)

Copy link
Member

@ktmud ktmud Sep 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this needs to be "easy to discover"? If someone really want this feature, they will dig into the docs of config setup then hopefully find some mention of this in the Security Manager section? If we really want to keep this config value for ease of use, then maybe there is no value in adding a security manager solution as it just adds more confusion.

Copy link
Member

@ktmud ktmud Sep 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think most enterprise users would probably have to have their custom security manager anyway. I understand a config value is easier to set up and would do the work for most people; re-reploying Superset every time a new service account needs to be added also doesn't sound like a big deal since it is likely not frequent. But sooner or later, we'd need to migrate current FAB views for user profiles to React and add proper REST API for users and roles. When the API is added, it'd be only natural to configure things like user types in the Edit User page.

A robust and scalable user layer is critical to Superset's success among enterprise users so it'd be prudent to be more future-proof here. Superset has long suffered from not having a built-in user and access control layer, which IMO had made access control in Superset more complex than it needs to be and hindered the development of features like RBAC. Current overrides on top of FAB are hacky and error-prone. Dynamic string-based permissions also feel unreliable. Not the scope of this PR, but I really hope there would be a discussion on what the user and auth models in Superset should look like in the far future.

I'm not opposed to merging this PR as it since it gets the job done, just thought, in general, this area needs a little bit more love.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the majority of your points. We have some initiatives for discussing string based data access permissions on the near future.

@dpgaspar dpgaspar merged commit ab7cfec into apache:master Sep 29, 2022
@dpgaspar dpgaspar deleted the danielgaspar/sc-46720/implement-option-on-apache-superset-to-remove branch September 29, 2022 11:30
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants