Refactor secrets stores to store all secret metadata in the DB #2193
Conversation
|
Important Auto Review SkippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the To trigger a single review, invoke the WalkthroughThe overall change indicates a significant refactoring of the secrets management system within an application, likely ZenML. The adjustments suggest a shift towards more granular permission controls, particularly for AWS S3 buckets, and the standardization of secret handling across different secret stores. There's a clear move away from broad permissions to a more secure, permission-specific approach. The codebase has been streamlined by removing outdated methods, improving secret handling for SQL stores, and updating the secrets store interface. Changes
*Note: The Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
github-actions
bot
added
internal
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| ) | ||
|
|
||
| return secret_model | ||
| logger.debug(f"Created AWS secret: {aws_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| # longer exists. Here we pretend that the secret does | ||
| # not exist. | ||
| continue | ||
| logger.debug(f"Fetched AWS secret: {aws_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| ) | ||
|
|
||
| return secret_model | ||
| logger.debug(f"Updated AWS secret: {aws_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| @@ -849,3 +397,137 @@ | |||
| raise RuntimeError( | |||
| f"Error deleting secret with ID {secret_id}: {e}" | |||
| ) | |||
|
|
|||
| logger.debug(f"Deleted AWS secret: {aws_secret_id}") | |||
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| workspace=workspace, | ||
| ), | ||
| ) | ||
| logger.debug(f"Created Azure secret: {azure_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| @@ -695,3 +382,90 @@ | |||
| raise KeyError(f"Secret with ID {secret_id} not found") | |||
| except Exception as e: | |||
| raise RuntimeError(f"Failed to delete secret: {str(e)}") from e | |||
|
|
|||
| logger.debug(f"Deleted GCP secret: {gcp_secret_name}") | |||
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| except VaultError as e: | ||
| raise RuntimeError(f"Error creating secret: {e}") | ||
|
|
||
| logger.debug(f"Created HashiCorp Vault secret: {vault_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| @track_decorator(AnalyticsEvent.CREATED_SECRET) | ||
| def create_secret(self, secret: SecretRequest) -> SecretResponse: | ||
| """Creates a new secret. | ||
| logger.debug(f"Fetched HashiCorp Vault secret: {vault_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
|
|
||
| return secret_model | ||
| logger.debug(f"Updated HashiCorp Vault secret: {vault_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
| vault_secret, | ||
| hydrate=hydrate, | ||
| ) | ||
| logger.debug(f"Deleted HashiCorp Vault secret: {vault_secret_id}") |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information
|
@coderabbitai review |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 10
Configuration used: .coderabbit.yaml
Files selected for processing (18)
- docs/book/stacks-and-components/component-guide/artifact-stores/s3.md (1 hunks)
- src/zenml/config/global_config.py (1 hunks)
- src/zenml/config/store_config.py (2 hunks)
- src/zenml/enums.py (1 hunks)
- src/zenml/models/v2/core/secret.py (2 hunks)
- src/zenml/zen_server/deploy/deployer.py (2 hunks)
- src/zenml/zen_stores/base_zen_store.py (7 hunks)
- src/zenml/zen_stores/rest_zen_store.py (8 hunks)
- src/zenml/zen_stores/schemas/secret_schemas.py (4 hunks)
- src/zenml/zen_stores/secrets_stores/aws_secrets_store.py (13 hunks)
- src/zenml/zen_stores/secrets_stores/azure_secrets_store.py (9 hunks)
- src/zenml/zen_stores/secrets_stores/base_secrets_store.py (10 hunks)
- src/zenml/zen_stores/secrets_stores/gcp_secrets_store.py (10 hunks)
- src/zenml/zen_stores/secrets_stores/hashicorp_secrets_store.py (7 hunks)
- src/zenml/zen_stores/secrets_stores/secrets_store_interface.py (2 hunks)
- src/zenml/zen_stores/secrets_stores/sql_secrets_store.py (3 hunks)
- src/zenml/zen_stores/sql_zen_store.py (13 hunks)
- src/zenml/zen_stores/zen_store_interface.py (2 hunks)
Files skipped from review due to trivial changes (1)
- src/zenml/zen_server/deploy/deployer.py
Additional comments: 84
src/zenml/zen_stores/secrets_stores/secrets_store_interface.py (5)
47-57: The method
store_secret_valueshas been renamed and its signature updated. Ensure that all implementations of this interface and calls to this method are updated accordingly.60-71: The method
get_secret_valueshas been refactored from the previousget_secret. Verify that the implementation and usage of this method reflect the new signature and intended behavior.75-88: The
update_secretmethod has been split intoupdate_secret_valuesanddelete_secret_values. Confirm that the logic for updating and deleting secret values is correctly implemented in the respective methods.108-115: A new method
list_secretshas been added. Verify that this method is implemented in all concrete classes and that it adheres to the intended functionality of listing secrets without their values.104-106: The old
delete_secretmethod is deprecated. Ensure that it is no longer used in the codebase and that the newdelete_secret_valuesmethod is used instead.src/zenml/config/store_config.py (1)
- 95-116: The
root_validatormethodvalidate_secrets_storehas been added to validate the secrets store configuration. Verify that this validation logic is correct and that it is being called appropriately during the configuration of a secrets store.src/zenml/enums.py (1)
- 122-128: The
SecretsStoreTypeenum has been modified, with theSQLandRESTvalues replaced by string literals. Confirm that these changes are reflected throughout the codebase and that the removal of the explanatory docstring does not reduce clarity.src/zenml/zen_stores/schemas/secret_schemas.py (5)
52-52: The
valuesfield in theSecretSchemaclass has been changed from a requiredbytestype to an optionalbytestype. Verify that this change is consistent with the new approach to handling secret values and that it does not introduce any regressions.136-146: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [136-157]
The
from_requestmethod in theSecretSchemaclass has been modified to no longer store secret values implicitly. Confirm that the SQL secret store is now responsible for callingstore_secret_valuesto store the values separately.
160-178: The
updatemethod in theSecretSchemaclass has been modified to no longer update secret values implicitly. Ensure that the SQL secret store is now responsible for callingset_secret_valuesto update the values separately.199-258: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [183-210]
The
to_modelmethod in theSecretSchemaclass has been modified to no longer load secret values implicitly. Confirm that the SQL secret store is now responsible for callingget_secret_valuesto load the values separately.
- 218-258: New methods
get_secret_valuesandset_secret_valueshave been added to theSecretSchemaclass to handle secret values explicitly. Verify that these methods are implemented correctly and that they are being used appropriately by the SQL secret store.src/zenml/zen_stores/secrets_stores/sql_secrets_store.py (5)
175-203: The
store_secret_valuesmethod has been added to theSqlSecretsStoreclass. Verify that this method correctly stores the secret values in the database and that it is being called in the appropriate places.205-226: The
get_secret_valuesmethod has been added to theSqlSecretsStoreclass. Confirm that this method correctly retrieves the secret values from the database and that it is being used where necessary.228-243: The
update_secret_valuesmethod has been added to theSqlSecretsStoreclass. Ensure that this method correctly updates the secret values in the database and that it is being called instead of the previous update method.245-263: The
delete_secret_valuesmethod has been added to theSqlSecretsStoreclass. Verify that this method correctly deletes the secret values from the database and that it is being used in place of the previous delete method.269-285: The
list_secretsmethod is deprecated in theSqlSecretsStoreclass. Confirm that this method is no longer being used and that the new method for listing secrets without their values is being used instead.src/zenml/models/v2/core/secret.py (3)
87-97: The
get_secret_values_updatemethod has been added to theSecretUpdateclass. Verify that this method correctly returns a dictionary with the secret values to update and that it is being used appropriately.209-210: The
remove_secretsmethod in theSecretUpdateclass has been extended to include the removal of secret values while keeping the keys. Confirm that this change is consistent with the intended functionality and that it does not introduce any issues.212-218: The
set_secretsmethod has been introduced in theSecretUpdateclass to set the secret values of the secret. Ensure that this method is implemented correctly and that it is being used where necessary.src/zenml/zen_stores/secrets_stores/base_secrets_store.py (10)
14-26: The import statements are standard and necessary for the functionality of the base secrets store.
30-40: The import statements and class imports are appropriate for the secrets store functionality.
53-59: The
BaseSecretsStoreclass is correctly defined as an abstract base class for secrets stores.88-93: The
__init__method for initializing the secrets store is correctly implemented with proper exception handling.150-155: The
get_store_classmethod correctly returns the appropriate class for the secrets store based on the configuration.229-230: The
zen_storeproperty is correctly implemented with proper error handling to ensure the store is initialized.263-270: The
_get_secret_metadataand_verify_secret_metadatamethods are correctly implemented to handle metadata associated with secrets.312-319: The
_create_secret_from_metadatamethod is marked as deprecated and is kept for migration purposes. Ensure that the deprecation is communicated and migration is tracked.355-356: The continuation of the
_create_secret_from_metadatamethod is correctly implemented with proper exception handling.383-386: The
_create_secret_from_metadatamethod correctly returns aSecretResponseobject, encapsulating the secret data.src/zenml/zen_stores/secrets_stores/azure_secrets_store.py (9)
15-20: The import statements are standard and necessary for the functionality of the Azure secrets store.
32-40: The import statements and class imports are appropriate for the Azure secrets store functionality.
44-50: The class imports for
SecretResponseand the logger initialization are correctly implemented.127-138: The
AzureSecretsStoreclass is correctly defined with a comment explaining the implementation highlights.208-230: The
store_secret_valuesmethod is correctly implemented to store secret values in Azure Key Vault.236-256: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [241-273]
The
get_secret_valuesmethod is correctly implemented to retrieve secret values from Azure Key Vault.
291-315: The
update_secret_valuesmethod is correctly implemented to update secret values in Azure Key Vault.326-342: The
delete_secret_valuesmethod is correctly implemented to delete secret values from Azure Key Vault.357-410: The
_convert_azure_secretandlist_secretsmethods are correctly implemented for secret conversion and listing. Ensure that the deprecation is communicated and migration is tracked.src/zenml/zen_stores/secrets_stores/gcp_secrets_store.py (9)
15-29: The import statements are standard and necessary for the functionality of the GCP secrets store.
32-40: The import statements and class imports are appropriate for the GCP secrets store functionality.
44-50: The class imports for
SecretResponseand the logger initialization are correctly implemented.15-29: > Note: This review was outside the patches, and no patch overlapping with it was found. Original lines [127-138]
The
GCPSecretsStoreclass is correctly defined with a comment explaining the implementation highlights.
202-222: The
store_secret_valuesmethod is correctly implemented to store secret values in GCP Secrets Manager.230-263: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [248-273]
The
get_secret_valuesmethod is correctly implemented to retrieve secret values from GCP Secrets Manager.
291-340: The
update_secret_valuesmethod is correctly implemented to update secret values in GCP Secrets Manager.362-375: The
delete_secret_valuesmethod is correctly implemented to delete secret values from GCP Secrets Manager.392-441: The
_convert_gcp_secretandlist_secretsmethods are correctly implemented for secret conversion and listing. Ensure that the deprecation is communicated and migration is tracked.docs/book/stacks-and-components/component-guide/artifact-stores/s3.md (1)
- 223-227: The documentation now correctly advises on granting minimal necessary permissions (
s3:PutObject,s3:GetObject,s3:ListBucket,s3:DeleteObject) for IAM users, which aligns with the principle of least privilege and is a security best practice.src/zenml/zen_stores/base_zen_store.py (3)
65-69: The removal of
SecretsStoreInterfacefromBaseZenStorealigns with the PR's objective to refactor secrets management. Ensure that all references and usages of the interface methods are also updated accordingly.303-313: The
secrets_storeproperty now raises aNotImplementedErrorif no secrets store is configured. This is a clear and appropriate way to handle the case where a secrets store is expected but not available.65-69: The removal of event handling methods (
register_event_handlerand_trigger_event) should be checked to ensure that all event-related functionality has been appropriately refactored or is no longer required.src/zenml/zen_stores/secrets_stores/hashicorp_secrets_store.py (6)
184-213: The
store_secret_valuesmethod has been updated to store secret values and metadata in HashiCorp Vault. Ensure that thecasparameter is set correctly to prevent overwriting existing secrets and that the metadata is being stored as intended.219-273: The
get_secret_valuesmethod now retrieves secret values and metadata from HashiCorp Vault. Verify that the error handling is robust and that the method correctly distinguishes between different types of errors (e.g., secret not found vs. Vault error).278-312: The
update_secret_valuesmethod has been refactored to update secret values in HashiCorp Vault. Ensure that the method correctly updates the secret values and metadata, and that it handles errors appropriately.316-338: The
delete_secret_valuesmethod has been refactored to delete secret values in HashiCorp Vault. Verify that the method correctly deletes the secret and all its versions, and that it provides clear error messages when the secret does not exist or other errors occur.343-387: The
_convert_vault_secretmethod is deprecated and retained for migration from version 0.53.0. Ensure that this deprecation is clearly communicated to developers and that the method is not used in new code.181-404: > Note: This review was outside the patches, so it was mapped to the patch with the greatest overlap. Original lines [389-456]
The
list_secretsmethod has been updated to list all secrets without including secret values. Verify that the method correctly lists secrets and handles errors appropriately, such as when the Vault API returns an unexpected error.src/zenml/zen_stores/secrets_stores/aws_secrets_store.py (14)
16-18: The imports for
jsonanddatetimeare still present, which implies they are used elsewhere in the file. Ensure that their usage aligns with the new architecture.29-31: The import of
ClientErrorfrombotocore.exceptionsis necessary for exception handling in AWS API calls, which is a good practice.41-42: The import of
get_loggeris used for logging, which is essential for debugging and tracking the flow of the application.59-62: The
AWSSecretsStoreConfigurationclass has had several attributes removed. Ensure that the removal oflist_page_sizeandsecret_list_refresh_timeoutaligns with the new architecture and that these configurations are no longer necessary.66-67: The
regionproperty has been removed from theAWSSecretsStoreConfigurationclass. Verify that the region is now being handled appropriately elsewhere in the configuration, possibly through theauth_config.117-118: The
Configclass withinAWSSecretsStoreConfigurationhas been modified to allow extra attributes. This change could be to maintain backward compatibility with configurations from previous versions. Ensure that this does not introduce any security or stability issues.135-138: The comments have been updated to reflect the new metadata storage strategy. Ensure that the actual implementation of metadata storage in the AWS tags is consistent with these comments.
221-235: The
store_secret_valuesmethod has been added. Ensure that this method correctly implements the logic for storing secret values in AWS Secrets Manager, including proper error handling and logging.254-268: The
get_secret_valuesmethod has been added. Verify that it correctly fetches secret values from AWS Secrets Manager and handles errors as expected.324-340: The
update_secret_valuesmethod has been added. Verify that it correctly updates secret values in AWS Secrets Manager and handles potential errors.366-377: The
delete_secret_valuesmethod has been added. Verify that it correctly deletes secret values from AWS Secrets Manager and handles errors, especially the immediate deletion without recovery.407-438: The
_convert_aws_secretmethod has been marked as deprecated. Ensure that the deprecation is part of a managed transition to the new architecture and that there is a clear migration path for existing secrets.441-469: The
_get_aws_secret_filtersmethod has been marked as deprecated. Verify that the deprecation is part of a managed transition to the new architecture and that there is a clear migration path for existing secrets.473-531: The
list_secretsmethod has been marked as deprecated. Ensure that the deprecation is part of a managed transition to the new architecture and that there is a clear migration path for existing secrets.src/zenml/config/global_config.py (1)
- 574-588: The conditional check added warns and ignores environment variable configuration for secrets stores if the type is not SQL. This is a good practice to ensure that the configuration is only applied when appropriate. Verify that this logic is consistent with the overall architecture and that it does not introduce any unexpected behavior.
src/zenml/zen_stores/zen_store_interface.py (6)
- 87-90: The addition of
SecretFilter,SecretRequest,SecretResponse, andSecretUpdateclasses aligns with the PR objective to refactor secrets management. Ensure that these classes are used consistently across the codebase and that their fields are properly validated.Verification successful
It appears that the
rgcommand did not recognize the file typepython. This could be due to a misconfiguration or an issue with the version ofripgrepbeing used. To ensure we can search for the usage of the new classes across the Python files in the codebase, we should use a different approach.Let's try using
rgwithout specifying the file type, asripgrepwill automatically search through text files, which should include Python files by default.
The search results confirm that the new classes
SecretFilter,SecretRequest,SecretResponse, andSecretUpdateare being used throughout the codebase, particularly in thezen_storesandmodelsmodules, as well as in the integration tests and documentation. This indicates that the additions are consistent with the PR's objective to refactor secrets management and are integrated into the system.* 1323-1350: The `create_secret` method includes comprehensive documentation and raises appropriate exceptions. Verify that the scoping rules enforced in the secrets store are correctly implemented and tested.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that the new classes are used in the methods that manage secrets. rg --type python 'SecretFilter|SecretRequest|SecretResponse|SecretUpdate'Length of output: 106
Script:
#!/bin/bash # Search for the new classes across all files, as ripgrep will include text files by default. rg 'SecretFilter|SecretRequest|SecretResponse|SecretUpdate'Length of output: 26515
Verification successful
The verification process has confirmed that the
create_secretmethod is implemented in multiple files within the codebase, includingclient.py,rest_zen_store.py,sql_zen_store.py,zen_store_interface.py,workspaces_endpoints.py, andsecret.py. Additionally, the presence of unit tests for thecreate_secretmethod has been established, with several tests found intest_client.pyandtest_secret.pythat cover default scope, user scope, existing name scope, and different scopes.Based on the output, it is evident that the scoping rules for the
create_secretmethod are likely to be implemented given the method's presence in various parts of the codebase, and the existence of multiple tests suggests that these rules are also being tested.* 1352-1368: The `get_secret` method should ensure that the `hydrate` flag is respected in the implementation. Confirm that the method correctly handles the hydration of the secret's metadata.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that the scoping rules are implemented and covered by unit tests. rg --type python 'create_secret' rg --type python 'test_create_secret'Length of output: 132
Script:
#!/bin/bash # Search for the implementation of the create_secret method to verify scoping rules. rg 'def create_secret' # Search for unit tests related to the create_secret method. rg 'def test_create_secret'Length of output: 1292
1370-1391: The
list_secretsmethod's documentation clearly states that secret values are not included in the response. This is a good security practice. Ensure that the implementation adheres to this specification and that pagination is handled correctly.1393-1425: The
update_secretmethod's logic for handling secret values appears sound, with clear rules for merging and removing values. Ensure that the implementation follows these rules and that the scoping rules are validated.1427-1436: The
delete_secretmethod is straightforward, but ensure that any cascading effects of deleting a secret are handled (e.g., updating configurations that relied on the deleted secret).src/zenml/zen_stores/rest_zen_store.py (4)
71-71: The addition of the
SECRETSconstant is consistent with the PR's objective to refactor secrets management. This constant likely serves as a route for REST API calls related to secrets.166-169: The addition of
SecretFilter,SecretRequest,SecretResponse, andSecretUpdateclasses aligns with the PR's objective to refactor the secrets management system. These classes are likely used to handle the new structure of secrets within the ZenML framework.245-252: The removal of the
secrets_storeattribute from theRestZenStoreConfigurationclass is consistent with the PR's objective to change how secrets are managed. This change likely reflects the shift to storing secret metadata in the ZenML database.1643-1761: The addition of methods for managing secrets (
create_secret,get_secret,list_secrets,update_secret,delete_secret) in theRestZenStoreclass is a critical part of the PR's objective to refactor secrets management. These methods will facilitate the CRUD operations for secrets within the new system architecture.src/zenml/zen_stores/sql_zen_store.py (1)
- 3557-3627: The method
_check_sql_secret_scopeprovides a clear mechanism for checking the existence of a secret within a given scope. It's important to ensure that the error message returned when a secret already exists is informative and actionable. The current message fulfills this requirement.
stefannica
force-pushed
the
feature/cloud-479-store-secrets-metadata-in-db
branch
2 times, most recently
from
9d64a4a
to
932927e
Compare
stefannica
changed the title
stefannica
force-pushed
the
feature/cloud-479-store-secrets-metadata-in-db
branch
from
caf0702
to
aaa3c0a
Compare
| ValueError: If the user account credentials JSON is not a valid | ||
| JSON object. | ||
| """ | ||
| if isinstance(values.get("user_account_json"), dict): |
There was a problem hiding this comment.
This is just something left over from the "use service connectors to authenticate secrets stores" PR and revealed during testing this PR.
| Returns: | ||
| The validated configuration values. | ||
| """ | ||
| if isinstance(values.get("service_account_json"), dict): |
There was a problem hiding this comment.
This is just something left over from the "use service connectors to authenticate secrets stores" PR and revealed during testing this PR.
|
|
||
| def upgrade() -> None: | ||
| """Upgrade database schema and/or data, creating a new revision.""" | ||
| from zenml.config.global_config import GlobalConfiguration |
There was a problem hiding this comment.
I commented something similar on a cloudapi PR before where we decided importing these classes in migrations is okay as it will just run once.
I think here it is a little more complicated: This migration will also run in zenml 0.75.x in a few months/years from now, which means we need to keep those classes and all their (deprecated) methods around. Do you think this will lead to any trouble in the future?
There was a problem hiding this comment.
You're absolutely correct, we have to keep a lot of deprecated code around, even though it's only used in a DB migration script. Specifically, I'm talking about the deprecated "list_secrets" method in each secrets store implementation. We have two inter-related situations that I need to address here:
- getting rid of deprecated code: this is a bit laborious, because I'd have to retest the migration again for all possible store types, but easy. I just have to move all those functions and their dependents inside the migration script.
- getting rid of all imports in the migration script: technically impossible. These imports are strictly required to allow the migration script to connect to the external secrets store back-end. This needs to happen today, tomorrow, a few months from now or even a few years from now. As long as we have to support direct upgrades from 0.50.0 or earlier versions. On the bright side, we're only importing major concepts here: the global configuration and the SQL store. As long as we don't call attributes or methods prone to removal or deprecation on these objects (see the previous point), this should be safe.
There was a problem hiding this comment.
So the point is, we won't be able to remove those classes anyway as long as we want to support upgrading from pre-0.50 right? In that case I guess that's the best case we have, even though ideally this would not even be an automated migration in my opinion but rather something users do manually as there might be multiple reasons for this to fail.
stefannica
force-pushed
the
feature/cloud-479-store-secrets-metadata-in-db
branch
from
5a74645
to
b9766fb
Compare
…store-secrets-metadata-in-db
…store-secrets-metadata-in-db
stefannica
force-pushed
the
feature/cloud-479-store-secrets-metadata-in-db
branch
from
1aaf15d
to
fff470c
Compare
Describe changes
Refactor all secrets store to ONLY store secret values in the external back-end, as opposed to storing all the meta-information (UUID, name, user, workspace, create & update timestamps) in the external back-end.
list_secretsmethod in particular doesn't need to interact with the external secrets manager anymore, which means that ZenML no longer needs "list all secrets" permissions from the Cloud Secrets Manager APIs.This change is fully backwards-compatible. All existing secrets stored in external secrets stores will be added to the DB as part of the DB migration, while their values and their external secret IDs will continue to be stored externally. Existing users will not be aware that this change even happened.
TODO
Pre-requisites
Please ensure you have done the following:
developand the open PR is targetingdevelop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes
Summary by CodeRabbit
Documentation
New Features
Refactor
Bug Fixes
Deprecations
Chores