Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collection-Level Masking Strategy Override and Erase After #17

Merged
merged 7 commits into from
Sep 24, 2024

Conversation

pattisdr
Copy link

@pattisdr pattisdr commented Sep 17, 2024

Closes #PROD-2743

Description Of Changes

Define two things on Collection Meta to support erasure operations at the row level for database connectors.

Defines a masking strategy override under Collection Meta that will override Policy-based masking strategies if applicable and supported. Further configuration options may be supported in the future, but for now, we just have the strategy name.

dataset:
  - fides_key: postgres_example_test_dataset
    name: Postgres Example Test Dataset
    description: Example of a Postgres dataset containing a variety of related tables like customers, products, addresses, etc.
    collections:
      - name: address
        fides_meta:
          masking_strategy_override:
            strategy: delete

Allow erase_after to be specified:

      - name: customer
        fides_meta:
          erase_after:
            - bigquery_example_test_dataset.address

Code Changes

  • Add new masking_strategy_override under collection meta
  • Add new erase_after under collection meta, meant to be specified as a list of dataset.collection references. This was supported for saas connectors but not database connectors directly

Steps to Confirm

  • list any manual steps taken to confirm the changes

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Documentation Updated
  • Issue Requirements are Met
  • Relevant Follow-Up Issues Created
  • Update CHANGELOG.md

… removing it from field and dataset level meta.

- Constrain the masking strategies into an enum.
…er". This is supported for saas connectors but not database connectors.
@pattisdr pattisdr changed the title Collection-Level Masking Strategy Override Collection-Level Masking Strategy Override and Erase After Sep 20, 2024
@pattisdr pattisdr merged commit 5a66350 into main Sep 24, 2024
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants