Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HJ-86 system endpoint new lifecycle table #5484

Merged
merged 14 commits into from
Nov 12, 2024

Conversation

thingscouldbeworse
Copy link
Contributor

@thingscouldbeworse thingscouldbeworse commented Nov 11, 2024

Closes #HJ-86

Description Of Changes

Necessary changes for lifecycle tables. Option to retrieve only DnD relevant systems when paginating/filtering, logic to filter out "hidden" systems, and corresponding migration to add hidden bool column

Code Changes

  • inside the conditional block for filtering/paginating systems in the GET endpoint also allow for dnd_relevant systems, decided as
    • systems with any connection config attached
    • systems with any dataset references
  • column on ctl_systems hidden, defaults to False
  • by default, filter out these hidden systems in the GET endpoint filtering, but provide a param to optionally include them

Steps to Confirm

  • using either the swagger UI and `/system/ update/create/upsert endpoints (or DB statements) create (or verify that you already have)
    • a system connected to a ConnectionConfig object (via connectionconfig.system_id)
    • a system connected to a dataset reference (via ctl_systems.dataset_references)
    • a system without either of these connections
    • any system with the hidden column set to True
  • use the /system endpoint with GET to retrieve systems with pagination parameters and with dnd_relevant set to true, and also false. true should exclude the system without a connection config or dataset reference
  • repeat, but flipping the show_hidden property on/off. When on, the hidden system should be returned, otherwise not

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
  • Followup issues:
    • Followup issues created (include link)
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

Copy link

vercel bot commented Nov 11, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
fides-plus-nightly ⬜️ Ignored (Inspect) Visit Preview Nov 12, 2024 4:54pm

Copy link

cypress bot commented Nov 11, 2024

fides    Run #10951

Run Properties:  status check passed Passed #10951  •  git commit 8640115a70 ℹ️: Merge 2a000506b28ec230f77093d2e69af382dcd73ab2 into 2583b1cf7ebb7a902806ad2f3713...
Project fides
Branch Review refs/pull/5484/merge
Run status status check passed Passed #10951
Run duration 00m 38s
Commit git commit 8640115a70 ℹ️: Merge 2a000506b28ec230f77093d2e69af382dcd73ab2 into 2583b1cf7ebb7a902806ad2f3713...
Committer Kirk Hardy
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

src/fides/api/api/v1/endpoints/system.py Show resolved Hide resolved
Comment on lines 424 to 425
(System.connection_configs != None)
| (System.dataset_references.any()) # noqa: E712
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we use != None for one but .any() for the other condition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dataset_references is a column on ctl_systems, so we just check for anything in that column. System.connection_configs is a derived relationship, Systems have a property connection_configs but it's attached via the connectionconfig table having a system_id, so the SQL that gets generated has a join and works better with != None

tests/ctl/core/test_api.py Show resolved Hide resolved
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.20%. Comparing base (2583b1c) to head (2a00050).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5484   +/-   ##
=======================================
  Coverage   85.20%   85.20%           
=======================================
  Files         386      386           
  Lines       24242    24247    +5     
  Branches     2642     2644    +2     
=======================================
+ Hits        20655    20660    +5     
  Misses       3033     3033           
  Partials      554      554           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@thingscouldbeworse thingscouldbeworse merged commit accf7b3 into main Nov 12, 2024
39 checks passed
@thingscouldbeworse thingscouldbeworse deleted the HJ-86_system-endpoint-new-lifecycle-table branch November 12, 2024 17:36
Copy link

cypress bot commented Nov 12, 2024

fides    Run #10954

Run Properties:  status check passed Passed #10954  •  git commit accf7b3c1a: HJ-86 system endpoint new lifecycle table (#5484)
Project fides
Branch Review main
Run status status check passed Passed #10954
Run duration 00m 37s
Commit git commit accf7b3c1a: HJ-86 system endpoint new lifecycle table (#5484)
Committer Kirk Hardy
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants