Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DVS-2961: Allow System Layout Service calls to generate the DVS node map #103

Conversation

DavidIbarr
Copy link
Contributor

@DavidIbarr DavidIbarr commented Jan 9, 2024

Adding policies for SLS under DVS to allow the dvs_generate_map script to use the data from SLS to generate the node map.

Tested the change by editing the opa-policy-ingressgateway-spire config map and restarting pods. Verified that the SLS data is sent via the API when using a new valid token.

Summary and Scope

Summarize what has changed. Explain why this PR is necessary. What is impacted? Is this a new feature, critical bug fix, etc?
This PR pertains to new feature development to use SLS in the dvs_generate_map CSM script. Previously the script generates the DVS node map by doing host lookups for all nodes, causing a large increase in traffic at boot time. By sending only a few requests to SLS to get IP addresses instead, this greatly reduces the network traffic.

Is this change backwards incompatible, backwards compatible, or a backwards compatible bugfix?
This change is backwards compatible, it does not remove any current functionality or take away access from any already existing requests made to the HSM or otherwise - it only adds additional access to the SLS APIs via the Spire token.

Issues and Related PRs

List and characterize relationship to Jira/Github issues and other pull requests. Be sure to list dependencies.

Resolves opened issue [CASMTRIAGE-6463] (https://jira-pro.it.hpe.com:8443/browse/CASMTRIAGE-6463)
Issue [DVS-2961] (https://jira-pro.it.hpe.com:8443/browse/DVS-2961) depends on [CASMTRIAGE-6463] (https://jira-pro.it.hpe.com:8443/browse/CASMTRIAGE-6463)

Work is required to complete the DVS feature [DVS-2961] (https://jira-pro.it.hpe.com:8443/browse/DVS-2961)

  • Resolves [issue id](issue link)
  • Change will also be needed in <insert branch name here>
  • Future work required by [issue id](issue link)
  • Documentation changes required in [issue id](issue link)
  • Merge with/before/after <insert PR URL here>

Testing

List the environments in which these changes were tested.

Tested on:

CSM 1.5 on Lemondrop (1.5.0-beta.70, compute-csm-1.5-5.2.34-x86_64, compute-csm-1.5-5.2.34-aarch64, cray-shasta-csm-sles15sp5-barebones-csm-1.5, secure-kubernetes-5.2.39-x86_64.squashfs)
CSM 1.4.3 on Groot (cray-shasta-csm-sles15sp4-barebones.x86_64-csm-1.4, secure-kubernetes-0.4.71-x86_64.squashfs)

  • <development system>
  • Local development environment
  • Virtual Shasta

Test description:

How were the changes tested and success verified? If schema changes were part of this change, how were those handled in your upgrade/downgrade testing?

  • Were the install/upgrade-based validation checks/tests run (goss tests/install-validation doc)?
  • Were continuous integration tests run? If not, why?
  • Was upgrade tested? If not, why?
  • Was downgrade tested? If not, why?
  • Were new tests (or test issues/Jiras) created for this change?

Tested the change by editing the opa-policy-ingressgateway-spire config map and restarting pods. Verified that the SLS data is sent via the API when using a new valid token.
No code or behavior of previously accessed APIs was changed, and those tests were not yet otherwise exercised. Created additional tests for this change to ensure when tests are run that access has been made available to the SLS api.

Risks and Mitigations

Are there known issues with these changes? Any other special considerations?
No known issues, the changes do not affect previous behavior.

Pull Request Checklist

  • Version number(s) incremented, if applicable
  • Copyrights updated
  • License file intact
  • Target branch correct
  • CHANGELOG.md updated
  • Testing is appropriate and complete, if applicable
  • HPC Product Announcement prepared, if applicable

David Ibarra added 2 commits January 5, 2024 13:32
Adding policies for SLS under DVS to allow the dvs_generate_map
script to use the data from SLS to generate the node map.

Tested the change by editing the opa-policy-ingressgateway-spire
config map and restarting pods. Verified that the SLS data is
sent via the API when using a new valid token.
Updated the tests for SLS to ensure access is available to the
path /apis/sls/v1/networks. Cleaned up comments for current spire
tests under APIs used by DVS, matching current keycloak tests.
@nieuwsma nieuwsma removed their request for review January 9, 2024 16:14
Copy link

@cliffwickman cliffwickman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this at all, but the proof was in the testing so I'll approve.

@DavidIbarr
Copy link
Contributor Author

DavidIbarr commented Jan 10, 2024

@bo-quan It looks like I am getting these problems with the checks not being successful:

snyk requires an authenticated account. Please run snyk auth and try again.

and

Run docker/login-action@v2
with:
registry: artifactory.algol60.net
ecr: auto
logout: true
Error: Username and password required

Do you have any idea how I resolve this? Also, I believe I only have write, not merge access.

@DavidIbarr
Copy link
Contributor Author

Given this remark from a discussion from a thread in #casm-devops: https://cray.slack.com/archives/C01JRKK8J2F/p1704919674353209?thread_ts=1704917712.617859&cid=C01JRKK8J2F, the documented method of creating a fork for the repo is no longer the standard procedure in order to pass the Snyk code checks. Closing this PR and continuing with #104.

@DavidIbarr DavidIbarr closed this Jan 11, 2024
@DavidIbarr DavidIbarr deleted the task/DVS-2961-allow-node-map-sls-queries branch January 11, 2024 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants