Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: PowerVS cluster creation with dynamic resource creation #1608

Merged
merged 7 commits into from
Feb 29, 2024

Conversation

Karthik-K-N
Copy link
Contributor

@Karthik-K-N Karthik-K-N commented Feb 9, 2024

What this PR does / why we need it:

This PR adds the ability to create required resources for creating Power VS cluster dynamically eliminating user to precreate resources before creating cluster. A reference to proposal has been added which contains the detailed implementation plan for this feature.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes ##1611

Special notes for your reviewer:

/area provider/ibmcloud

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

Add ability to create required resource dynamically during PowerVS cluster creation

Proposal: #1488
API Pr: #1485
#1592

TODO:

  • Fix and add unit test cases
  • Add default and validating webhooks
  • Update cos object creation and deletion logic

@k8s-ci-robot k8s-ci-robot added the area/provider/ibmcloud Issues or PRs related to ibmcloud provider label Feb 9, 2024
Copy link

netlify bot commented Feb 9, 2024

Deploy Preview for kubernetes-sigs-cluster-api-ibmcloud ready!

Name Link
🔨 Latest commit 825789b
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-cluster-api-ibmcloud/deploys/65e06bd4017f570008c16464
😎 Deploy Preview https://deploy-preview-1608--kubernetes-sigs-cluster-api-ibmcloud.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 9, 2024
@Karthik-K-N
Copy link
Contributor Author

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 9, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 12, 2024
@mkumatag
Copy link
Member

/cc @mkumatag @dharaneeshvrd @Amulyam24 @Prajyot-Parab

team lets review this code by this weekend

cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
controllers/ibmpowervscluster_controller.go Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2024
@Karthik-K-N
Copy link
Contributor Author

Thanks for Dhraneesh for reviewing, I have addressed the comments and marked it as resolved whichever is possible, We will discuss for unresolved comments to finalize the approaches.


func (s *PowerVSClusterScope) createCOSBucket() error {
input := &s3.CreateBucketInput{
Bucket: pointer.String(s.COSInstance().BucketName),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a scenario where when we retry bucket creation immediately with the same name, it throws an error saying bucket exists even though the service instance and bucket get deleted successfully. Maybe we can handle by adding a random string to the bucket name?
(check out util.RandomString() from sigs.k8s.io/cluster-api/util)

util/util.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Show resolved Hide resolved
controllers/ibmpowervscluster_controller.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
controllers/ibmpowervsmachine_controller.go Outdated Show resolved Hide resolved
cloud/scope/powervs_machine.go Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
cloud/scope/powervs_cluster.go Outdated Show resolved Hide resolved
go.mod Outdated
github.com/IBM/platform-services-go-sdk v0.59.0
github.com/IBM/vpc-go-sdk v0.48.0
github.com/blang/semver/v4 v4.0.0
github.com/coreos/ignition v0.35.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a CVE for this package, need to explore how to fix this? @Prajyot-Parab

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is the CVE link -- GHSA-hj57-j5cw-2mwp, as this is a very old release, I don't think they will even patch this one. Lets keep this schema offline somewhere in the pkg folder.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +1515 to +1517
if s.ResourceGroup() == nil || s.ResourceGroup().Name == nil {
return "", fmt.Errorf("resource group name is not set")
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Use default resource group for the account if not specified.

// GetBootstrapData returns the base64 encoded bootstrap data from the secret in the Machine's bootstrap.dataSecretName.
func (m *PowerVSMachineScope) GetBootstrapData() (string, error) {
// DeleteMachineIgnition deletes the ignition associated with machine.
func (m *PowerVSMachineScope) DeleteMachineIgnition() error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this called?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Karthik-K-N did you drop it by mistake? I have it in my other PR, if need be I can split it in a diff PR and push.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intentioally removed it for this PR, Lets add it along with ignition 2.4 support PR. For now cosInstance delete will automatically delete all the buckets within it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated it in the latest change to delete ignition as part of machine deletion.

@Karthik-K-N Karthik-K-N force-pushed the update-ignition branch 2 times, most recently from df44c4a to 274f8f9 Compare February 29, 2024 06:19
@mkumatag
Copy link
Member

@Prajyot-Parab @Karthik-K-N I see linting issue for that file, see if we can somehow ignore that for that entire package

@Prajyot-Parab
Copy link
Contributor

@Prajyot-Parab @Karthik-K-N I see linting issue for that file, see if we can somehow ignore that for that entire package

fixed.

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 29, 2024
@mkumatag
Copy link
Member

I have approved the PR, one of you @Amulyam24 @Prajyot-Parab give lgtm

@k8s-ci-robot
Copy link
Contributor

@Karthik-K-N: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cluster-api-provider-ibmcloud-apidiff 825789b link false /test pull-cluster-api-provider-ibmcloud-apidiff

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link
Contributor

@Prajyot-Parab Prajyot-Parab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 29, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Karthik-K-N, mkumatag, Prajyot-Parab

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/provider/ibmcloud Issues or PRs related to ibmcloud provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants