Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

renameSDC operation updated for multiarray cluster support #163

Merged
merged 1 commit into from
Feb 16, 2023
Merged

renameSDC operation updated for multiarray cluster support #163

merged 1 commit into from
Feb 16, 2023

Conversation

khareRajshree
Copy link
Contributor

@khareRajshree khareRajshree commented Feb 16, 2023 •
edited
Loading

Description

RenameSDC operation was failing for the edge case where SDC is connected to more systems than the driver is currently configured to.

logs:

[root@master-1-8ip7VMdeU3NSg ~]# k logs vxflexos-node-cf7xw -n vxflexos
Defaulted container "driver" out of: driver, registrar, sdc (init)
time="2023-02-16T12:39:41Z" level=info msg="configured 0e7a082862fedf0f" allSystemNames= endpoint="https://10.247.101.69" isDefault=false password="********" skipCertificateValidation=true systemID=0e7a082862fedf0f user=admin
time="2023-02-16T12:39:41Z" level=info msg="driver configuration file " file=/vxflexos-config-params/driver-config-params.yaml
time="2023-02-16T12:39:41Z" level=info msg="Read CSI_LOG_FORMAT from log configuration file" format=text
time="2023-02-16T12:39:41Z" level=info msg="Read CSI_LOG_LEVEL from log configuration file" fields.level=debug
time="2023-02-16T12:39:41Z" level=info msg="array configuration file" file=/vxflexos-config/config
time="2023-02-16T12:39:41Z" level=info msg="set SDC GUID" guid=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T12:39:41Z" level=info msg="Found connected system" ID=0e7a082862fedf0f
time="2023-02-16T12:39:41Z" level=info msg="Found connected system" ID=1a99af710210af0f
time="2023-02-16T12:39:41Z" level=info msg="NodeInfo ID" node info=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T12:39:41Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/6f7c20586e54bf0f
time="2023-02-16T12:39:41Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/0e7a082862fedf0f
time="2023-02-16T12:39:41Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/1a99af710210af0f
time="2023-02-16T12:39:41Z" level=info msg="Probing all arrays. Number of arrays: 1"
time="2023-02-16T12:39:41Z" level=info msg="vol id in UpdateVolumePrefixToSystemsMap is: 457e31ed00000090  from systemID: 0e7a082862fedf0f \n"
time="2023-02-16T12:39:41Z" level=info msg="volumePrefixToSystems: adding new key, value pair: key 457, systemID: 0e7a082862fedf0f \n"
time="2023-02-16T12:39:41Z" level=info msg="array 0e7a082862fedf0f probed successfully"
time="2023-02-16T12:39:41Z" level=info msg="set SDC GUID" guid=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T12:39:42Z" level=info msg="SDC is already named: s-worker-2-8ip7VMdeU3NSg.domain."
time="2023-02-16T12:39:42Z" level=info msg="configured csi-vxflexos.dellemc.com" IsApproveSDCEnabled=false IsHealthMonitorEnabled=false IsSdcRenameEnabled=true allowRWOMultiPodAccess=false autoprobe=true mode=node privatedir=/var/lib/kubelet/plugins/vxflexos.emc.dell.com/disks sdcGUID=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32 sdcPrefix=s thickprovision=false
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12ec2e1]

goroutine 1 [running]:
github.com/dell/goscaleio.(*System).GetSdc(0x0)
        /go/pkg/mod/github.com/dell/goscaleio@v1.9.1-0.20230210195957-57ee14d5f084/sdc.go:46 +0xc1
github.com/dell/goscaleio.(*System).FindSdc(0x0, {0x186e328, 0x7}, {0xc0001fc150, 0x24})
        /go/pkg/mod/github.com/dell/goscaleio@v1.9.1-0.20230210195957-57ee14d5f084/sdc.go:99 +0xd4
github.com/dell/csi-vxflexos/v2/service.(*service).renameSDC(0xc00056b760, {0xc000512f00, {0x0, 0x0}, {0xc0001fc150, 0x24}, 0x0, 0x1, 0x0, {0x0, ...}, ...})
        /go/src/service/node.go:480 +0x153
github.com/dell/csi-vxflexos/v2/service.(*service).nodeProbe(0xc00056b760, {0x1ae2860, 0xc000512c60})
        /go/src/service/node.go:404 +0x298
github.com/dell/csi-vxflexos/v2/service.(*service).doProbe(0xc00056b760, {0x1ae2860, 0xc000512c60})
        /go/src/service/service.go:468 +0x13e
github.com/dell/csi-vxflexos/v2/service.(*service).BeforeServe(0xc00056b760, {0x1ae2860, 0xc000512c60}, 0x0?, {0x0?, 0x0?})
        /go/src/service/service.go:443 +0x659
github.com/dell/gocsi.(*StoragePlugin).Serve.func1()
        /go/pkg/mod/github.com/dell/gocsi@v1.6.0/gocsi.go:253 +0x311
sync.(*Once).doSlow(0x30?, 0xc0003018c0?)
        /usr/local/go/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
        /usr/local/go/src/sync/once.go:65
github.com/dell/gocsi.(*StoragePlugin).Serve(0xc000512ba0?, {0x1ae2860, 0xc000512c60}, {0x1ae1a90?, 0xc000512b70?})
        /go/pkg/mod/github.com/dell/gocsi@v1.6.0/gocsi.go:218 +0x94
github.com/dell/gocsi.Run({0x1ae27f0?, 0xc000048048}, {0x1887f2e, 0x18}, {0x18c79ba, 0x34}, {0x193d8d4, 0x387}, {0x1ad5d90, 0xc00029c400})
        /go/pkg/mod/github.com/dell/gocsi@v1.6.0/gocsi.go:132 +0x69b
main.main.func1({0x1ae27f0, 0xc000048048})
        /go/src/main.go:61 +0x314
main.main()
        /go/src/main.go:65 +0x52e

Fix is been tested on multiarray environment where the same SDC is associated with 2 arrays:

[root@worker-1-8ip7VMdeU3NSg ~]# /opt/emc/scaleio/sdc/bin/drv_cfg --query_mdm
Retrieved 2 mdm(s)
MDM-ID 0e7a082862fedf0f SDC ID c42a67560000009e INSTALLATION ID 701f8ebf53cce2a1 IPs [0]-10.247.101.60 [1]-10.247.101.68
MDM-ID 1a99af710210af0f SDC ID 2a2a49c9000006ff INSTALLATION ID 62d122183c5e9fbe IPs [0]-10.247.13.202

GitHub Issues

List the GitHub issues impacted by this PR:

GitHub Issue #
dell/csm#402

Checklist:

  • I have performed a self-review of my own code to ensure there are no formatting, vetting, linting, or security issues
  • I have verified that new and existing unit tests pass locally with my changes
  • I have not allowed coverage numbers to degenerate
  • I have maintained at least 90% code coverage
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • Backward compatibility is not broken

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

  • Unit testing (Refer GitHub workflow for more details)
  • Functional testing
  • cert-csi
[root@master-1-8ip7VMdeU3NSg ~]# k logs vxflexos-node-dt5vm -n vxflexos
Defaulted container "driver" out of: driver, registrar, sdc (init)
time="2023-02-16T14:56:21Z" level=info msg="configured 0e7a082862fedf0f" allSystemNames= endpoint="https://10.247.101.69" isDefault=false password="********" skipCertificateValidation=true systemID=0e7a082862fedf0f user=admin
time="2023-02-16T14:56:21Z" level=info msg="driver configuration file " file=/vxflexos-config-params/driver-config-params.yaml
time="2023-02-16T14:56:21Z" level=info msg="Read CSI_LOG_FORMAT from log configuration file" format=text
time="2023-02-16T14:56:21Z" level=info msg="Read CSI_LOG_LEVEL from log configuration file" fields.level=debug
time="2023-02-16T14:56:21Z" level=info msg="array configuration file" file=/vxflexos-config/config
time="2023-02-16T14:56:21Z" level=info msg="set SDC GUID" guid=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T14:56:21Z" level=info msg="Found connected system" ID=0e7a082862fedf0f
time="2023-02-16T14:56:21Z" level=info msg="Found connected system" ID=1a99af710210af0f
time="2023-02-16T14:56:21Z" level=info msg="Found connected system" ID=6f7c20586e54bf0f
time="2023-02-16T14:56:21Z" level=info msg="NodeInfo ID" node info=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T14:56:21Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/0e7a082862fedf0f
time="2023-02-16T14:56:21Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/1a99af710210af0f
time="2023-02-16T14:56:21Z" level=info msg="NodeInfo topologykeys" node info key=csi-vxflexos.dellemc.com/6f7c20586e54bf0f
time="2023-02-16T14:56:21Z" level=info msg="Probing all arrays. Number of arrays: 1"
time="2023-02-16T14:56:21Z" level=info msg="vol id in UpdateVolumePrefixToSystemsMap is: 457e31ed00000090  from systemID: 0e7a082862fedf0f \n"
time="2023-02-16T14:56:21Z" level=info msg="volumePrefixToSystems: adding new key, value pair: key 457, systemID: 0e7a082862fedf0f \n"
time="2023-02-16T14:56:21Z" level=info msg="array 0e7a082862fedf0f probed successfully"
time="2023-02-16T14:56:21Z" level=info msg="set SDC GUID" guid=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32
time="2023-02-16T14:56:21Z" level=info msg="Assigning name: a-worker-2-8ip7VMdeU3NSg.domain to SDC with GUID 8770F2EE-750A-50AF-9F6C-5B2E42EA5A32 on system 0e7a082862fedf0f"
time="2023-02-16T14:56:21Z" level=info msg="SDC name set to: a-worker-2-8ip7VMdeU3NSg.domain."
time="2023-02-16T14:56:21Z" level=info msg="configured csi-vxflexos.dellemc.com" IsApproveSDCEnabled=false IsHealthMonitorEnabled=false IsSdcRenameEnabled=true allowRWOMultiPodAccess=false autoprobe=true mode=node privatedir=/var/lib/kubelet/plugins/vxflexos.emc.dell.com/disks sdcGUID=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32 sdcPrefix=a thickprovision=false
time="2023-02-16T14:56:21Z" level=info msg="identity service registered"
time="2023-02-16T14:56:21Z" level=info msg="node service registered"
time="2023-02-16T14:56:21Z" level=info msg="Registering additional GRPC servers"
time="2023-02-16T14:56:21Z" level=info msg=serving endpoint="unix:///var/lib/kubelet/plugins/vxflexos.emc.dell.com/csi_sock"
time="2023-02-16T14:56:21Z" level=info msg="/csi.v1.Identity/GetPluginInfo: REQ 0001: XXX_NoUnkeyedLiteral={}, XXX_sizecache=0"
time="2023-02-16T14:56:21Z" level=info msg="/csi.v1.Identity/GetPluginInfo: REP 0001: Name=csi-vxflexos.dellemc.com, VendorVersion=2.5.0+23+dirty, Manifest=map[commit:1f22b2f46e49f445edf659bab7eb177be27f6024 formed:Wed, 15 Feb 2023 18:42:00 UTC semver:2.5.0+23+dirty url:http://github.com/dell/csi-vxflexos], XXX_NoUnkeyedLiteral={}, XXX_sizecache=0"
time="2023-02-16T14:56:22Z" level=info msg="/csi.v1.Node/NodeGetInfo: REQ 0002: XXX_NoUnkeyedLiteral={}, XXX_sizecache=0"
time="2023-02-16T14:56:22Z" level=info msg="/csi.v1.Node/NodeGetInfo: REP 0002: NodeId=8770F2EE-750A-50AF-9F6C-5B2E42EA5A32, MaxVolumesPerNode=0, AccessibleTopology=segments:<key:\"csi-vxflexos.dellemc.com/0e7a082862fedf0f\" value:\"csi-vxflexos.dellemc.com\" > segments:<key:\"csi-vxflexos.dellemc.com/1a99af710210af0f\" value:\"csi-vxflexos.dellemc.com\" > segments:<key:\"csi-vxflexos.dellemc.com/6f7c20586e54bf0f\" value:\"csi-vxflexos.dellemc.com\" > , XXX_NoUnkeyedLiteral={}, XXX_sizecache=0"

cert-csi results:

[2023-02-16 11:13:39]  INFO Avg time of a run:   58.25s
[2023-02-16 11:13:39]  INFO Avg time of a del:   13.14s
[2023-02-16 11:13:39]  INFO Avg time of all:     74.81s
[2023-02-16 11:13:39]  INFO During this run 100.0% of suites succeeded

bharathsreekanth
bharathsreekanth approved these changes Feb 16, 2023
View reviewed changes
service/node.go
@@ -477,6 +477,9 @@ func (s *service) renameSDC(opts Opts) error {

// fetch SDC details
for _, systemID := range connectedSystemID {
if s.systems[systemID] == nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you log any message in this case?

@khareRajshree khareRajshree merged commit 3ad4e38 into main Feb 16, 2023
@anandrajak1 anandrajak1 deleted the renameSDC-fix branch October 17, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JacobGros JacobGros JacobGros approved these changes

@bharathsreekanth bharathsreekanth bharathsreekanth approved these changes

@VamsiSiddu-7 VamsiSiddu-7 VamsiSiddu-7 approved these changes

@abhi16394 abhi16394 Awaiting requested review from abhi16394 abhi16394 is a code owner

@mjsdell mjsdell Awaiting requested review from mjsdell

@jooseppi-luna jooseppi-luna Awaiting requested review from jooseppi-luna

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@khareRajshree @JacobGros @bharathsreekanth @VamsiSiddu-7