[BUG]: driver logs gets flooded with "Checking if host information is added to array" in debug mode

[karthikk92]

Customer provided info:
I set LogLevel to debug and found out that 15 minutes after starting the pods the driver container floods the log the following message. Then the cpu usage raises.
time="2021-11-11T11:50:47Z" level=debug runid=node-1 msg="Checking if host information is added to array" func="github.com/dell/csi-unity/service.(*service).syncNodeInfoRoutine()" file="/go/src/csi-unity/service/node.go:1648"
time="2021-11-11T11:50:47Z" level=debug runid=node-1 msg="Checking if host information is added to array" func="github.com/dell/csi-unity/service.(*service).syncNodeInfoRoutine()" file="/go/src/csi-unity/service/node.go:1648"
edit 1:
I guess this is related to this env var: X_CSI_UNITY_SYNC_NODEINFO_INTERVAL.
edit 2:
Seems that the function is stuck in the for loop: https://github.com/dell/csi-unity/blob/a6688df88ddf23d45fd268dd1a70880903ddd320/service/node.go#L1668 (edited)
node.go
func (s *service) syncNodeInfoRoutine(ctx context.Context) {

[mkej]

I have also observed this behavior using csi-unity driver versions v1.6.0 and v2.0.0, on Kubernetes cluster running v1.21.3 and v1.22.4 (same cluster, upgraded).

Changing X_CSI_UNITY_SYNC_NODEINFO_INTERVAL to 5 made this issue appear after 5 minutes from pod creation, as expected.

[muellerfabi]

Hello,
I was the one creating that initial issue on Slack and want to give an update.

I just updated both Operator and Driver, but Issue still occurs on

• Openshift 4.8.20 (K8s 1.21.4)
• with Dell CSI Operator v1.6.0
• and Unity Driver v2.1.0

oc adm top pods --use-protocol-buffers  
NAME                                                   CPU(cores)   MEMORY(bytes)     
dell-csi-operator-controller-manager-8c8b9f98f-m7njb   2m           47Mi              
unity-controller-74bfdc88f5-ctwh9                      0m           135Mi             
unity-node-gcrm6                                       1012m        39Mi              
unity-node-pmhfr                                       1270m        42Mi              
unity-node-zpssf                                       1274m        40Mi          

oc get po
NAME                                                   READY   STATUS      RESTARTS   AGE
dell-csi-operator-controller-manager-8c8b9f98f-m7njb   1/1     Running     1          114m
installplan-approver-txc88                             0/1     Completed   0          4h11m
unity-controller-74bfdc88f5-ctwh9                      5/5     Running     5          114m
unity-node-gcrm6                                       2/2     Running     1          114m
unity-node-pmhfr                                       2/2     Running     2          114m    
unity-node-zpssf                                       2/2     Running     2          114m

Cheers

[rensyct]

Thank you fubsle for the update. Unity driver v2.1.0 does not have the fix for this issue. Our team has started looking into this issue and work is in progress. We will update this ticket once the fix is available

[karthikk92]

This issue is fixed and please go ahead and use latest nightly build tag available.

Please use version tag as "nightly" as specified bellow:

./csi-install.sh --namespace unity --skip-verify-node --version "nightly" --values ./myvalues.yaml. and use version as nightly in myvalues.yaml as shown:
version: nightly
@fubsle

[karthikk92]

Hi,

The fix is tested on OCP 4.8 environment with the drivers installed and csi-unity tests runs against it.
The user can pick the latest ‘nightly’ build tag for csi-unity.
https://hub.docker.com/r/dellemc/csi-unity/tags

NOTE: during csi-driver driver deployment using operator, the user would need to edit the value for SYNC_NODE_TIME_INTERVAL in ConfigMap as follows

apiVersion: v1
kind: ConfigMap
metadata:
name: unity-config-params
namespace: test-unity
data:
driver-config-params.yaml: |
CSI_LOG_LEVEL: "info"
ALLOW_RWO_MULTIPOD_ACCESS: "false"
MAX_UNITY_VOLUMES_PER_NODE: "0"
SYNC_NODE_INFO_TIME_INTERVAL: "15"
TENANT_NAME: ""

This operator related sample default value change would be updated in the upcoming release of dell-csi-operator.

Thanks,
Keerthi vardhan