return abnormal if the mount is corrupted #3462
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Added DNM as still need to test it out. |
Madhu-1
added
ci/skip/e2e
I1024 11:00:47.851488 1 utils.go:195] ID: 12 GRPC call: /csi.v1.Node/NodeGetVolumeStats
I1024 11:00:47.851916 1 utils.go:206] ID: 12 GRPC request: {"volume_id":"0001-0009-rook-ceph-0000000000000001-7fed1ce7-97cf-43ef-9b84-2a49ab992515","volume_path":"/var/lib/kubelet/pods/8087df68-9756-4f38-86ef-6c81e1075607/volumes/kubernetes.io~csi/pvc-15e63d0a-77de-4886-8d0f-516f9fecbeb4/mount"}
I1024 11:00:47.854448 1 utils.go:212] ID: 12 GRPC response: {"usage":[{"available":1073741824,"total":1073741824,"unit":1}]}
I1024 11:02:11.800530 1 utils.go:195] ID: 13 GRPC call: /csi.v1.Node/NodeGetCapabilities
I1024 11:02:11.800683 1 utils.go:206] ID: 13 GRPC request: {}
I1024 11:02:11.801084 1 utils.go:212] ID: 13 GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":2}}},{"Type":{"Rpc":{"type":5}}}]}
I1024 11:02:11.803017 1 utils.go:195] ID: 14 GRPC call: /csi.v1.Node/NodeGetVolumeStats
I1024 11:02:11.803429 1 utils.go:206] ID: 14 GRPC request: {"volume_id":"0001-0009-rook-ceph-0000000000000001-7fed1ce7-97cf-43ef-9b84-2a49ab992515","volume_path":"/var/lib/kubelet/pods/8087df68-9756-4f38-86ef-6c81e1075607/volumes/kubernetes.io~csi/pvc-15e63d0a-77de-4886-8d0f-516f9fecbeb4/mount"}
W1024 11:02:12.879745 1 nodeserver.go:637] ID: 14 corrupted mount detected in "/var/lib/kubelet/pods/8087df68-9756-4f38-86ef-6c81e1075607/volumes/kubernetes.io~csi/pvc-15e63d0a-77de-4886-8d0f-516f9fecbeb4/mount": stat /var/lib/kubelet/pods/8087df68-9756-4f38-86ef-6c81e1075607/volumes/kubernetes.io~csi/pvc-15e63d0a-77de-4886-8d0f-516f9fecbeb4/mount: permission denied
I1024 11:02:12.880189 1 utils.go:212] ID: 14 GRPC response: {"volume_condition":{"abnormal":true,"message":"stat /var/lib/kubelet/pods/8087df68-9756-4f38-86ef-6c81e1075607/volumes/kubernetes.io~csi/pvc-15e63d0a-77de-4886-8d0f-516f9fecbeb4/mount: permission denied"}} |
Madhu-1
added
component/cephfs
Madhu-1
force-pushed
the
cephfs-volume-condition
branch
from
5c036ba
to
90aa2a8
Compare
|
@Mergifyio rebase |
When we do stat on the targetpath, if there is any error we can check is it due to corruption. If yes, cephcsi can return abnormal in the NodeGetVolumeStats so that consumer (CO/admin) and detect and take further action. Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
When we do stat on the targetpath, if there is any error we can check is it due to corruption. If yes, cephcsi can return abnormal in the NodeGetVolumeStats so that consumer (CO/admin) and detect and take further action. Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Added a new section for the ceph kernel client mount corruption detection and recovery. Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
✅ Branch has been successfully rebased |
Madhu-1
force-pushed
the
cephfs-volume-condition
branch
from
90aa2a8
to
14a8225
Compare
|
/test ci/centos/k8s-e2e-external-storage/1.22 |
|
/test ci/centos/k8s-e2e-external-storage/1.23 |
|
/test ci/centos/k8s-e2e-external-storage/1.24 |
|
/test ci/centos/mini-e2e-helm/k8s-1.22 |
|
/test ci/centos/mini-e2e-helm/k8s-1.23 |
|
/test ci/centos/mini-e2e-helm/k8s-1.24 |
|
/test ci/centos/mini-e2e/k8s-1.22 |
|
/test ci/centos/mini-e2e/k8s-1.23 |
|
/test ci/centos/mini-e2e/k8s-1.24 |
|
/test ci/centos/upgrade-tests-cephfs |
|
/test ci/centos/upgrade-tests-rbd |
Rakshith-R
approved these changes
Oct 26, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we do stat on the targetpath, if there is any error we can check is it due to corruption. If yes, cephcsi can return abnormal in the NodeGetVolumeStats so that consumer (CO/admin) and detect and take further action.
Signed-off-by: Madhu Rajanna madhupr007@gmail.com