Skip to content

Commit

Permalink
Remove mount cache for cephfs
Browse files Browse the repository at this point in the history 
PR ceph#282 introduces the mount cache to
solve cephfs fuse mount issue when cephfs plugin pod
restarts .This is not working as intended. This PR removes
the code for maintainability.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
  • Loading branch information
@Madhu-1
Madhu-1 committed Feb 10, 2020
1 parent 1695c69 commit a4a2705
Show file tree
Hide file tree
Showing 8 changed files with 21 additions and 420 deletions.
5 changes: 0 additions & 5 deletions charts/ceph-csi-cephfs/templates/nodeplugin-daemonset.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ spec:
- "--v=5"
- "--drivername=$(DRIVER_NAME)"
- "--metadatastorage=k8s_configmap"
- "--mountcachedir=/mount-cache-dir"
env:
- name: POD_IP
valueFrom:
Expand All @@ -109,8 +108,6 @@ spec:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
volumeMounts:
- name: mount-cache-dir
mountPath: /mount-cache-dir
- name: socket-dir
mountPath: /csi
- name: mountpoint-dir
Expand Down Expand Up @@ -161,8 +158,6 @@ spec:
{{ toYaml .Values.nodeplugin.plugin.resources | indent 12 }}
{{- end }}
volumes:
- name: mount-cache-dir
emptyDir: {}
- name: socket-dir
hostPath:
path: {{ .Values.socketDir }}
Expand Down
4 changes: 4 additions & 0 deletions cmd/cephcsi.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ func init() {
flag.BoolVar(&conf.IsNodeServer, "nodeserver", false, "start cephcsi node server")

// cephfs related flags
// marking this as deprecated, remove it in next major release
flag.StringVar(&conf.MountCacheDir, "mountcachedir", "", "mount info cache save dir")
flag.BoolVar(&conf.ForceKernelCephFS, "forcecephkernelclient", false, "enable Ceph Kernel clients on kernel < 4.17 which support quotas")

Expand Down Expand Up @@ -172,6 +173,9 @@ func main() {
driver.Run(&conf, cp)

case cephfsType:
if conf.MountCacheDir != "" {
klog.Warning("mountcachedir option is deprecated")
}
driver := cephfs.NewDriver()
driver.Run(&conf, cp)

Expand Down
5 changes: 0 additions & 5 deletions deploy/cephfs/kubernetes/csi-cephfsplugin.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@ spec:
- "--v=5"
- "--drivername=cephfs.csi.ceph.com"
- "--metadatastorage=k8s_configmap"
- "--mountcachedir=/mount-cache-dir"
- "--metricsport=8090"
- "--metricspath=/metrics"
- "--enablegrpcmetrics=false"
Expand All @@ -84,8 +83,6 @@ spec:
value: unix:///csi/csi.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: mount-cache-dir
mountPath: /mount-cache-dir
- name: socket-dir
mountPath: /csi
- name: mountpoint-dir
Expand Down Expand Up @@ -130,8 +127,6 @@ spec:
mountPath: /csi
imagePullPolicy: "IfNotPresent"
volumes:
- name: mount-cache-dir
emptyDir: {}
- name: socket-dir
hostPath:
path: /var/lib/kubelet/plugins/cephfs.csi.ceph.com/
Expand Down
35 changes: 17 additions & 18 deletions docs/deploy-cephfs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,24 +43,23 @@ that should be resolved in v14.2.3.

**Available command line arguments:**

| Option | Default value | Description |
| -------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `--endpoint` | `unix://tmp/csi.sock` | CSI endpoint, must be a UNIX socket |
| `--drivername` | `cephfs.csi.ceph.com` | Name of the driver (Kubernetes: `provisioner` field in StorageClass must correspond to this value) |
| `--nodeid` | _empty_ | This node's ID |
| `--type` | _empty_ | Driver type `[rbd | cephfs]` If the driver type is set to `rbd` it will act as a `rbd plugin` or if it's set to `cephfs` will act as a `cephfs plugin` |
| `--mountcachedir` | _empty_ | Volume mount cache info save dir. If left unspecified, the dirver will not record mount info, or it will save mount info and when driver restart it will remount volume it cached. |
| `--instanceid` | "default" | Unique ID distinguishing this instance of Ceph CSI among other instances, when sharing Ceph clusters across CSI instances for provisioning |
| `--pluginpath` | "/var/lib/kubelet/plugins/" | The location of cephcsi plugin on host |
| `--metadatastorage` | _empty_ | Points to where older (1.0.0 or older plugin versions) metadata about provisioned volumes are kept, as file or in as k8s configmap (`node` or `k8s_configmap` respectively) |
| `--pidlimit` | _0_ | Configure the PID limit in cgroups. The container runtime can restrict the number of processes/tasks which can cause problems while provisioning (or deleting) a large number of volumes. A value of `-1` configures the limit to the maximum, `0` does not configure limits at all. |
| `--metricsport` | `8080` | TCP port for /grpc metrics requests |
| `--metricspath` | `/metrics` | Path of prometheus endpoint where metrics will be available |
| `--enablegrpcmetrics` | `false` | Enable grpc metrics collection and start prometheus server |
| `--polltime` | `60s` | Time interval in between each poll |
| `--timeout` | `3s` | Probe timeout in seconds |
| `--histogramoption` | `0.5,2,6` | Histogram option for grpc metrics, should be comma separated value (ex:= "0.5,2,6" where start=0.5 factor=2, count=6) |
| `--forcecephkernelclient` | `false` | Force enabling Ceph Kernel clients for mounting on kernels < 4.17 |
| Option | Default value | Description |
| ------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `--endpoint` | `unix://tmp/csi.sock` | CSI endpoint, must be a UNIX socket |
| `--drivername` | `cephfs.csi.ceph.com` | Name of the driver (Kubernetes: `provisioner` field in StorageClass must correspond to this value) |
| `--nodeid` | _empty_ | This node's ID |
| `--type` | _empty_ | Driver type `[rbd | cephfs]` If the driver type is set to `rbd` it will act as a `rbd plugin` or if it's set to `cephfs` will act as a `cephfs plugin` |
| `--instanceid` | "default" | Unique ID distinguishing this instance of Ceph CSI among other instances, when sharing Ceph clusters across CSI instances for provisioning |
| `--pluginpath` | "/var/lib/kubelet/plugins/" | The location of cephcsi plugin on host |
| `--metadatastorage` | _empty_ | Points to where older (1.0.0 or older plugin versions) metadata about provisioned volumes are kept, as file or in as k8s configmap (`node` or `k8s_configmap` respectively) |
| `--pidlimit` | _0_ | Configure the PID limit in cgroups. The container runtime can restrict the number of processes/tasks which can cause problems while provisioning (or deleting) a large number of volumes. A value of `-1` configures the limit to the maximum, `0` does not configure limits at all. |
| `--metricsport` | `8080` | TCP port for /grpc metrics requests |
| `--metricspath` | `/metrics` | Path of prometheus endpoint where metrics will be available |
| `--enablegrpcmetrics` | `false` | Enable grpc metrics collection and start prometheus server |
| `--polltime` | `60s` | Time interval in between each poll |
| `--timeout` | `3s` | Probe timeout in seconds |
| `--histogramoption` | `0.5,2,6` | Histogram option for grpc metrics, should be comma separated value (ex:= "0.5,2,6" where start=0.5 factor=2, count=6) |
| `--forcecephkernelclient` | `false` | Force enabling Ceph Kernel clients for mounting on kernels < 4.17 |

**NOTE:** The parameter `-forcecephkernelclient` enables the Kernel
CephFS mounter on kernels < 4.17.
Expand Down
8 changes: 0 additions & 8 deletions pkg/cephfs/driver.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -115,14 +115,6 @@ func (fs *Driver) Run(conf *util.Config, cachePersister util.CachePersister) {
// Update namespace for storing keys into a specific namespace on RADOS, in the CephFS
// metadata pool
volJournal.SetNamespace(radosNamespace)

initVolumeMountCache(conf.DriverName, conf.MountCacheDir)
if conf.MountCacheDir != "" {
if err := remountCachedVolumes(); err != nil {
klog.Warningf("failed to remount cached volumes: %v", err)
// ignore remount fail
}
}
// Initialize default library driver

fs.cd = csicommon.NewCSIDriver(conf.DriverName, util.DriverVersion, conf.NodeID)
Expand Down
Loading

0 comments on commit a4a2705

Please sign in to comment.