Skip to content

Commit

Permalink
feat: add an option to enable pprof profiling
Browse files Browse the repository at this point in the history 
Signed-off-by: Anish Ramasekar <anish.ramasekar@gmail.com>
  • Loading branch information
@aramase
aramase committed Dec 9, 2021
1 parent 61e5b2d commit 7bd0757
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 19 deletions.
8 changes: 8 additions & 0 deletions .vscode/settings.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"markdownlint.config": {
"MD028": false,
"MD025": {
"front_matter_title": ""
}
}
}
38 changes: 20 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,23 @@ The external-provisioner is an external controller that monitors `PersistentVolu

This information reflects the head of this branch.

| Compatible with CSI Version | Container Image | [Min K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#minimum-version) | [Recommended K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#recommended-version) |
| ------------------------------------------------------------------------------------------ | -------------------------------| --------------- | ------------- |
| [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/csi-provisioner | 1.20 | 1.22 |
| Compatible with CSI Version | Container Image | [Min K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#minimum-version) | [Recommended K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#recommended-version) |
| ------------------------------------------------------------------------------------------ | -------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
| [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/csi-provisioner | 1.20 | 1.22 |

## Feature status

Various external-provisioner releases come with different alpha / beta features. Check `--help` output for alpha/beta features in each release.

Following table reflects the head of this branch.

| Feature | Status | Default | Description | Provisioner Feature Gate Required |
| -------------- | ------- | ------- | --------------------------------------------------------------------------------------------- | --------------------------------- |
| Snapshots | GA | On | [Snapshots and Restore](https://kubernetes-csi.github.io/docs/snapshot-restore-feature.html). | No |
| CSIMigration | Beta | On | [Migrating in-tree volume plugins to CSI](https://kubernetes.io/docs/concepts/storage/volumes/#csi-migration). | No |
| CSIStorageCapacity | Beta | On | Publish [capacity information](https://kubernetes.io/docs/concepts/storage/volumes/#storage-capacity) for the Kubernetes scheduler. | No |
| ReadWriteOncePod | Alpha | Off | [Single pod access mode for PersistentVolumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes). | No |
| HonorPVReclaimPolicy| Alpha |Off | [Honor the PV reclaim policy](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2644-honor-pv-reclaim-policy) | No
| Feature | Status | Default | Description | Provisioner Feature Gate Required |
| -------------------- | ------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
| Snapshots | GA | On | [Snapshots and Restore](https://kubernetes-csi.github.io/docs/snapshot-restore-feature.html). | No |
| CSIMigration | Beta | On | [Migrating in-tree volume plugins to CSI](https://kubernetes.io/docs/concepts/storage/volumes/#csi-migration). | No |
| CSIStorageCapacity | Beta | On | Publish [capacity information](https://kubernetes.io/docs/concepts/storage/volumes/#storage-capacity) for the Kubernetes scheduler. | No |
| ReadWriteOncePod | Alpha | Off | [Single pod access mode for PersistentVolumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes). | No |
| HonorPVReclaimPolicy | Alpha | Off | [Honor the PV reclaim policy](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2644-honor-pv-reclaim-policy) | No |

All other external-provisioner features and the external-provisioner itself is considered GA and fully supported.

Expand Down Expand Up @@ -74,6 +74,8 @@ Note that the external-provisioner does not scale with more replicas. Only one e

* `--metrics-path`: The HTTP path where prometheus metrics will be exposed. Default is `/metrics`.

* `--enable-pprof`: Enable pprof profiling on the TCP network address specified by `--http-endpoint`.

* `--extra-create-metadata`: Enables the injection of extra PVC and PV metadata as parameters when calling `CreateVolume` on the driver (keys: "csi.storage.k8s.io/pvc/name", "csi.storage.k8s.io/pvc/namespace", "csi.storage.k8s.io/pv/name")

* `controller-publish-readonly`: This option enables PV to be marked as readonly at controller publish volume call if PVC accessmode has been set to ROX. Defaults to `false`.
Expand Down Expand Up @@ -132,14 +134,14 @@ The [design document](./doc/design.md) explains this in more detail.
### Topology support
When `Topology` feature is enabled and the driver specifies `VOLUME_ACCESSIBILITY_CONSTRAINTS` in its plugin capabilities, external-provisioner prepares `CreateVolumeRequest.AccessibilityRequirements` while calling `Controller.CreateVolume`. The driver has to consider these topology constraints while creating the volume. Below table shows how these `AccessibilityRequirements` are prepared:

[Delayed binding](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode) | Strict topology | [Allowed topologies](https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies) | Immediate Topology | [Resulting accessability requirements](https://github.com/container-storage-interface/spec/blob/master/spec.md#createvolume)
:---: |:---:|:---:|:---:|:---|
Yes | Yes | Irrelevant | Irrelevant | `Requisite` = `Preferred` = Selected node topology
Yes | No | No | Irrelevant | `Requisite` = Aggregated cluster topology<br>`Preferred` = `Requisite` with selected node topology as first element
Yes | No | Yes | Irrelevant | `Requisite` = Allowed topologies<br>`Preferred` = `Requisite` with selected node topology as first element
No | Irrelevant | Yes | Irrelevant | `Requisite` = Allowed topologies<br>`Preferred` = `Requisite` with randomly selected node topology as first element
No | Irrelevant | No | Yes | `Requisite` = Aggregated cluster topology<br>`Preferred` = `Requisite` with randomly selected node topology as first element
No | Irrelevant | No | No | `Requisite` and `Preferred` both nil
| [Delayed binding](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode) | Strict topology | [Allowed topologies](https://kubernetes.io/docs/concepts/storage/storage-classes/#allowed-topologies) | Immediate Topology | [Resulting accessability requirements](https://github.com/container-storage-interface/spec/blob/master/spec.md#createvolume) |
| :-------------------------------------------------------------------------------------------------: | :-------------: | :---------------------------------------------------------------------------------------------------: | :----------------: | :--------------------------------------------------------------------------------------------------------------------------- |
| Yes | Yes | Irrelevant | Irrelevant | `Requisite` = `Preferred` = Selected node topology |
| Yes | No | No | Irrelevant | `Requisite` = Aggregated cluster topology<br>`Preferred` = `Requisite` with selected node topology as first element |
| Yes | No | Yes | Irrelevant | `Requisite` = Allowed topologies<br>`Preferred` = `Requisite` with selected node topology as first element |
| No | Irrelevant | Yes | Irrelevant | `Requisite` = Allowed topologies<br>`Preferred` = `Requisite` with randomly selected node topology as first element |
| No | Irrelevant | No | Yes | `Requisite` = Aggregated cluster topology<br>`Preferred` = `Requisite` with randomly selected node topology as first element |
| No | Irrelevant | No | No | `Requisite` and `Preferred` both nil |

### Capacity support

Expand Down
14 changes: 13 additions & 1 deletion cmd/csi-provisioner/csi-provisioner.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"github.com/kubernetes-csi/external-provisioner/pkg/features"
"math/rand"
"net/http"
"net/http/pprof"
"os"
"strconv"
"strings"
Expand Down Expand Up @@ -83,8 +84,9 @@ var (
immediateTopology = flag.Bool("immediate-topology", true, "Immediate binding: pass aggregated cluster topologies for all nodes where the CSI driver is available (enabled, the default) or no topology requirements (if disabled).")
extraCreateMetadata = flag.Bool("extra-create-metadata", false, "If set, add pv/pvc metadata to plugin create requests as parameters.")
metricsAddress = flag.String("metrics-address", "", "(deprecated) The TCP network address where the prometheus metrics endpoint will listen (example: `:8080`). The default is empty string, which means metrics endpoint is disabled. Only one of `--metrics-address` and `--http-endpoint` can be set.")
httpEndpoint = flag.String("http-endpoint", "", "The TCP network address where the HTTP server for diagnostics, including metrics and leader election health check, will listen (example: `:8080`). The default is empty string, which means the server is disabled. Only one of `--metrics-address` and `--http-endpoint` can be set.")
httpEndpoint = flag.String("http-endpoint", "", "The TCP network address where the HTTP server for diagnostics, including pprof, metrics and leader election health check, will listen (example: `:8080`). The default is empty string, which means the server is disabled. Only one of `--metrics-address` and `--http-endpoint` can be set.")
metricsPath = flag.String("metrics-path", "/metrics", "The HTTP path where prometheus metrics will be exposed. Default is `/metrics`.")
enableProfile = flag.Bool("enable-pprof", false, "enable pprof profiling")

leaderElectionLeaseDuration = flag.Duration("leader-election-lease-duration", 15*time.Second, "Duration, in seconds, that non-leader candidates will wait to force acquire leadership. Defaults to 15 seconds.")
leaderElectionRenewDeadline = flag.Duration("leader-election-renew-deadline", 10*time.Second, "Duration, in seconds, that the acting leader will retry refreshing leadership before giving up. Defaults to 10 seconds.")
Expand Down Expand Up @@ -513,6 +515,16 @@ func main() {
promhttp.InstrumentMetricHandler(
reg,
promhttp.HandlerFor(gatherers, promhttp.HandlerOpts{})))

if *enableProfile {
klog.InfoS("Starting profiling", "endpoint", httpEndpoint)

mux.HandleFunc("/debug/pprof/", pprof.Index)
mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
}
go func() {
klog.Infof("ServeMux listening at %q", addr)
err := http.ListenAndServe(addr, mux)
Expand Down

0 comments on commit 7bd0757

Please sign in to comment.