chore: attempt to improve documentation (#496)

Signed-off-by: Kavindu Dodanduwa <kavindudodanduwa@gmail.com> Signed-off-by: Kavindu Dodanduwa <Kavindu-Dodan@users.noreply.github.com> Co-authored-by: Giovanni Liva <giovanni.liva@dynatrace.com> Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
open-feature · Jul 10, 2023 · 603e74e · 603e74e
1 parent 1ec3183
commit 603e74e
Show file tree

Hide file tree

Showing 13 changed files with 2,024 additions and 173 deletions.
diff --git a/docs/README.md b/docs/README.md
@@ -1,24 +1,41 @@
-# Docs
+# Documentation
 
-This directory contains all OpenFeature Operator documentation, see table of contents below:
+This directory contains OpenFeature Operator documentation.
 
-## Usage
+Interested in having OpenFeature Operator up and running within 5 minutes? Follow the quick start guide found below.
 
-Follow the documentation below to deploy the open feature operator to your local cluster, followed by a simple example app using `curl` to evaluate a static flag.
+- [Quick Start](./quick_start.md)
+
+Follow the detailed installation guide to deploy open feature operator to your local cluster.
 
 - [Installation](./installation.md)
-- [Getting Started](./getting_started.md)
 
 ## Configuration
 
-Configuration of the deployed sidecars is handled through the `FeatureFlagConfiguration` CRs defined in the `openfeature.dev/featureflagconfiguration` annotation of a deployed `PodSpec`. 
-> Further configuration of the operator will be possible in the future, to help contribute [click here](https://github.com/open-feature/open-feature-operator/issues)
+Configuration of the deployed sidecars is handled through the `FlagSourceConfiguration` custom resources defined at`openfeature.dev/flagsourceconfiguration` annotation of the deployed `PodSpec`.
+
+The relationship between the deployment and custom resources is highlighted in the diagram below,
+
+```mermaid
+flowchart TD
+    A[Pod]-->|Annotation: openfeature.dev/flagsourceconfiguration| B[FlagSourceConfiguration CR]
+    B--> |Flag source| C[FeatureFlagConfiguration CR]
+    B--> |Flag source| D[HTTP sync]
+    B--> |Flag source| E[Filepath sync]
+    B--> |Flag source| F[GRPC sync]
+    B--> |Flag source| G[flagd-proxy]
+```
+
+To configure and understand more,
 
-- [Annotations](./annotations.md)
-- [FeatureFlagConfigurations](./feature_flag_configuration.md)
-- [FlagSourceConfiguration](./flag_source_configuration.md)
+- Deployment configurations: [Annotations](./annotations.md)
+- Define flag sources for the deployment: [FlagSourceConfiguration](./flag_source_configuration.md)
+- Define feature flags as custom resource: [FeatureFlagConfigurations](./feature_flag_configuration.md)
 
 ## Other Resources
 - [Permissions](./permissions.md)
+- [Concepts](./concepts.md)
 - [Development Notes](./development_notes.md)
-- [flagd Kube Proxy](./flagd_proxy.md)
+- [Threat model](./threat_model.md)
+- [flagd Kube Proxy](./flagd_proxy.md)
+- [API Reference](./crds.md)
diff --git a/docs/annotations.md b/docs/annotations.md
@@ -3,7 +3,9 @@
 The following annotations are used by the operator to control the injection and define configuration of the flagd sidecar.
 
 ### `openfeature.dev/enabled`
+
 When a value of `"true"` is provided, the operator will inject a flagd sidecar into the annotated pods.  
+
 Example: 
 ```yaml
     metadata:
@@ -12,10 +14,14 @@ Example:
 ```
 
 ### `openfeature.dev/flagsourceconfiguration`
-This annotation specifies the names of the FlagSourceConfigurations used to configure the injected flagd sidecar.
+
+This annotation specifies the names of the `FlagSourceConfigurations` used to configure the injected flagd sidecar.
 The annotation value is a comma separated list of values following one of 2 patterns: {NAME} or {NAMESPACE}/{NAME}. 
-If no namespace is provided it is assumed that the CR is within the same namespace as the deployed pod.
-If multiple CRs are provided, they are merged with the latest taking precedence, for example, in the scenario below, `config-B` will take priority in the merge, replacing duplicated values that are set in `config-A`.
+
+If no namespace is provided, it is assumed that the custom resource is within the **same namespace** as the annotated pod.
+If multiple CRs are provided, they are merged with the latest taking precedence. 
+
+For example, in the scenario below, `config-B` will take priority in the merge, replacing duplicated values that are set in `config-A`.
 
 Example:
 ```yaml
@@ -26,9 +32,14 @@ Example:
 ```
 
 ### `openfeature.dev/allowkubernetessync`
-*This annotation is used internally by the operator.*  
-This annotation is used to mark pods which should have their permissions backfilled in the event of an upgrade. When the OFO manager pod is started, all `Service Accounts` of any `Pods` with this annotation set to `"true"` will be added to the `flagd-kubernetes-sync` `Cluster Role Binding`.
+*This annotation is used INTERNALLY by the operator.*
+
+This annotation is used to mark pods which should have their permissions backfilled in the event of an upgrade.
+When the OFO manager pod is started, all `Service Accounts` of any `Pods` with this annotation set to `"true"` will be added to the `flagd-kubernetes-sync` `Cluster Role Binding`.
+
+## Deprecated annotations
 
+Given below are references to **deprecated** annotations used by previous versions of the operator.
 
 ### `openfeature.dev/featureflagconfiguration`
 *This annotation is DEPRECATED in favour of the `openfeature.dev/flagsourceconfiguration` annotation and should no longer be used.* 

diff --git a/docs/concepts.md b/docs/concepts.md
@@ -1,23 +1,36 @@
 # Concepts
 
-This document covers some key concepts related to the OpenFeature operator (OFO). For general OpenFeature or feature flag concepts, see our [online documentation](https://openfeature.dev/docs/reference/intro).
+This document covers some key concepts related to the OpenFeature operator (OFO).
+
+For general OpenFeature or feature flag concepts, see our [online documentation](https://openfeature.dev/docs/reference/intro).
 
 ## Architecture
 
 The high level architecture of the operator is as follows:  
-<p align="center">
+<p>
     <img src="../images/arch-0.png" width="650">
 </p>
 
-## Modes of operation
+## Modes of flag syncs
+
+- Kubernetes:  sync configuration which configures injected flagd sidecar instances to monitor the Kubernetes API 
+  for changes in flag configuration custom resources (`FeatureFlagConfiguration`).
+- filepath:  sync configuration which creates and mounts ConfigMap files from flag configuration custom  resources
+  (`FeatureFlagConfiguration`) and configures injected flagd sidecar instances to monitor them.
+- grpc: sync configuration which listen for flagd compatible grpc stream
+- http: sync configuration which watch and periodically poll flagd compatible http endpoint
+- [flagd-proxy](./flagd_proxy.md)
+
+Each approach have their advantages and disadvantages. 
 
-OFO supports two primary modes of operation for supplying flag configurations to sidecar flagd instances:
+The kubernetes, grpc and flagd-proxy sync configuration has the advantage of providing near real-time flag updates(on the order of seconds) to the flagd sidecar. 
 
-- The `"kubernetes"` sync configuration (default), which configures injected flagd sidecar instances to monitor the Kubernetes API for changes in flag configuration custom resources (`featureflagconfiguration`).
-- The `"filepath"` sync configuration, which creates and mounts ConfigMap files from flag configuration custom resources (`featureflagconfiguration`) and configures injected flagd sidecar instances to monitor them.
+For example, Kubernetes syncs require the flagd sidecar(and consequently the workload pod) to communicate with the 
+Kubernetes API. This may violate the security or network policies of some organizations.
 
-Both approaches have their advantages and disadvantages. The `"kubernetes"` sync configuration has the advantage of providing near-realtime flag updates (on the order of seconds) to the flagd sidecar, and therefore the associated workload. However, it also requires the flagd sidecar (and consequently the workload pod) to communicate with the Kubernetes API. This may violate the security or network policies of some organizations. The `"filepath"` provider requires no such communication, but relies on the fact that [Kubernetes automatically updates mounted ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automatically). The disadvantage of this approach is that flag configuration updates may take as long as two minutes to propagate, depending on cluster configuration:
+The `"filepath"` provider requires no such communication, but relies on the fact that [Kubernetes automatically updates mounted ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automatically). 
+The disadvantage of this approach is that flag configuration updates may take as long as two minutes to propagate, depending on cluster configuration:
 
 > "the total delay from the moment when the ConfigMap is updated to the moment when new keys are projected to the Pod can be as long as the kubelet sync period + cache propagation delay"
 
-Consider your individual requirements and select the configuration most appropriate for your needs. Note that the sync provider configuration to use can be configured globally and overridden per `featureflagconfiguration`. For details, see [the sync provider documentation](./feature_flag_configuration.md#syncprovider).
+Consider your individual requirements and select the configuration most appropriate for your needs.
diff --git a/docs/feature_flag_configuration.md b/docs/feature_flag_configuration.md
@@ -18,6 +18,11 @@ spec:
         defaultVariant: "bar"
 ```
 
+In the example above, we have defined a `String` type feature flag named `foo` and it is in the `ENABLED` state. 
+It has variants of `bar` and `baz`, referring to respected values of `BAR` and `BAZ`.
+The default variant is set to`bar`.
+
 ## featureFlagSpec
 
-The `featureFlagSpec` is an object representing the flag configurations themselves, the documentation for this object can be found [here](https://github.com/open-feature/flagd/blob/main/docs/configuration/flag_configuration.md).
+The `featureFlagSpec` is an object representing the flag configurations themselves.
+The documentation for this object can be found [here](https://github.com/open-feature/flagd/blob/main/docs/configuration/flag_configuration.md).
diff --git a/docs/flag_source_configuration.md b/docs/flag_source_configuration.md
@@ -1,13 +1,111 @@
 # Flag Source configuration
 
-The injected sidecar is configured using the `FlagSourceConfiguration` CRD, the `openfeature.dev/flagsourceconfiguration` annotation is used to assign `Pods` with their respective `FlagSourceConfiguration` CRs. The annotation value is a comma separated list of values following one of 2 patterns: {NAME} or {NAMESPACE}/{NAME}. If no namespace is provided, it is assumed that the CR is within the same namespace as the deployed pod, for example:
+The injected sidecar is configured using the `FlagSourceConfiguration` custom resource definition. 
+The `openfeature.dev/flagsourceconfiguration` annotation is used to assign Pods with their respective`FlagSourceConfiguration` custom resources.
+
+A minimal example of a `FlagSourceConfiguration` is given below,
+
+```yaml
+apiVersion: core.openfeature.dev/v1alpha3
+kind: FlagSourceConfiguration
+metadata:
+  name: flag-source-configuration
+spec:
+  sources:                        # flag sources for the injected flagd
+    - source: flags/sample-flags  # FlagSourceConfiguration - namespace/name
+      provider: kubernetes        # kubernetes flag source backed by FlagSourceConfiguration custom resource
+  port: 8080                      # port of the flagd sidecar
+```
+
+## Feature flag sources
+
+This section explains how to configure feature flag sources to injected flag sidecar.
+
+`FlagSourceConfiguration` support multiple flag sources. Sources are configured as a list and given below are supported sources and their configurations,
+
+### kubernetes aka `FeatureFlagConfiguration`
+
+This is `FeatureFlagConfiguration` custom resource backed flagd feature flag definition.
+Read more on the custom resource at the dedicated documentation of [FeatureFlagConfiguration](./feature_flag_configuration.md)
+
+To refer this custom resource in `FlagSourceConfiguration`, provider type `kubernetes` is used as below example,
+
+```yaml
+sources:                        
+  - source: flags/sample-flags  # FeatureFlagConfiguration - namespace/custom_resource_name
+    provider: kubernetes        # kubernetes flag source backed by FeatureFlagConfiguration custom resource
+```
+
+### flagd-proxy
+
+`flagd-proxy` is an alternative to direct resource access on `FeatureFlagConfiguration` custom resources.
+This source type is useful when there is a need for restricting workload permissions and/or to reduce k8s API load.
+
+Read more about proxy approach to access kubernetes resources: [flagd-proxy](./flagd_proxy.md)
+
+### filepath
+
+Injected sidecar can use volume mounted files as flag sources. 
+For this, provider type `filepath` is used as below example,
+
+```yaml
+sources:                        
+  - source: /etc/flagd/config.json 
+    provider: filepath          
 ```
+
+### http
+
+Feature flags can be sources from a http endpoint using provider type `http`,
+
+```yaml
+sources:
+  - source: http://my-flag-source.json
+    provider: http
+    httpSyncBearerToken: token                  # optional bearer token for the http connection
+```
+
+### grpc
+
+Given below is an example configuration with provider type `grpc` and supported options, 
+
+```yaml
+sources:                        
+  - source: my-flag-source:8080
+    provider: grpc
+    certPath: /certs/ca.cert                    # certificate for tls connectivity
+    tls: true                                   # enforce tls connectivity
+    providerID: flagd-weatherapp-sidecar        # identifier for this connection 
+    selector: 'source=database,app=weatherapp'  # flag filtering options
+```
+
+## Sidecar configurations
+
+`FlagSourceConfiguration` further allows to provide configurations to the injected flagd sidecar.
+Table given below is non-exhaustive list of overriding options,
+
+| Configuration | Explanation                   | Default                    |
+|---------------|-------------------------------|----------------------------|
+| port          | Flag evaluation endpoint port | 8013                       |
+| metricsPort   | Metrics port                  | 8014                       |
+| evaluator     | Evaluator to use              | json                       |
+| image         | flagD image                   | ghcr.io/open-feature/flagd |
+| tag           | flagD image tag               | Latest tag                 |
+| probesEnabled | Enable/Disable health probes  | true                       |
+
+## Merging of configurations
+
+The annotation value is a comma separated list of values following one of two patterns: {NAME} or {NAMESPACE}/{NAME}. 
+If no namespace is provided, it is assumed that the CR is within the same namespace as the deployed pod, for example:
+
+```yaml
     metadata:
         namespace: test-ns
         annotations:
             openfeature.dev/enabled: "true"
-            openfeature.dev/flagsourceconfiguration:"config-A, test-ns-2/config-B"
+            openfeature.dev/flagsourceconfiguration: "config-A, test-ns-2/config-B"
 ```
+
 In this example, 2 CRs are being used to configure the injected container (by default the operator uses the `flagd:main` image), `config-A` (which is assumed to be in the namespace `test-ns`) and `config-B` from the `test-ns-2` namespace, with `config-B` taking precedence in the configuration merge.
 
 The `FlagSourceConfiguration` version `v1alpha3` CRD defines a CR with the following example structure, the documentation for this CRD can be found [here](crds.md#flagsourceconfiguration):
@@ -16,7 +114,7 @@ The `FlagSourceConfiguration` version `v1alpha3` CRD defines a CR with the follo
 apiVersion: core.openfeature.dev/v1alpha3
 kind: FlagSourceConfiguration
 metadata:
-    name: flagsourceconfiguration-sample
+    name: flag-source-sample
 spec:
     metricsPort: 8080
     Port: 80