Doc: regenerate API refs (FlowCollector + Flow Format) (#550)

* Doc: regenerate API refs (FlowCollector + Flow Format)

Flow format is now generated from a fields.yaml file stored here,
instead of the console plugin's type definition previously used, as it
became more likely to be outdated

A check is added to make sure any field referenced in the frontend
config is also defined in fields.yaml

* fix validation script

* Merge fields.yaml in static-frontend-config.yaml

Removed hack/flows-format-validation.sh, replaced with a go test
Add type information

* Minor tweaks for IBM style guide

- for instance => for example
- may => might

Also use head directive ":_mod-docs-content-type: REFERENCE"

* Update controllers/consoleplugin/config/static-frontend-config.yaml

Co-authored-by: Julien Pinsonneau <91894519+jpinsonneau@users.noreply.github.com>

* Update hack/asciidoc-flows-gen.sh

Co-authored-by: Julien Pinsonneau <91894519+jpinsonneau@users.noreply.github.com>

* Revert changes on flaky test

* regen doc

---------

Co-authored-by: Julien Pinsonneau <91894519+jpinsonneau@users.noreply.github.com>

apis/flowcollector/v1beta1/flowcollector_types.go

@@ -40,7 +40,7 @@ const (
 // Defines the desired state of the FlowCollector resource.
 // <br><br>
 // *: the mention of "unsupported", or "deprecated" for a feature throughout this document means that this feature
-// is not officially supported by Red Hat. It might have been, for instance, contributed by the community
+// is not officially supported by Red Hat. It might have been, for example, contributed by the community
 // and accepted without a formal agreement for maintenance. The product maintainers might provide some support
 // for these features as a best effort only.
 type FlowCollectorSpec struct {

apis/flowcollector/v1beta2/flowcollector_types.go

@@ -40,7 +40,7 @@ const (
 // Defines the desired state of the FlowCollector resource.
 // <br><br>
 // *: the mention of "unsupported", or "deprecated" for a feature throughout this document means that this feature
-// is not officially supported by Red Hat. It might have been, for instance, contributed by the community
+// is not officially supported by Red Hat. It might have been, for example, contributed by the community
 // and accepted without a formal agreement for maintenance. The product maintainers might provide some support
 // for these features as a best effort only.
 type FlowCollectorSpec struct {

bundle/manifests/flows.netobserv.io_flowcollectors.yaml

@@ -2485,7 +2485,7 @@ spec:
             description: 'Defines the desired state of the FlowCollector resource.
               <br><br> *: the mention of "unsupported", or "deprecated" for a feature
               throughout this document means that this feature is not officially supported
-              by Red Hat. It might have been, for instance, contributed by the community
+              by Red Hat. It might have been, for example, contributed by the community
               and accepted without a formal agreement for maintenance. The product
               maintainers might provide some support for these features as a best
               effort only.'
@@ -5179,7 +5179,7 @@ spec:
             description: 'Defines the desired state of the FlowCollector resource.
               <br><br> *: the mention of "unsupported", or "deprecated" for a feature
               throughout this document means that this feature is not officially supported
-              by Red Hat. It might have been, for instance, contributed by the community
+              by Red Hat. It might have been, for example, contributed by the community
               and accepted without a formal agreement for maintenance. The product
               maintainers might provide some support for these features as a best
               effort only.'

bundle/manifests/netobserv-operator.clusterserviceversion.yaml

@@ -768,7 +768,7 @@ spec:
     - description: '`FlowMetric` is the schema for the custom metrics API, which allows
         to generate more metrics out of flow logs. It is at an early stage of development
         (dev preview) and thus is currently not supported. Creating metrics with high
-        labels cardinality may impact the cluster stability.'
+        labels cardinality might impact the cluster stability.'
       displayName: Flow Metric
       kind: FlowMetric
       name: flowmetrics.flows.netobserv.io
@@ -800,7 +800,7 @@ spec:
 
     ### Kafka
 
-    [Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for instance [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
+    [Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for example [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
 
     ### Grafana
 
@@ -820,11 +820,11 @@ spec:
 
     A couple of settings deserve special attention:
 
-    - Agent (`spec.agent.type`) can be `EBPF` (default) or `IPFIX`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using [OVN-Kubernetes](https://github.com/ovn-org/ovn-kubernetes/) CNI. Other CNIs are not officially supported, but you may still be able to configure them manually if they allow IPFIX exports.
+    - Agent (`spec.agent.type`) can be `EBPF` (default) or `IPFIX`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using [OVN-Kubernetes](https://github.com/ovn-org/ovn-kubernetes/) CNI. Other CNIs are not officially supported, but you might still be able to configure them manually if they allow IPFIX exports.
 
     - Sampling (`spec.agent.ebpf.sampling` and `spec.agent.ipfix.sampling`): a value of `100` means: one flow every 100 is sampled. `1` means all flows are sampled. The lower it is, the more flows you get, and the more accurate are derived metrics, but the higher amount of resources are consumed. By default, sampling is set to 50 (ie. 1:50) for eBPF and 400 (1:400) for IPFIX. Note that more sampled flows also means more storage needed. We recommend to start with default values and refine empirically, to figure out which setting your cluster can manage.
 
-    - Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you may have to configure differently if you used another installation method.
+    - Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you might have to configure differently if you used another installation method.
 
     - Quick filters (`spec.consolePlugin.quickFilters`): configure preset filters to be displayed in the Console plugin. They offer a way to quickly switch from filters to others, such as showing / hiding pods network, or infrastructure network, or application network, etc. They can be tuned to reflect the different workloads running on your cluster. For a list of available filters, [check this page](https://github.com/netobserv/network-observability-operator/blob/1.0.4/docs/QuickFilters.md).
 

config/crd/bases/flows.netobserv.io_flowcollectors.yaml

@@ -2471,7 +2471,7 @@ spec:
             description: 'Defines the desired state of the FlowCollector resource.
               <br><br> *: the mention of "unsupported", or "deprecated" for a feature
               throughout this document means that this feature is not officially supported
-              by Red Hat. It might have been, for instance, contributed by the community
+              by Red Hat. It might have been, for example, contributed by the community
               and accepted without a formal agreement for maintenance. The product
               maintainers might provide some support for these features as a best
               effort only.'
@@ -5165,7 +5165,7 @@ spec:
             description: 'Defines the desired state of the FlowCollector resource.
               <br><br> *: the mention of "unsupported", or "deprecated" for a feature
               throughout this document means that this feature is not officially supported
-              by Red Hat. It might have been, for instance, contributed by the community
+              by Red Hat. It might have been, for example, contributed by the community
               and accepted without a formal agreement for maintenance. The product
               maintainers might provide some support for these features as a best
               effort only.'

config/csv/bases/netobserv-operator.clusterserviceversion.yaml

@@ -291,7 +291,7 @@ spec:
         which allows to generate more metrics out of flow logs.
         It is at an early stage of development (dev preview)
         and thus is currently not supported.
-        Creating metrics with high labels cardinality may impact the cluster stability.'
+        Creating metrics with high labels cardinality might impact the cluster stability.'
       displayName: Flow Metric
       kind: FlowMetric
       name: flowmetrics.flows.netobserv.io

config/descriptions/ocp.md

@@ -20,7 +20,7 @@ oc apply -f <(curl -L https://raw.githubusercontent.com/netobserv/documents/252b
 
 ### Kafka
 
-[Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for instance [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
+[Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for example [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
 
 ### Grafana
 
@@ -42,7 +42,7 @@ A couple of settings deserve special attention:
 
 - Sampling (`spec.agent.ebpf.sampling`): a value of `100` means: one flow every 100 is sampled. `1` means all flows are sampled. The lower it is, the more flows you get, and the more accurate are derived metrics, but the higher amount of resources are consumed. By default, sampling is set to 50 (ie. 1:50). Note that more sampled flows also means more storage needed. We recommend to start with default values and refine empirically, to figure out which setting your cluster can manage.
 
-- Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you may have to configure differently if you used another installation method.
+- Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you might have to configure differently if you used another installation method.
 
 - Quick filters (`spec.consolePlugin.quickFilters`): configure preset filters to be displayed in the Console plugin. They offer a way to quickly switch from filters to others, such as showing / hiding pods network, or infrastructure network, or application network, etc. They can be tuned to reflect the different workloads running on your cluster. For a list of available filters, [check this page](https://github.com/netobserv/network-observability-operator/blob/1.0.4/docs/QuickFilters.md).
 

config/descriptions/upstream.md

@@ -24,7 +24,7 @@ kubectl apply -f <(curl -L https://raw.githubusercontent.com/netobserv/documents
 
 ### Kafka
 
-[Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for instance [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
+[Apache Kafka](https://kafka.apache.org/) can optionally be used for a more resilient and scalable architecture. You can use for example [Strimzi](https://strimzi.io/), an operator-based distribution of Kafka for Kubernetes and OpenShift.
 
 ### Grafana
 
@@ -44,11 +44,11 @@ As it operates cluster-wide, only a single `FlowCollector` is allowed, and it ha
 
 A couple of settings deserve special attention:
 
-- Agent (`spec.agent.type`) can be `EBPF` (default) or `IPFIX`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using [OVN-Kubernetes](https://github.com/ovn-org/ovn-kubernetes/) CNI. Other CNIs are not officially supported, but you may still be able to configure them manually if they allow IPFIX exports.
+- Agent (`spec.agent.type`) can be `EBPF` (default) or `IPFIX`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using [OVN-Kubernetes](https://github.com/ovn-org/ovn-kubernetes/) CNI. Other CNIs are not officially supported, but you might still be able to configure them manually if they allow IPFIX exports.
 
 - Sampling (`spec.agent.ebpf.sampling` and `spec.agent.ipfix.sampling`): a value of `100` means: one flow every 100 is sampled. `1` means all flows are sampled. The lower it is, the more flows you get, and the more accurate are derived metrics, but the higher amount of resources are consumed. By default, sampling is set to 50 (ie. 1:50) for eBPF and 400 (1:400) for IPFIX. Note that more sampled flows also means more storage needed. We recommend to start with default values and refine empirically, to figure out which setting your cluster can manage.
 
-- Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you may have to configure differently if you used another installation method.
+- Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned above, but you might have to configure differently if you used another installation method.
 
 - Quick filters (`spec.consolePlugin.quickFilters`): configure preset filters to be displayed in the Console plugin. They offer a way to quickly switch from filters to others, such as showing / hiding pods network, or infrastructure network, or application network, etc. They can be tuned to reflect the different workloads running on your cluster. For a list of available filters, [check this page](https://github.com/netobserv/network-observability-operator/blob/1.0.4/docs/QuickFilters.md).
 

controllers/consoleplugin/config/config.go

@@ -62,6 +62,14 @@ type FilterConfig struct {
 	Placeholder            string `yaml:"placeholder,omitempty" json:"placeholder,omitempty"`
 }
 
+type FieldConfig struct {
+	Name        string `yaml:"name" json:"name"`
+	Type        string `yaml:"type" json:"type"`
+	Description string `yaml:"description" json:"description"`
+	LokiLabel   bool   `yaml:"lokiLabel,omitempty" json:"lokiLabel,omitempty"`
+	Filter      string `yaml:"filter,omitempty" json:"filter,omitempty"`
+}
+
 type Deduper struct {
 	Mark  bool `yaml:"mark" json:"mark"`
 	Merge bool `yaml:"merge" json:"merge"`
@@ -73,6 +81,7 @@ type FrontendConfig struct {
 	Sampling    int            `yaml:"sampling" json:"sampling"`
 	Features    []string       `yaml:"features" json:"features"`
 	Deduper     Deduper        `yaml:"deduper" json:"deduper"`
+	Fields      []FieldConfig  `yaml:"fields" json:"fields"`
 
 	PortNaming      flowslatest.ConsolePluginPortConfig `yaml:"portNaming,omitempty" json:"portNaming,omitempty"`
 	Filters         []FilterConfig                      `yaml:"filters,omitempty" json:"filters,omitempty"`