Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-448: Moving agent type, ipfix and ebpf config into their own common subsection. #162

Merged
merged 2 commits into from
Sep 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 13 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,12 @@ BUILD_SHA := $(shell git rev-parse --short HEAD)

# Other component versions when building bundle / release
PREVIOUS_VERSION ?= tochange
PLG_VERSION ?= v0.1.0 # console plugin
FLP_VERSION ?= v0.1.0 # flowlogs-pipeline
BPF_VERSION ?= v0.1.0 # eBPF agent
PLG_VERSION ?= v0.1.4 # console plugin
FLP_VERSION ?= v0.1.3 # flowlogs-pipeline
BPF_VERSION ?= v0.1.2 # eBPF agent

# Allows building bundles in Mac replacing BSD 'sed' command by GNU-compatible 'gsed'
SED ?= sed

# Port-forward (for loki/grafana deployments)
PORT_FWD ?= true
Expand Down Expand Up @@ -211,12 +214,12 @@ bundle: generate kustomize ## Generate bundle manifests and metadata, then valid
operator-sdk generate kustomize manifests -q
cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG)
cp config/samples/flows_v1alpha1_flowcollector.yaml config/samples/flows_v1alpha1_flowcollector_versioned.yaml
sed -i -r 's~flowlogs-pipeline:main~flowlogs-pipeline:$(FLP_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
sed -i -r 's~console-plugin:main~console-plugin:$(PLG_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
sed -i -r 's~ebpf-agent:main~ebpf-agent:$(BPF_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
sed -i -r 's~blob/[0-9]+\.[0-9]+\.[0-9]+(-rc[0-9]+)\?/~blob/$(VERSION)/~g' ./config/manifests/bases/netobserv-operator.clusterserviceversion.yaml
sed -i -r 's~replaces: netobserv-operator\.v.*~replaces: netobserv-operator\.$(PREVIOUS_VERSION)~' ./config/manifests/bases/netobserv-operator.clusterserviceversion.yaml
$(KUSTOMIZE) build config/manifests | sed -e 's~:container-image:~$(IMG)~' | sed -e 's~:created-at:~$(DATE)~' | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
$(SED) -i -r 's~flowlogs-pipeline:main~flowlogs-pipeline:$(FLP_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
$(SED) -i -r 's~console-plugin:main~console-plugin:$(PLG_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
$(SED) -i -r 's~ebpf-agent:main~ebpf-agent:$(BPF_VERSION)~' config/samples/flows_v1alpha1_flowcollector_versioned.yaml
$(SED) -i -r 's~blob/[0-9]+\.[0-9]+\.[0-9]+(-rc[0-9]+)\?/~blob/$(VERSION)/~g' ./config/manifests/bases/netobserv-operator.clusterserviceversion.yaml
$(SED) -i -r 's~replaces: netobserv-operator\.v.*~replaces: netobserv-operator\.$(PREVIOUS_VERSION)~' ./config/manifests/bases/netobserv-operator.clusterserviceversion.yaml
$(KUSTOMIZE) build config/manifests | $(SED) -e 's~:container-image:~$(IMG)~' | $(SED) -e 's~:created-at:~$(DATE)~' | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
operator-sdk bundle validate ./bundle

.PHONY: bundle-build
Expand Down Expand Up @@ -271,7 +274,7 @@ catalog-push: ## Push a catalog image.
# Deploy the catalog.
.PHONY: catalog-deploy
catalog-deploy:
sed -e 's~<IMG>~$(CATALOG_IMG)~' ./config/samples/catalog/catalog.yaml | kubectl apply -f -
$(SED) -e 's~<IMG>~$(CATALOG_IMG)~' ./config/samples/catalog/catalog.yaml | kubectl apply -f -

# Undeploy the catalog.
.PHONY: catalog-undeploy
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,9 @@ As it operates cluster-wide, only a single `FlowCollector` is allowed, and it ha

A couple of settings deserve special attention:

- Agent (`spec.agent`) can be `ebpf` (default) or `ipfix`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using OVN-Kubernetes CNI. Other CNIs are not officially supported, but you may still be able to configure them manually if they allow IPFIX exports.
- Agent (`spec.agent.type`) can be `EBPF` (default) or `IPFIX`. eBPF is recommended, as it should work in more situations and offers better performances. If you can't, or don't want to use eBPF, note that the IPFIX option is fully functional only when using OVN-Kubernetes CNI. Other CNIs are not officially supported, but you may still be able to configure them manually if they allow IPFIX exports.

- Sampling (`spec.ebpf.sampling` and `spec.ipfix.sampling`): 24/7, 1:1 sampled flow collection may consume a non-negligible amount of resources. While we are doing our best to make it a viable option in production, it is still sometimes necessary to mitigate by setting a sampling ratio. A value of `100` means: one flow every 100 is sampled. `1` means all flows are sampled. The lower it is, the more flows you get, and the more accurate are derived metrics. By default, sampling is set to 50 (ie. 1:50) for eBPF and 400 (1:400) for IPFIX. Note that more sampled flows also means more storage needed. We recommend to start with default values and refine empirically, to figure out which setting your cluster can manage.
- Sampling (`spec.agent.ebpf.sampling` and `spec.agent.ipfix.sampling`): 24/7, 1:1 sampled flow collection may consume a non-negligible amount of resources. While we are doing our best to make it a viable option in production, it is still sometimes necessary to mitigate by setting a sampling ratio. A value of `100` means: one flow every 100 is sampled. `1` means all flows are sampled. The lower it is, the more flows you get, and the more accurate are derived metrics. By default, sampling is set to 50 (ie. 1:50) for eBPF and 400 (1:400) for IPFIX. Note that more sampled flows also means more storage needed. We recommend to start with default values and refine empirically, to figure out which setting your cluster can manage.

- Loki (`spec.loki`): configure here how to reach Loki. The default values match the Loki quick install paths mentioned in the _Getting Started_ section, but you may have to configure differently if you used another installation method.

Expand Down Expand Up @@ -204,7 +204,7 @@ It should return some json in this form:

### Everything seems correctly deployed but there isn't any flow showing up

If using IPFIX (ie. `spec.agent` is `ipfix` in FlowCollector), wait 10 minutes and check again. There is sometimes a delay, up to 10 minutes, before the flows appear. This is due to the IPFIX protocol requiring exporter and collector to exchange record template definitions as a preliminary step. The eBPF agent doesn't have such a delay.
If using IPFIX (ie. `spec.agent.type` is `IPFIX` in FlowCollector), wait 10 minutes and check again. There is sometimes a delay, up to 10 minutes, before the flows appear. This is due to the IPFIX protocol requiring exporter and collector to exchange record template definitions as a preliminary step. The eBPF agent doesn't have such a delay.

Else, check for any suspicious error in logs, especially in the `flowlogs-pipeline` pods and the eBPF agent pods. You may also take a look at prometheus metrics prefixed with `netobserv_`: they can give you clues if flows are processed, if errors are reported, etc.

Expand Down
55 changes: 32 additions & 23 deletions api/v1alpha1/flowcollector_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
Expand All @@ -24,8 +24,8 @@ import (
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

const (
AgentIPFIX = "ipfix"
AgentEBPF = "ebpf"
AgentIPFIX = "IPFIX"
AgentEBPF = "EBPF"
)

// Please notice that the FlowCollectorSpec's properties MUST redefine one of the default
Expand All @@ -39,29 +39,13 @@ const (
type FlowCollectorSpec struct {
// Important: Run "make generate" to regenerate code after modifying this file

//+kubebuilder:default:=""
// Namespace where NetObserv pods are deployed.
// If empty, the namespace of the operator is going to be used.
// +optional
Namespace string `json:"namespace,omitempty"`

//+kubebuilder:validation:Enum=ipfix;ebpf
//+kubebuilder:default:=ebpf
// Select the flows tracing agent. Possible values are "ipfix" to use
// the IPFIX collector, or "ebpf" (default) to use NetObserv eBPF agent.
// eBPF is recommended, as it should work in more situations and offers better performances.
// When using IPFIX with OVN-Kubernetes CNI, NetObserv will configure OVN's IPFIX exporter.
// Other CNIs are not supported, they could work but necessitate manual configuration.
Agent string `json:"agent"`

// Settings related to IPFIX-based flow reporter when the "agent" property is set
// to "ipfix".
// +kubebuilder:default:={sampling:400}
IPFIX FlowCollectorIPFIX `json:"ipfix,omitempty"`

// Settings related to eBPF-based flow reporter when the "agent" property is set
// to "ebpf".
// +kubebuilder:default={imagePullPolicy:"IfNotPresent"}
EBPF FlowCollectorEBPF `json:"ebpf,omitempty"`
// +kubebuilder:default:={type:"EBPF"}
Agent FlowCollectorAgent `json:"agent"`

// Settings related to the flowlogs-pipeline component, which collects and enriches the flows, and produces metrics.
FlowlogsPipeline FlowCollectorFLP `json:"flowlogsPipeline,omitempty"`
Expand All @@ -84,6 +68,31 @@ type FlowCollectorSpec struct {
OVNKubernetes OVNKubernetesConfig `json:"ovnKubernetes,omitempty"`
}

// FlowCollectorAgent is a discriminated union that allows to select either ipfix or ebpf, but does not
// allow defining both fields.
// +union
type FlowCollectorAgent struct {
// Select the flows tracing agent. Possible values are "IPFIX" (default) to use
// the IPFIX collector, or "EBPF" to use NetObserv eBPF agent. When using IPFIX with OVN-Kubernetes
// CNI, NetObserv will configure OVN's IPFIX exporter. Other CNIs are not supported, they could
// work but require manual configuration.
// +unionDiscriminator
// +kubebuilder:validation:Enum:="IPFIX";"EBPF"
// +kubebuilder:validation:Required
// +kubebuilder:default:=EBPF
Type string `json:"type"`

// Settings related to IPFIX-based flow reporter when the "agent.type" property is set
// to "IPFIX".
// +optional
IPFIX FlowCollectorIPFIX `json:"ipfix,omitempty"`

// Settings related to eBPF-based flow reporter when the "agent.type" property is set
// to "EBPF".
// +optional
EBPF FlowCollectorEBPF `json:"ebpf,omitempty"`
}

// FlowCollectorIPFIX defines a FlowCollector that uses IPFIX on OVN-Kubernetes to collect the
// flows information
type FlowCollectorIPFIX struct {
Expand Down Expand Up @@ -495,7 +504,7 @@ type FlowCollectorStatus struct {
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
//+kubebuilder:resource:scope=Cluster
//+kubebuilder:printcolumn:name="Agent",type="string",JSONPath=`.spec.agent`
//+kubebuilder:printcolumn:name="Agent",type="string",JSONPath=`.spec.agent.type`
//+kubebuilder:printcolumn:name="Kafka",type="boolean",JSONPath=`.spec.kafka.enable`
//+kubebuilder:printcolumn:name="Status",type="string",JSONPath=".status.conditions[*].reason"

Expand Down
20 changes: 18 additions & 2 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion bundle.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/
LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/
LABEL operators.operatorframework.io.bundle.package.v1=netobserv-operator
LABEL operators.operatorframework.io.bundle.channels.v1=alpha
LABEL operators.operatorframework.io.metrics.builder=operator-sdk-v1.22.2
LABEL operators.operatorframework.io.metrics.builder=operator-sdk-v1.23.0
LABEL operators.operatorframework.io.metrics.mediatype.v1=metrics+v1
LABEL operators.operatorframework.io.metrics.project_layout=go.kubebuilder.io/v3

Expand Down
Loading