Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1110: Enable support for Flow RTT #394

Merged
merged 9 commits into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions api/v1alpha1/flowcollector_webhook.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,9 @@ func (r *FlowCollector) ConvertTo(dstRaw conversion.Hub) error {
}

dst.Spec.Loki.Enable = restored.Spec.Loki.Enable
if restored.Spec.Agent.EBPF.EnablePktDrop != nil {
*dst.Spec.Agent.EBPF.EnablePktDrop = *restored.Spec.Agent.EBPF.EnablePktDrop
}

if restored.Spec.Agent.EBPF.EnableDNSTracking != nil {
*dst.Spec.Agent.EBPF.EnableDNSTracking = *restored.Spec.Agent.EBPF.EnableDNSTracking
if restored.Spec.Agent.EBPF.Features != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a slice u need to allocate the space then copy content

dst.Spec.Agent.EBPF.Features = make([]AgentFeature, len(restored.Spec.Agent.EBPF.Features))
copy(dst.Spec.Agent.EBPF.Features, restored.Spec.Agent.EBPF.Features)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

dst.Spec.Agent.EBPF.Features = restored.Spec.Agent.EBPF.Features
}

dst.Spec.Loki.StatusTLS = restored.Spec.Loki.StatusTLS
Expand Down
3 changes: 1 addition & 2 deletions api/v1alpha1/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 22 additions & 11 deletions api/v1beta1/flowcollector_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,19 @@ type FlowCollectorIPFIX struct {
OVNKubernetes OVNKubernetesConfig `json:"ovnKubernetes,omitempty" mapstructure:"-"`
}

// Agent feature, can be one of:<br>
// - `PKT_DROP`, to track packet drops.<br>
// - `DNS_TRACKING`, to track specific information on DNS traffic.<br>
// - `FLOW_RTT`, to track L4 latency. <i>Unsupported (*)</i><br>
Copy link
Contributor

@msherif1234 msherif1234 Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls remove
and other weired chars at the end of the above lines and every else in this file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msherif1234 I guess you are talking about the HTML blocks right ? That's not new to this PR: https://issues.redhat.com/browse/NETOBSERV-1104 ; it's related to asciidocs
Let's keep this as is and solve it in the related ticket

jotak marked this conversation as resolved.
Show resolved Hide resolved
// +kubebuilder:validation:Enum:="PKT_DROP";"DNS_TRACKING";"FLOW_RTT"
type AgentFeature string

const (
PktDrop AgentFeature = "PKT_DROP"
DNSTracking AgentFeature = "DNS_TRACKING"
FlowRTT AgentFeature = "FLOW_RTT"
Copy link
Contributor

@msherif1234 msherif1234 Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I am not mistaken Strings need to use Pascal not snake case ?

        PktDrop     AgentFeature = "PacketsDrop"
	DNSTracking AgentFeature = "DnsTacking"
	FlowRTT     AgentFeature = "FlowRtt"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, idk why I had in mind the best practice was upper case

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

)

// `FlowCollectorEBPF` defines a FlowCollector that uses eBPF to collect the flows information
type FlowCollectorEBPF struct {
// Important: Run "make generate" to regenerate code after modifying this file
Expand Down Expand Up @@ -219,19 +232,17 @@ type FlowCollectorEBPF struct {
// +optional
Debug DebugConfig `json:"debug,omitempty"`

// Enable the Packets drop flows logging feature. This feature requires mounting
// List of additional features to enable. They are all disabled by default. Enabling additional features may have performance impacts. Possible values are:<br>
// - `PKT_DROP`: enable the packets drop flows logging feature. This feature requires mounting
// the kernel debug filesystem, so the eBPF pod has to run as privileged.
// If the spec.agent.eBPF.privileged parameter is not set, an error is reported.
//+kubebuilder:default:=false
//+optional
EnablePktDrop *bool `json:"enablePktDrop,omitempty"`

// Enable the DNS tracking feature. This feature requires mounting
// If the `spec.agent.eBPF.privileged` parameter is not set, an error is reported.<br>
// - `DNS_TRACKING`: enable the DNS tracking feature. This feature requires mounting
// the kernel debug filesystem hence the eBPF pod has to run as privileged.
// If the spec.agent.eBPF.privileged parameter is not set, an error is reported.
//+kubebuilder:default:=false
//+optional
EnableDNSTracking *bool `json:"enableDNSTracking,omitempty"`
// If the `spec.agent.eBPF.privileged` parameter is not set, an error is reported.<br>
// - `FLOW_RTT` <i>Unsupported (*)</i>: allows enabling flow latency (RTT) calculations in the eBPF agent during TCP handshakes.
// This feature needs both INGRESS and EGRESS direction flow capture and will be disabled if they are not both enabled.<br>
// +optional
Copy link
Contributor

@msherif1234 msherif1234 Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add or we need to allow Features: block with no features ?

// +kubebuilder:validation:Required
// +kubebuilder:validation:MinItems:=1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need that check. We could simply initialize it empty []

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I don't even think an empty array initialization is needed, we don't do that in other places (e.g. exporters list)

Features []AgentFeature `json:"features,omitempty"`
}

// `FlowCollectorKafka` defines the desired Kafka config of FlowCollector
Expand Down
13 changes: 4 additions & 9 deletions api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 27 additions & 14 deletions bundle/manifests/flows.netobserv.io_flowcollectors.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2300,20 +2300,6 @@ spec:
they are only useful in edge debug or support scenarios.'
type: object
type: object
enableDNSTracking:
default: false
description: Enable the DNS tracking feature. This feature
requires mounting the kernel debug filesystem hence the
eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged
parameter is not set, an error is reported.
type: boolean
enablePktDrop:
default: false
description: Enable the Packets drop flows logging feature.
This feature requires mounting the kernel debug filesystem,
so the eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged
parameter is not set, an error is reported.
type: boolean
excludeInterfaces:
default:
- lo
Expand All @@ -2324,6 +2310,33 @@ spec:
items:
type: string
type: array
features:
description: 'List of additional features to enable. They
are all disabled by default. Enabling additional features
may have performance impacts. Possible values are:<br> -
`PKT_DROP`: enable the packets drop flows logging feature.
This feature requires mounting the kernel debug filesystem,
so the eBPF pod has to run as privileged. If the `spec.agent.eBPF.privileged`
parameter is not set, an error is reported.<br> - `DNS_TRACKING`:
enable the DNS tracking feature. This feature requires mounting
the kernel debug filesystem hence the eBPF pod has to run
as privileged. If the `spec.agent.eBPF.privileged` parameter
is not set, an error is reported.<br> - `FLOW_RTT` <i>Unsupported
(*)</i>: allows enabling flow latency (RTT) calculations
in the eBPF agent during TCP handshakes. This feature needs
both INGRESS and EGRESS direction flow capture and will
be disabled if they are not both enabled.<br>'
items:
description: Agent feature, can be one of:<br> - `PKT_DROP`,
to track packet drops.<br> - `DNS_TRACKING`, to track
specific information on DNS traffic.<br> - `FLOW_RTT`,
to track L4 latency. <i>Unsupported (*)</i><br>
enum:
- PKT_DROP
- DNS_TRACKING
- FLOW_RTT
type: string
type: array
imagePullPolicy:
default: IfNotPresent
description: '`imagePullPolicy` is the Kubernetes pull policy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,6 @@ metadata:
"ebpf": {
"cacheActiveTimeout": "5s",
"cacheMaxFlows": 100000,
"enableDNSTracking": false,
"enablePktDrop": false,
"excludeInterfaces": [
"lo"
],
Expand Down
41 changes: 27 additions & 14 deletions config/crd/bases/flows.netobserv.io_flowcollectors.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2287,20 +2287,6 @@ spec:
they are only useful in edge debug or support scenarios.'
type: object
type: object
enableDNSTracking:
default: false
description: Enable the DNS tracking feature. This feature
requires mounting the kernel debug filesystem hence the
eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged
parameter is not set, an error is reported.
type: boolean
enablePktDrop:
default: false
description: Enable the Packets drop flows logging feature.
This feature requires mounting the kernel debug filesystem,
so the eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged
parameter is not set, an error is reported.
type: boolean
excludeInterfaces:
default:
- lo
Expand All @@ -2311,6 +2297,33 @@ spec:
items:
type: string
type: array
features:
description: 'List of additional features to enable. They
are all disabled by default. Enabling additional features
may have performance impacts. Possible values are:<br> -
`PKT_DROP`: enable the packets drop flows logging feature.
This feature requires mounting the kernel debug filesystem,
so the eBPF pod has to run as privileged. If the `spec.agent.eBPF.privileged`
parameter is not set, an error is reported.<br> - `DNS_TRACKING`:
enable the DNS tracking feature. This feature requires mounting
the kernel debug filesystem hence the eBPF pod has to run
as privileged. If the `spec.agent.eBPF.privileged` parameter
is not set, an error is reported.<br> - `FLOW_RTT` <i>Unsupported
(*)</i>: allows enabling flow latency (RTT) calculations
in the eBPF agent during TCP handshakes. This feature needs
both INGRESS and EGRESS direction flow capture and will
be disabled if they are not both enabled.<br>'
items:
description: Agent feature, can be one of:<br> - `PKT_DROP`,
to track packet drops.<br> - `DNS_TRACKING`, to track
specific information on DNS traffic.<br> - `FLOW_RTT`,
to track L4 latency. <i>Unsupported (*)</i><br>
enum:
- PKT_DROP
- DNS_TRACKING
- FLOW_RTT
type: string
type: array
imagePullPolicy:
default: IfNotPresent
description: '`imagePullPolicy` is the Kubernetes pull policy
Expand Down
6 changes: 4 additions & 2 deletions config/samples/flows_v1beta1_flowcollector.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@ spec:
cacheActiveTimeout: 5s
cacheMaxFlows: 100000
privileged: false
enablePktDrop: false
enableDNSTracking: false
# features:
# - "PKT_DROP"
# - "DNS_TRACKING"
# - "FLOW_RTT"
interfaces: [ ]
excludeInterfaces: [ "lo" ]
logLevel: info
Expand Down
8 changes: 6 additions & 2 deletions controllers/consoleplugin/consoleplugin_objects.go
Original file line number Diff line number Diff line change
Expand Up @@ -350,13 +350,17 @@ func (b *builder) configMap() (*corev1.ConfigMap, string) {

var features []string
if helper.UseEBPF(b.desired) {
if helper.IsPktDropEnabled(b.desired) {
if helper.IsPktDropEnabled(&b.desired.Agent.EBPF) {
features = append(features, "pktDrop")
}

if helper.IsDNSTrackingEnabled(b.desired) {
if helper.IsDNSTrackingEnabled(&b.desired.Agent.EBPF) {
features = append(features, "dnsTracking")
}

if helper.IsFlowRTTEnabled(&b.desired.Agent.EBPF) {
features = append(features, "flowRTT")
}
}

config := map[string]interface{}{
Expand Down
17 changes: 12 additions & 5 deletions controllers/ebpf/agent_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ const (
envGoMemLimit = "GOMEMLIMIT"
envEnablePktDrop = "ENABLE_PKT_DROPS"
envEnableDNSTracking = "ENABLE_DNS_TRACKING"
envEnableFlowRTT = "ENABLE_RTT"
dushyantbehl marked this conversation as resolved.
Show resolved Hide resolved
envListSeparator = ","
)

Expand Down Expand Up @@ -185,10 +186,9 @@ func (c *AgentController) desired(ctx context.Context, coll *flowslatest.FlowCol
volumeMounts := c.volumes.GetMounts()
volumes := c.volumes.GetVolumes()

if helper.IsPktDropEnabled(&coll.Spec) || helper.IsDNSTrackingEnabled(&coll.Spec) {
if helper.IsFeatureEnabled(&coll.Spec.Agent.EBPF, flowslatest.PktDrop) || helper.IsFeatureEnabled(&coll.Spec.Agent.EBPF, flowslatest.DNSTracking) {
if !coll.Spec.Agent.EBPF.Privileged {
rlog.Error(fmt.Errorf("invalid configuration"),
"To use PktDrop and/or DNSTracking feature(s) privileged mode needs to be enabled", nil)
rlog.Error(fmt.Errorf("invalid configuration"), "To use PktDrop and/or DNSTracking feature(s) privileged mode needs to be enabled")
} else {
volume := corev1.Volume{
Name: bpfTraceMountName,
Expand Down Expand Up @@ -407,6 +407,13 @@ func (c *AgentController) setEnvConfig(coll *flowslatest.FlowCollector) []corev1
})
}

if helper.IsFlowRTTEnabled(&coll.Spec.Agent.EBPF) {
config = append(config, corev1.EnvVar{
Name: envEnableFlowRTT,
Value: "true",
})
}

// set GOMEMLIMIT which allows specifying a soft memory cap to force GC when resource limit is reached
// to prevent OOM
if coll.Spec.Agent.EBPF.Resources.Limits.Memory() != nil {
Expand All @@ -418,14 +425,14 @@ func (c *AgentController) setEnvConfig(coll *flowslatest.FlowCollector) []corev1
}
}

if helper.IsPktDropEnabled(&coll.Spec) {
if helper.IsPktDropEnabled(&coll.Spec.Agent.EBPF) {
config = append(config, corev1.EnvVar{
Name: envEnablePktDrop,
Value: "true",
})
}

if helper.IsDNSTrackingEnabled(&coll.Spec) {
if helper.IsDNSTrackingEnabled(&coll.Spec.Agent.EBPF) {
config = append(config, corev1.EnvVar{
Name: envEnableDNSTracking,
Value: "true",
Expand Down
7 changes: 1 addition & 6 deletions controllers/ebpf/internal/permissions/permissions.go
Original file line number Diff line number Diff line change
Expand Up @@ -155,14 +155,9 @@ func (c *Reconciler) reconcileOpenshiftPermissions(
} else {
scc.AllowedCapabilities = AllowedCapabilities
}
if (desired.EnablePktDrop != nil && *desired.EnablePktDrop) ||
(desired.EnableDNSTracking != nil && *desired.EnableDNSTracking) {
if helper.IsPktDropEnabled(desired) || helper.IsDNSTrackingEnabled(desired) {
scc.AllowHostDirVolumePlugin = true
}
if (desired.EnablePktDrop != nil && !*desired.EnablePktDrop) &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added to handle special case where u start with both features on then u just disable one since we don't compare current vs desired we could endup clearing the AllowHostDirVolumePlugin there was an issue about this by @memodi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msherif1234 scc is created a few lines above with AllowHostDirVolumePlugin not initialized , and since it's a bool , it means it's false already . So I don't see the point of setting it to false here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msherif1234 if you're talking about the nil-check, it's not relevant anymore now, as the new Features is a slice, unlike the previous EnablePktDrop / EnableDNSTracking that were bool pointers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u try the steps mentioned in the bug shared above to be sure, the reason I added the explicit false to make scc desired different than actual anyway I kind of forget the details its been awhile just make sure the steps showing in the above issue are fine

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tried to reproduce that old ticket, and it works fine: I get no error on the daemonset, pods are updated as expected, flows are flowing with dns/drops info

(desired.EnableDNSTracking != nil && !*desired.EnableDNSTracking) {
scc.AllowHostDirVolumePlugin = false
}
actual := &osv1.SecurityContextConstraints{}
if err := c.Get(ctx, client.ObjectKeyFromObject(scc), actual); err != nil {
if errors.IsNotFound(err) {
Expand Down
3 changes: 1 addition & 2 deletions controllers/flowcollector_controller_iso_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,7 @@ func flowCollectorIsoSpecs() {
ExcludeInterfaces: []string{},
Privileged: false,
KafkaBatchSize: 0,
EnablePktDrop: pointer.Bool(false),
EnableDNSTracking: pointer.Bool(false),
Features: nil,
},
},
ConsolePlugin: flowslatest.FlowCollectorConsolePlugin{
Expand Down
12 changes: 10 additions & 2 deletions controllers/flowlogspipeline/flp_common_objects.go
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ func (b *builder) addConnectionTracking(indexFields []string, lastStage config.P
},
}

if helper.IsPktDropEnabled(b.desired) {
if helper.IsPktDropEnabled(&b.desired.Agent.EBPF) {
outputPktDropFields := []api.OutputField{
{
Name: "PktDropBytes",
Expand Down Expand Up @@ -468,7 +468,7 @@ func (b *builder) addConnectionTracking(indexFields []string, lastStage config.P
outputFields = append(outputFields, outputPktDropFields...)
}

if helper.IsDNSTrackingEnabled(b.desired) {
if helper.IsDNSTrackingEnabled(&b.desired.Agent.EBPF) {
outDNSTrackingFields := []api.OutputField{
{
Name: "DnsFlagsResponseCode",
Expand All @@ -482,6 +482,14 @@ func (b *builder) addConnectionTracking(indexFields []string, lastStage config.P
outputFields = append(outputFields, outDNSTrackingFields...)
}

if helper.IsFlowRTTEnabled(&b.desired.Agent.EBPF) {
outputFields = append(outputFields, api.OutputField{
Name: "MaxTimeFlowRttNs",
Operation: "max",
Input: "TimeFlowRttNs",
})
}

// Connection tracking stage (only if LogTypes is not FLOWS)
if b.desired.Processor.LogTypes != nil && *b.desired.Processor.LogTypes != flowslatest.LogTypeFlows {
indexFields = append(indexFields, constants.LokiConnectionIndexFields...)
Expand Down
25 changes: 7 additions & 18 deletions docs/FlowCollector.md
Original file line number Diff line number Diff line change
Expand Up @@ -4070,24 +4070,6 @@ Agent configuration for flows extraction.
`debug` allows setting some aspects of the internal configuration of the eBPF agent. This section is aimed exclusively for debugging and fine-grained performance optimizations, such as GOGC and GOMAXPROCS env vars. Users setting its values do it at their own risk.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>enableDNSTracking</b></td>
<td>boolean</td>
<td>
Enable the DNS tracking feature. This feature requires mounting the kernel debug filesystem hence the eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged parameter is not set, an error is reported.<br/>
<br/>
<i>Default</i>: false<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>enablePktDrop</b></td>
<td>boolean</td>
<td>
Enable the Packets drop flows logging feature. This feature requires mounting the kernel debug filesystem, so the eBPF pod has to run as privileged. If the spec.agent.eBPF.privileged parameter is not set, an error is reported.<br/>
<br/>
<i>Default</i>: false<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>excludeInterfaces</b></td>
<td>[]string</td>
Expand All @@ -4097,6 +4079,13 @@ Agent configuration for flows extraction.
<i>Default</i>: [lo]<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>features</b></td>
<td>[]enum</td>
<td>
List of additional features to enable. They are all disabled by default. Enabling additional features may have performance impacts. Possible values are:<br> - `PKT_DROP`: enable the packets drop flows logging feature. This feature requires mounting the kernel debug filesystem, so the eBPF pod has to run as privileged. If the `spec.agent.eBPF.privileged` parameter is not set, an error is reported.<br> - `DNS_TRACKING`: enable the DNS tracking feature. This feature requires mounting the kernel debug filesystem hence the eBPF pod has to run as privileged. If the `spec.agent.eBPF.privileged` parameter is not set, an error is reported.<br> - `FLOW_RTT` <i>Unsupported (*)</i>: allows enabling flow latency (RTT) calculations in the eBPF agent during TCP handshakes. This feature needs both INGRESS and EGRESS direction flow capture and will be disabled if they are not both enabled.<br><br/>
</td>
<td>false</td>
</tr><tr>
<td><b>imagePullPolicy</b></td>
<td>enum</td>
Expand Down
Loading
Loading