updated README and yml files #249

KalmanMeth · 2024-01-18T14:04:35Z

Description

The stand-alone ebpf and flp were implemented against an older version of flp.
This PR updateds the README and yml files to work with the current flp.

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

KalmanMeth · 2024-01-18T14:06:31Z

@jpinsonneau The flp-daemonset-cap.yml doesn't yet work properly. Some permissions probably need to be adjusted.

codecov · 2024-01-18T14:08:56Z

Codecov Report

Attention: 24 lines in your changes are missing coverage. Please review.

Comparison is base (6f1d70a) 33.86% compared to head (a023a01) 34.06%.
Report is 1 commits behind head on main.

Files	Patch %	Lines
pkg/exporter/proto.go	0.00%	16 Missing and 2 partials ⚠️
pkg/flow/deduper.go	85.00%	2 Missing and 1 partial ⚠️
pkg/utils/utils.go	50.00%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #249      +/-   ##
==========================================
+ Coverage   33.86%   34.06%   +0.20%     
==========================================
  Files          39       39              
  Lines        3479     3508      +29     
==========================================
+ Hits         1178     1195      +17     
- Misses       2233     2243      +10     
- Partials       68       70       +2

Flag	Coverage Δ
unittests	`34.06% <54.71%> (+0.20%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jpinsonneau

Fixed perms in a023a01 and changed the instructions order to create namespace properly at first command.

@KalmanMeth note that if you are using oc command, ommiting labels section for pod-security is actually working !

Using kubectl, you must specify labels 🙃

jpinsonneau · 2024-01-19T12:10:42Z

deployments/README.md

+kubectl create namespace netobserv
+curl -S -L https://raw.githubusercontent.com/netobserv/documents/main/examples/zero-click-loki/1-storage.yaml | kubectl create -n netobserv -f - 
+curl -S -L https://raw.githubusercontent.com/netobserv/documents/main/examples/zero-click-loki/2-loki.yaml       | kubectl create -n netobserv -f - 
+'''


Do we really want to keep loki in the examples ?

We could simply have stdout here and then the users can dig into different export capabilities.

I don't have a strong preference. The existing example assumed loki, so I kept it that way. I simply added the instructions on how to bring up loki so that it will work.

jpinsonneau · 2024-01-19T12:46:51Z

I'm getting flows in FLP after applying the new changes:

Starting flowlogs-pipeline:
=====
Build Version: -dirty
Build Date: 2024-01-17 17:34

Using configuration:
{
    "PipeLine": "[{\"name\":\"ingest\"},{\"follows\":\"ingest\",\"name\":\"enrich\"},{\"follows\":\"enrich\",\"name\":\"loki\"}]",
    "Parameters": "[{\"ingest\":{\"grpc\":{\"port\":9999},\"type\":\"grpc\"},\"name\":\"ingest\"},{\"name\":\"enrich\",\"transform\":{\"network\":{\"rules\":[{\"input\":\"SrcAddr\",\"output\":\"SrcK8S\",\"type\":\"add_kubernetes\"},{\"input\":\"DstAddr\",\"output\":\"DstK8S\",\"type\":\"add_kubernetes\"}]},\"type\":\"network\"}},{\"name\":\"loki\",\"write\":{\"loki\":{\"labels\":[\"SrcK8S_Namespace\",\"SrcK8S_OwnerName\",\"DstK8S_Namespace\",\"DstK8S_OwnerName\",\"FlowDirection\"],\"staticlabels\":{\"app\":\"netobserv-flowcollector\"},\"timestamplabel\":\"TimeFlowEndMs\",\"timestampscale\":\"1ms\",\"url\":\"http://loki.netobserv.svc:3100\"},\"type\":\"loki\"}}]",
    "MetricsSettings": "",
    "Health": {
        "Address": "0.0.0.0",
        "Port": "8080"
    },
    "Profile": {
        "Port": 0
    }
}
time=2024-01-19T12:41:23Z level=debug msg=opts.PipeLine = [{"name":"ingest"},{"follows":"ingest","name":"enrich"},{"follows":"enrich","name":"loki"}] 
time=2024-01-19T12:41:23Z level=debug msg=stages = [{ingest } {enrich ingest} {loki enrich}] 
time=2024-01-19T12:41:23Z level=debug msg=params = [{ingest 0xc000130f00 <nil> <nil> <nil> <nil>} {enrich <nil> 0xc0002bd080 <nil> <nil> <nil>} {loki <nil> <nil> <nil> <nil> 0xc0002bd0e0}] 
time=2024-01-19T12:41:23Z level=info msg=using default metrics settings
time=2024-01-19T12:41:23Z level=debug msg=entering SetupElegantExit
time=2024-01-19T12:41:23Z level=debug msg=registered exit signal channel
time=2024-01-19T12:41:23Z level=debug msg=exiting SetupElegantExit
time=2024-01-19T12:41:23Z level=info msg=StartServerAsync: addr = :9090 component=prometheus
time=2024-01-19T12:41:23Z level=debug msg=entering newPipelineFromIngester
time=2024-01-19T12:41:23Z level=debug msg=stages = [{ingest } {enrich ingest} {loki enrich}] 
time=2024-01-19T12:41:23Z level=debug msg=configParams = [{ingest 0xc000130f00 <nil> <nil> <nil> <nil>} {enrich <nil> 0xc0002bd080 <nil> <nil> <nil>} {loki <nil> <nil> <nil> <nil> 0xc0002bd0e0}] 
time=2024-01-19T12:41:23Z level=debug msg=stage = ingest
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = ingest
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0]
time=2024-01-19T12:41:23Z level=debug msg=stage = enrich
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = enrich
time=2024-01-19T12:41:23Z level=debug msg=starting kubernetes informers, waiting for synchronization
time=2024-01-19T12:41:23Z level=debug msg=kubernetes informers started
time=2024-01-19T12:41:23Z level=debug msg=starting kubernetes metadata informers, waiting for synchronization
time=2024-01-19T12:41:23Z level=debug msg=kubernetes metadata informers started
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10]
time=2024-01-19T12:41:23Z level=debug msg=stage = loki
time=2024-01-19T12:41:23Z level=debug msg=findStageType: stage = loki
time=2024-01-19T12:41:23Z level=debug msg=entering NewWriteLoki component=write.Loki
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10 0xc0001a2d90]
time=2024-01-19T12:41:23Z level=debug msg=pipeline = [0xc00024c7e0 0xc0001a2a10 0xc0001a2d90]
time=2024-01-19T12:41:23Z level=info msg=connecting stages: ingest --> enrich
time=2024-01-19T12:41:23Z level=info msg=connecting stages: enrich --> loki
time=2024-01-19T12:41:27Z level=debug msg=Ingested 875 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:27Z level=debug msg=Ingested 920 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:32Z level=debug msg=Ingested 1069 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:32Z level=debug msg=Ingested 1027 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:37Z level=debug msg=Ingested 1125 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:37Z level=debug msg=Ingested 844 records component=ingest.GRPCProtobuf
time=2024-01-19T12:41:42Z level=debug msg=Ingested 1094 records component=ingest.GRPCProtobuf

KalmanMeth · 2024-01-21T12:07:31Z

I'm getting flows in FLP after applying the new changes:

To get rid of these messages, we need to change log-level from debug to info or error.

KalmanMeth · 2024-01-21T12:11:31Z

Fixed perms in a023a01 and changed the instructions order to create namespace properly at first command.

@KalmanMeth note that if you are using oc command, ommiting labels section for pod-security is actually working !

Using kubectl, you must specify labels 🙃

@jpinsonneau Can you merge your commit with this PR?

jpinsonneau · 2024-01-22T09:08:35Z

@jpinsonneau Can you merge your commit with this PR?

It's already part of this PR, we can merge as is to me.
Thanks @KalmanMeth !

openshift-ci · 2024-01-22T09:08:52Z

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

KalmanMeth requested a review from jpinsonneau January 18, 2024 14:04

updated README and yml files

e9c4ad3

KalmanMeth force-pushed the fix-standalone-yaml branch from 2e42c0f to e9c4ad3 Compare January 18, 2024 14:10

fix perms

a023a01

jpinsonneau approved these changes Jan 19, 2024

View reviewed changes

openshift-ci bot assigned jpinsonneau Jan 19, 2024

openshift-ci bot added the lgtm label Jan 19, 2024

jpinsonneau added the approved label Jan 22, 2024

openshift-merge-bot bot merged commit e100947 into netobserv:main Jan 22, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updated README and yml files #249

updated README and yml files #249

KalmanMeth commented Jan 18, 2024

KalmanMeth commented Jan 18, 2024

codecov bot commented Jan 18, 2024 •

edited

Loading

jpinsonneau left a comment

jpinsonneau Jan 19, 2024

KalmanMeth Jan 21, 2024

jpinsonneau commented Jan 19, 2024

KalmanMeth commented Jan 21, 2024

KalmanMeth commented Jan 21, 2024

jpinsonneau commented Jan 22, 2024

openshift-ci bot commented Jan 22, 2024

updated README and yml files #249

updated README and yml files #249

Conversation

KalmanMeth commented Jan 18, 2024

Description

Dependencies

Checklist

KalmanMeth commented Jan 18, 2024

codecov bot commented Jan 18, 2024 • edited Loading

Codecov Report

jpinsonneau left a comment

Choose a reason for hiding this comment

jpinsonneau Jan 19, 2024

Choose a reason for hiding this comment

KalmanMeth Jan 21, 2024

Choose a reason for hiding this comment

jpinsonneau commented Jan 19, 2024

KalmanMeth commented Jan 21, 2024

KalmanMeth commented Jan 21, 2024

jpinsonneau commented Jan 22, 2024

openshift-ci bot commented Jan 22, 2024

codecov bot commented Jan 18, 2024 •

edited

Loading