NETOBSERV-346 revamp upstream doc / readme #125

jotak · 2022-06-21T10:39:13Z

No description provided.

openshift-ci · 2022-06-21T10:39:17Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2022-06-21T10:39:20Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from jotak after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jotak · 2022-06-21T10:40:19Z

WIP:

README.md first pass almost finished (until I get feedback) => check it here: https://github.com/netobserv/network-observability-operator/blob/a92495c3e42e1f66dd39d3304f6b9c97776d3518/README.md
new file DEVELOPMENT.md still to do

jotak · 2022-06-21T10:42:32Z

README.md

+
+## Understanding the deployed components
+
+TODO


Hey @netobserv/maintainers
does anybody have a full diagram depicting the architecture, including the epbf & kafka options? It would be nice to include here

jira created to do it later: https://issues.redhat.com/browse/NETOBSERV-412

jotak · 2022-06-21T10:43:10Z

README.md

+#### To run the eBPF agent
+
+What matters is the version of the Linux kernel.
+TODO: precisions


@mariomac do you know which minimal version is allowed here?

We stated Kernel 4.18 or higher because it is supported by RHEL 8. Lower versions could still work, however.

jotak · 2022-06-21T10:44:05Z

README.md

+
+#### To use IPFIX exports
+
+OpenShift 4.10 or above, or upstream OVN-Kubernetes [TODO: upstream version?] are recommended, as the operator will configure OVS for you. Else, you need to configure it manually.


@mariomac I didn't find how upstream ovn-k is versionned, do you know? their github doesn't tell much about it (or I missed it)

No, I'm sorry I don't know.

OlivierCazade

LGTM, my comment is just a suggestion

OlivierCazade · 2022-06-21T13:24:32Z

README.md


-[![Docker Repository on Quay](https://quay.io/repository/netobserv/network-observability-operator/status "Docker Repository on Quay")](https://quay.io/repository/netobserv/network-observability-operator)
+A Kubernetes / OpenShift operator for network observability. It deploys a flow monitoring pipeline based on an [eBPF agent](https://github.com/netobserv/netobserv-ebpf-agent/) or [IPFIX](https://en.wikipedia.org/wiki/IP_Flow_Information_Export) exports. It provides dashboards, metrics, and keeps flows accessible in a queryable log store: [Grafana Loki](https://grafana.com/oss/loki/). When used in OpenShift, new dashboards are available in the Console.



How about adding a screenshot here to show what the NOO can do?
May be we can reuse the topology screenshot?

I was also thinking about showing a top banner, like on operatorhub: https://operatorhub.io/operator/netobserv-operator/
but actually, I don't know, we don't want too many screenshots either, as it takes a lot of space and forces to scroll more...

There's now 6 screenshots in this version (compared to 3 before, and they were buried at the bottom of the page), I would have said it's enough (?)

Ok, I understand your point of not having too many screenshots.

if you want to put a lot of screens, what about making a dedicated page for that ? and we keep only top 3 items in the main page.

openshift-ci · 2022-06-21T13:54:50Z

New changes are detected. LGTM label has been removed.

stleerh · 2022-06-21T16:55:47Z

.mk/sample.mk

+	kubectl apply -f ./config/samples/flows_v1alpha1_flowcollector.yaml || true
+else
+	kubectl apply -f ./config/samples/flows_v1alpha1_flowcollector_versioned.yaml || true
+endif


Remove "|| true". If this fails, do you really want the next potential target to continue?

This is what @eranra wrote, on purpose I think .. I'm not against discussing that, but it doesn't really relate to that PR

To bring some context: I did did small change in the makefile because, while writing the doc, I figured out that deploying a "versionned" sample CR wouldn't work as expected, especially as the underlying component images are not tied anymore with the operator version. So now, I recommend switching to the version tag in git before deploying / applying CR

Okay, that change is fine. We can ignore my comment.

DEVELOPMENT.md

README.md

stleerh · 2022-06-21T18:52:44Z

README.md

+
+- Agent (`spec.agent`) can be `ipfix` or `ebpf`. As mentioned above, the IPFIX option is fully functional when using OVN-Kubernetes CNI (other CNIs are not supported, but you may still be able to configure them manually if they allow IPFIX exports) whereas eBPF is expected to work regardless the running CNI.
+
+- Sampling (`spec.ipfix.sampling` and `spec.ebpf.sampling`): 24/7 unsampled flow collection may consume a non-negligible amount of resources. While we are doing our best to make it a viable option in production, it is still often necessary to mitigate by setting a sampling ratio. A value of `100` means: one flow every 100 is sampled. `1` means no sampling. The lower it is, the more accurate are flows and derived metrics.


Specify the sampling defaults for ipfix and ebpf.

+1
@mariomac , looking for default sampling in eBPF, I see it is disabled. Is it on purpose? Shouldn't we set a bigger value, maybe 100, at least until the epbf perf enhancement epic is completed?
Second question: a default of 0 is explicitely set in the sample CR (which is used by OLM) but not via kubebuilder / CRD. I guess that means it defaults to golang's 0-value, hence 0 too ? But we should set an explicit value if we want to enable sampling by default

Yes, it was on purpose. Even if it can be vastly improved, we considered the performance as "good enough". I'd keep it as disabled, considering that eBPF right now is not the default agent.

Second question: yes, we assumed it would default to 0. We can set it explicitly to 0 or 1 if you prefer to make it clearer. Or to another value if you consider that we should enable sampling by default.

Fair enough, let's keep it that way :)

README.md

jotak · 2022-06-22T09:23:52Z

Thank you @stleerh 🙏
I've integrated most of your suggestions; when I didn't, I replied here.

jotak · 2022-06-22T09:27:51Z

/retest

stleerh · 2022-06-22T16:56:28Z

README.md

-_Pod traffic_
+This project is licensed under [Apache 2.0](./LICENSE) and accepts contributions via GitHub pull requests. Other related `netobserv` projects follow the same rules:
+- [Flowlogs-pipeline](https://github.com/netobserv/flowlogs-pipeline)
+- The [eBPF agent](https://github.com/netobserv/netobserv-ebpf-agent)


Remove 'The' here too.

jotak · 2022-06-23T08:53:09Z

/approved

jotak · 2022-06-23T08:54:20Z

@stleerh I'm going to merge it. We can continue discussion & refine before we release final v0.1.3

add filter transform

NETOBSERV-346 revamp upstream doc / readme

8a42d8e

openshift-ci bot added the do-not-merge/work-in-progress label Jun 21, 2022

jotak commented Jun 21, 2022

View reviewed changes

update development guide

05b3073

jotak requested review from jpinsonneau, mariomac and OlivierCazade June 21, 2022 12:46

mariomac approved these changes Jun 21, 2022

View reviewed changes

openshift-ci bot assigned mariomac Jun 21, 2022

openshift-ci bot added the lgtm label Jun 21, 2022

OlivierCazade approved these changes Jun 21, 2022

View reviewed changes

openshift-ci bot assigned OlivierCazade Jun 21, 2022

older versions deployment

9ea3ab5

openshift-ci bot removed the lgtm label Jun 21, 2022

typos

a92495c

jotak requested review from eranra and stleerh June 21, 2022 14:43

Fix todo / remove incomplete sections

e973974

jotak marked this pull request as ready for review June 21, 2022 16:27

openshift-ci bot removed the do-not-merge/work-in-progress label Jun 21, 2022

stleerh reviewed Jun 21, 2022

View reviewed changes

jotak added this to the v0.1.3 milestone Jun 22, 2022

Doc: integrate steven's comments

4334aad

stleerh reviewed Jun 22, 2022

View reviewed changes

Rename Views -> Pages and tabs

2e06e38

jotak merged commit fd4b77d into netobserv:main Jun 23, 2022

jotak deleted the revamp-readme branch December 7, 2022 04:03

KalmanMeth pushed a commit to KalmanMeth/network-observability-operator that referenced this pull request Feb 13, 2023

Merge pull request netobserv#125 from eranra/add_remove_fields_65

8ba43eb

add filter transform

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-346 revamp upstream doc / readme #125

NETOBSERV-346 revamp upstream doc / readme #125

jotak commented Jun 21, 2022

openshift-ci bot commented Jun 21, 2022

openshift-ci bot commented Jun 21, 2022

jotak commented Jun 21, 2022 •

edited

Loading

jotak Jun 21, 2022

jotak Jun 21, 2022

jotak Jun 21, 2022

mariomac Jun 21, 2022

jotak Jun 21, 2022

mariomac Jun 21, 2022

OlivierCazade left a comment

OlivierCazade Jun 21, 2022

jotak Jun 21, 2022

OlivierCazade Jun 22, 2022

jpinsonneau Jun 22, 2022

openshift-ci bot commented Jun 21, 2022

stleerh Jun 21, 2022

jotak Jun 22, 2022

stleerh Jun 22, 2022

stleerh Jun 21, 2022

jotak Jun 22, 2022 •

edited

Loading

mariomac Jun 23, 2022

jotak Jun 23, 2022

jotak commented Jun 22, 2022

jotak commented Jun 22, 2022

stleerh Jun 22, 2022

jotak commented Jun 23, 2022

jotak commented Jun 23, 2022


		#### To use IPFIX exports

		OpenShift 4.10 or above, or upstream OVN-Kubernetes [TODO: upstream version?] are recommended, as the operator will configure OVS for you. Else, you need to configure it manually.


		[![Docker Repository on Quay](https://quay.io/repository/netobserv/network-observability-operator/status "Docker Repository on Quay")](https://quay.io/repository/netobserv/network-observability-operator)
		A Kubernetes / OpenShift operator for network observability. It deploys a flow monitoring pipeline based on an [eBPF agent](https://github.com/netobserv/netobserv-ebpf-agent/) or [IPFIX](https://en.wikipedia.org/wiki/IP_Flow_Information_Export) exports. It provides dashboards, metrics, and keeps flows accessible in a queryable log store: [Grafana Loki](https://grafana.com/oss/loki/). When used in OpenShift, new dashboards are available in the Console.


		- Agent (`spec.agent`) can be `ipfix` or `ebpf`. As mentioned above, the IPFIX option is fully functional when using OVN-Kubernetes CNI (other CNIs are not supported, but you may still be able to configure them manually if they allow IPFIX exports) whereas eBPF is expected to work regardless the running CNI.

		- Sampling (`spec.ipfix.sampling` and `spec.ebpf.sampling`): 24/7 unsampled flow collection may consume a non-negligible amount of resources. While we are doing our best to make it a viable option in production, it is still often necessary to mitigate by setting a sampling ratio. A value of `100` means: one flow every 100 is sampled. `1` means no sampling. The lower it is, the more accurate are flows and derived metrics.

NETOBSERV-346 revamp upstream doc / readme #125

NETOBSERV-346 revamp upstream doc / readme #125

Conversation

jotak commented Jun 21, 2022

openshift-ci bot commented Jun 21, 2022

openshift-ci bot commented Jun 21, 2022

jotak commented Jun 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OlivierCazade left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Jun 21, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jotak Jun 22, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jotak commented Jun 22, 2022

jotak commented Jun 22, 2022

Choose a reason for hiding this comment

jotak commented Jun 23, 2022

jotak commented Jun 23, 2022

jotak commented Jun 21, 2022 •

edited

Loading

jotak Jun 22, 2022 •

edited

Loading