Enhancements in confgen's dashboards generator

[jotak]

@OlivierCazade did not use the FLP dashboards generator in netobserv/network-observability-operator#299 because he found some limitations. It would be nice to see if these limitations could be addressed. Quoting him:

• Not possible to use rows definitions to have multiple foldable subsections
• Not possible to have multiple query on the same graph. Here I wanted to have ingested flows and sent to loki flows in the same graph
• Not possible to manage graph width so we can have mulitple graphs on the same row. Here for CPU and memory utilization.

cc @KalmanMeth @eranra , thoughts?

[KalmanMeth]

@jotak If we want more control over the grafana dashboards, then we probably should not be generating them completely automatically and uploading them directly. We should probably store the generated dashboards in a file, and then edit the file with the adjustments we want to make, and load the files into the operator. The drawback of this approach is that we need to have additional config files that are uploaded when bringing up the operator. We could perhaps embed them in the code, like we did with the metrics definitions files.

[jotak]

@KalmanMeth embedding a manually generated json is what @OlivierCazade did in his PR, however I think maybe it could be interesting to support on confgen these features?
It would certainly require to update the document design, especially by uncoupling the metric and its vizualisation. E.g. maybe something like:

• metric 1 in its own file (without the visualization part)
• metric 2 in its own file (without the visualization part)
• visualization file such as:

visualization:
  type: grafana
  grafana:
    - rowTitle: "Top 5 node traffic"
      - title: "Top 5 Flows rate egress per source and destination nodes"
        expr:
        - 'topk(5,rate(netobserv_node_egress_bytes_total[1m]))'
        legendFormat: '{{pod}} : ({{SrcK8S_HostName}}) -> ({{DstK8S_HostName}})'
        type: graphPanel
        dashboard: netobserv
      - title: "Top 5 Flows rate ingress per source and destination nodes"
        expr:
        - 'topk(5,rate(netobserv_node_ingress_bytes_total[1m]))'
        legendFormat: '{{pod}} : ({{SrcK8S_HostName}}) -> ({{DstK8S_HostName}})'
        type: graphPanel
        dashboard: netobserv

Note here, there can be several chart per row, and several expressions per chart, to comply with what Olivier did.