Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SupportBundle collection functionalities #140

Merged
merged 3 commits into from
Dec 22, 2022

Conversation

wsquan171
Copy link
Contributor

@wsquan171 wsquan171 commented Nov 24, 2022

This PR adds support bundle collection functionalities to Theia manager.

A new API endpoint apis/system.theia.antrea.io/v1alpha1/supportbundles is added to trigger / retrieve log bundle collection. The following logs will be collected as part of the operation:

  • Flow Aggregator logs (copied from /var/log/antrea/flow-aggregator/)
  • Theia Manager logs (local log files)
  • ClickHouse server logs (copied from /var/log/clickhouse-server/)
  • Grafana logs (copied from /var/log/grafana/)
  • Spark operator, driver and executor logs, if available during PR job (via streaming console logs)
  • zookeeper (via streaming console logs)

The following file structure is to be expected after successful collection:

vagrant@k8s-node-worker-1:~/logs$ tree
.
├── clickhouse-server
│   └── chi-clickhouse-clickhouse-0-0-0
│       ├── clickhouse-server.err.log
│       └── clickhouse-server.log
├── flow-aggregator
│   └── flow-aggregator-7f7cb66494-2w7dd
│       ├── flow-aggregator.flow-aggregator-7f7cb66494-2w7dd.root.log.INFO.20221123-203357.1
│       └── flow-aggregator.flow-aggregator-7f7cb66494-2w7dd.root.log.WARNING.20221123-210132.1
├── grafana
│   └── grafana-5b756cff8f-8ldb5
│       └── grafana.log
├── spark-driver
│   ├── pr-31011183-e862-4d0c-8d77-ab662312d278-driver
│   │   └── pr-31011183-e862-4d0c-8d77-ab662312d278-driver.log
│   └── pr-c36ce363-cc40-423f-b9ce-e39b8aa135de-driver
│       └── pr-c36ce363-cc40-423f-b9ce-e39b8aa135de-driver.log
├── spark-executor
│   └── pr-31011183-e862-4d0c-8d77-ab662312d278-fc3a7484a6b7564d-exec-1
│       └── pr-31011183-e862-4d0c-8d77-ab662312d278-fc3a7484a6b7564d-exec-1.log
├── spark-operator
│   └── policy-recommendation-spark-operator-84fb967df5-rs6mh
│       └── policy-recommendation-spark-operator-84fb967df5-rs6mh.log
├── theia-manager
│   ├── theia-manager.theia-manager-c54b85496-mzlzh.root.log.ERROR.20221123-214555.1
│   ├── theia-manager.theia-manager-c54b85496-mzlzh.root.log.INFO.20221123-213428.1
│   └── theia-manager.theia-manager-c54b85496-mzlzh.root.log.WARNING.20221123-214555.1
└── zookeeper
    └── zookeeper-0
        └── zookeeper-0.log

Logging configuration of ClickHouse and Grafana are also added to enable better configuration of log level and detention control.

To trigger log collection, or retrieve a log bundle job that finished earlier, simply run:
theia supportbundle

Theia CLI optional arg --since currently only supports filtering logs collected from Theia manager and Flow aggregator. Support of other components will be added in a future change.

Close #62

@wsquan171 wsquan171 force-pushed the supportbundle branch 6 times, most recently from 1616468 to cbaaa8e Compare November 30, 2022 12:09
@salv-orlando salv-orlando added this to the Theia v0.4 release milestone Nov 30, 2022
@wsquan171 wsquan171 force-pushed the supportbundle branch 2 times, most recently from fc0d758 to 574ce22 Compare November 30, 2022 19:50
@wsquan171
Copy link
Contributor Author

/theia-test-e2e

@wsquan171 wsquan171 force-pushed the supportbundle branch 2 times, most recently from 982bc6b to 6d31169 Compare December 7, 2022 23:24
Copy link
Contributor

@yuntanghsu yuntanghsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, some nits.

pkg/apiserver/registry/system/supportbundle/rest.go Outdated Show resolved Hide resolved
pkg/apiserver/registry/system/supportbundle/rest.go Outdated Show resolved Hide resolved
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Copy link
Contributor

@ziyouw ziyouw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, check this late due to the sick leave. LGTM, thanks!

@salv-orlando
Copy link
Contributor

@wsquan171 no additional comments from me. I think we can merge this PR.

@wsquan171 wsquan171 merged commit bdcaf0c into antrea-io:main Dec 22, 2022
@wsquan171 wsquan171 deleted the supportbundle branch December 22, 2022 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support bundle collection for Flow Visibility Components
4 participants