🌱 Rewrite fetch_target_logs.sh in Golang #1763

peppi-lotta · 2024-06-13T10:02:57Z

What this PR does / why we need it: Running an sh file from go file will gives warning: G204: Audit use of command execution. In this PR I've rewritten the fetch_target_logs.sh in Go to avoid getting this warning.

Compared to the original: I've added kubect get pods -A
and removed the split between stdout and stderr in the logs. Some of the logs had warning

W0613 11:04:05.645729       1 options.go:215] 
==== Removed Flag Warning ======================

logtostderr is removed in the k8s upstream and has no effect any more.

and based on further investigation it seems Kubernetes only has one API endpoint for logs and everything is logged into stdout.

metal3-io-bot · 2024-06-13T10:03:02Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mboukhalfa for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tuminoid

Fix the build, and some comments on top.

tuminoid · 2024-06-13T10:21:29Z

test/go.mod

@@ -18,6 +18,7 @@ require (
 	k8s.io/apimachinery v0.29.5
 	k8s.io/client-go v0.29.5
 	k8s.io/klog/v2 v2.110.1
+	k8s.io/kubectl v0.29.3


Match the version with other k8s.io deps, ie. v0.29.5.

tuminoid · 2024-06-13T10:23:06Z

test/e2e/fetch_target_logs.go

+				}
+				defer podLogs.Close()
+
+				// Read the logs into a string


This only reads stdout. Original script reads also stderr.

tuminoid · 2024-06-13T10:23:28Z

test/e2e/fetch_target_logs.go

+			// kubectl --kubeconfig="${KUBECONFIG_WORKLOAD}" describe pods -n "${NAMESPACE}" "${POD}"
+			describerSettings := describe.DescriberSettings{
+				ShowEvents: true,
+


tuminoid · 2024-06-13T10:26:14Z

test/e2e/fetch_target_logs.go

+	if err != nil {
+		log.Fatal(err)
+	}
+


Would you add here also equivalent of kubectl --kubeconfig .... get pods -A before namespace iteration, we've been missing that.

Follow-up PR is also OK, but since you want to get this merged before vacations, adding it here makes sense.

peppi-lotta · 2024-06-14T15:01:21Z

/test metal3-centos-e2e-integration-test-main
/test metal3-ubuntu-e2e-integration-test-main

peppi-lotta · 2024-06-14T15:01:28Z

/test ?

metal3-io-bot · 2024-06-14T15:01:32Z

@peppi-lotta: The following commands are available to trigger required jobs:

/test build
/test generate
/test gomod
/test manifestlint
/test markdownlint
/test metal3-centos-e2e-integration-test-main
/test metal3-ubuntu-e2e-integration-test-main
/test shellcheck
/test test
/test unit

The following commands are available to trigger optional jobs:

/test metal3-centos-e2e-basic-test-main
/test metal3-centos-e2e-feature-test-main
/test metal3-e2e-1-26-1-27-upgrade-test-main
/test metal3-e2e-1-27-1-28-upgrade-test-main
/test metal3-e2e-1-28-1-29-upgrade-test-main
/test metal3-e2e-clusterctl-upgrade-test-main
/test metal3-ubuntu-e2e-basic-test-main
/test metal3-ubuntu-e2e-feature-test-main

Use /test all to run the following jobs that were automatically triggered:

build
generate
gomod
manifestlint
shellcheck
unit

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

adilGhaffarDev · 2024-06-20T12:43:47Z

test/e2e/fetch_target_logs.go

+	"k8s.io/kubectl/pkg/describe"
+)
+
+func FetchTargetLogs() {


Suggested change

func FetchTargetLogs() {

func FetchLogs(clusterProxy framework.ClusterProxy) {

We can pass clusterProxy and make this function reusable. If in the future we want to collect logs from some other cluster it will be very helpful.

adilGhaffarDev · 2024-07-08T07:41:19Z

Let's also implement CAPI CollectInfrastructureLogs on our side in this PR.

peppi-lotta · 2024-08-08T11:21:15Z

/test ?

metal3-io-bot · 2024-08-08T11:21:20Z

@peppi-lotta: The following commands are available to trigger required jobs:

/test build
/test generate
/test gomod
/test manifestlint
/test markdownlint
/test metal3-centos-e2e-integration-test-main
/test metal3-ubuntu-e2e-integration-test-main
/test shellcheck
/test test
/test unit

The following commands are available to trigger optional jobs:

/test metal3-centos-e2e-basic-test-main
/test metal3-centos-e2e-feature-test-main-features
/test metal3-centos-e2e-feature-test-main-pivoting
/test metal3-centos-e2e-feature-test-main-remediation
/test metal3-e2e-1-26-1-27-upgrade-test-main
/test metal3-e2e-1-27-1-28-upgrade-test-main
/test metal3-e2e-1-28-1-29-upgrade-test-main
/test metal3-e2e-clusterctl-upgrade-test-main
/test metal3-ubuntu-e2e-basic-test-main
/test metal3-ubuntu-e2e-feature-test-main-features
/test metal3-ubuntu-e2e-feature-test-main-pivoting
/test metal3-ubuntu-e2e-feature-test-main-remediation

Use /test all to run the following jobs that were automatically triggered:

build
generate
gomod
manifestlint
shellcheck
unit

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

peppi-lotta · 2024-08-08T11:23:36Z

/test metal3-ubuntu-e2e-feature-test-main-features

peppi-lotta · 2024-08-09T10:28:33Z

/test metal3-ubuntu-e2e-feature-test-main-pivoting

After running this test there should be a k8_target_cluster directory in the logs. If it looks as it should (same as before except for a few minor changes) this PR's code is working correctly.

Signed-off-by: peppi-lotta <peppi-lotta.saari@est.tech>

peppi-lotta · 2024-08-12T08:18:38Z

/test metal3-ubuntu-e2e-feature-test-main-pivoting

metal3-io-bot · 2024-08-12T09:26:56Z

@peppi-lotta: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
metal3-ubuntu-e2e-feature-test-main-pivoting	`af9d0a8`	link	false	`/test metal3-ubuntu-e2e-feature-test-main-pivoting`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

peppi-lotta · 2024-08-12T10:18:38Z

/test metal3-centos-e2e-integration-test-main
/test metal3-ubuntu-e2e-integration-test-main

peppi-lotta · 2024-08-12T10:22:37Z

metal3-ubuntu-e2e-feature-test-main-pivoting test is failing but it doesn't seem to me that it is related to the changes in this PR. Correct me if I'm wrong.

In the logs-jenkins-metal3-ubuntu-e2e-feature-test-main-pivoting-4-pivoting-based.tgz the logs in directory k8s_target_cluster are looking as they should. So to me it seems everything is working as it should.

Please take a look @adilGhaffarDev @mboukhalfa @Sunnatillo. This needs a lgtm and approve. Thank you!

adilGhaffarDev · 2024-08-12T10:29:00Z

/test metal3-e2e-clusterctl-upgrade-test-main

adilGhaffarDev · 2024-08-12T10:55:34Z

In the logs-jenkins-metal3-ubuntu-e2e-feature-test-main-pivoting-4-pivoting-based.tgz the logs in directory k8s_target_cluster are looking as they should. So to me it seems everything is working as it should.

It looks good. I want to see how this will work when we have multiple management clusters, like how we have in clusterctl tests.
Also if possible let's try to reuse CAPI's implementation of same functionality: https://github.com/kubernetes-sigs/cluster-api/blob/8f31a865b6cf562f8fe831ae603b6e27752d8525/test/framework/deployment_helpers.go#L152 , if its too much work lets merge this PR and do it in another PR.

metal3-io-bot · 2024-08-23T00:49:23Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

metal3-io-bot requested review from mboukhalfa and Sunnatillo June 13, 2024 10:03

metal3-io-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 13, 2024

peppi-lotta changed the title ~~🌱 Rewrite fetch_target_logs.sh in Golang~~ 🌱 WIP: Rewrite fetch_target_logs.sh in Golang Jun 13, 2024

tuminoid reviewed Jun 13, 2024

View reviewed changes

peppi-lotta force-pushed the peppi-lotta/fetch_target_logs.sh_to_golang branch 9 times, most recently from 1f60260 to bb6e1ab Compare June 14, 2024 13:44

peppi-lotta changed the title ~~🌱 WIP: Rewrite fetch_target_logs.sh in Golang~~ 🌱 Rewrite fetch_target_logs.sh in Golang Jun 14, 2024

metal3-io-bot added the needs-rebase Indicates that a PR cannot be merged because it has merge conflicts with HEAD. label Jun 15, 2024

adilGhaffarDev reviewed Jun 20, 2024

View reviewed changes

Rozzii added this to the 1.8.0 milestone Jun 28, 2024

peppi-lotta force-pushed the peppi-lotta/fetch_target_logs.sh_to_golang branch from bb6e1ab to 508874b Compare August 8, 2024 10:16

metal3-io-bot removed the needs-rebase Indicates that a PR cannot be merged because it has merge conflicts with HEAD. label Aug 8, 2024

peppi-lotta force-pushed the peppi-lotta/fetch_target_logs.sh_to_golang branch 3 times, most recently from b0362fc to 17bd439 Compare August 8, 2024 11:18

peppi-lotta force-pushed the peppi-lotta/fetch_target_logs.sh_to_golang branch from 17bd439 to 944ae12 Compare August 9, 2024 10:23

Rewrite fetch_target_logs.sh in Golang

af9d0a8

Signed-off-by: peppi-lotta <peppi-lotta.saari@est.tech>

peppi-lotta force-pushed the peppi-lotta/fetch_target_logs.sh_to_golang branch from 944ae12 to af9d0a8 Compare August 12, 2024 08:16

metal3-io-bot added the needs-rebase Indicates that a PR cannot be merged because it has merge conflicts with HEAD. label Aug 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🌱 Rewrite fetch_target_logs.sh in Golang #1763

🌱 Rewrite fetch_target_logs.sh in Golang #1763

peppi-lotta commented Jun 13, 2024 •

edited

Loading

metal3-io-bot commented Jun 13, 2024

tuminoid left a comment

tuminoid Jun 13, 2024

tuminoid Jun 13, 2024

tuminoid Jun 13, 2024

tuminoid Jun 13, 2024

peppi-lotta commented Jun 14, 2024

peppi-lotta commented Jun 14, 2024

metal3-io-bot commented Jun 14, 2024

adilGhaffarDev Jun 20, 2024

adilGhaffarDev commented Jul 8, 2024

peppi-lotta commented Aug 8, 2024

metal3-io-bot commented Aug 8, 2024

peppi-lotta commented Aug 8, 2024

peppi-lotta commented Aug 9, 2024

peppi-lotta commented Aug 12, 2024

metal3-io-bot commented Aug 12, 2024 •

edited

Loading

peppi-lotta commented Aug 12, 2024

peppi-lotta commented Aug 12, 2024

adilGhaffarDev commented Aug 12, 2024

adilGhaffarDev commented Aug 12, 2024

metal3-io-bot commented Aug 23, 2024

	func FetchTargetLogs() {
	func FetchLogs(clusterProxy framework.ClusterProxy) {

🌱 Rewrite fetch_target_logs.sh in Golang #1763

Are you sure you want to change the base?

🌱 Rewrite fetch_target_logs.sh in Golang #1763

Conversation

peppi-lotta commented Jun 13, 2024 • edited Loading

metal3-io-bot commented Jun 13, 2024

tuminoid left a comment

Choose a reason for hiding this comment

tuminoid Jun 13, 2024

Choose a reason for hiding this comment

tuminoid Jun 13, 2024

Choose a reason for hiding this comment

tuminoid Jun 13, 2024

Choose a reason for hiding this comment

tuminoid Jun 13, 2024

Choose a reason for hiding this comment

peppi-lotta commented Jun 14, 2024

peppi-lotta commented Jun 14, 2024

metal3-io-bot commented Jun 14, 2024

adilGhaffarDev Jun 20, 2024

Choose a reason for hiding this comment

adilGhaffarDev commented Jul 8, 2024

peppi-lotta commented Aug 8, 2024

metal3-io-bot commented Aug 8, 2024

peppi-lotta commented Aug 8, 2024

peppi-lotta commented Aug 9, 2024

peppi-lotta commented Aug 12, 2024

metal3-io-bot commented Aug 12, 2024 • edited Loading

peppi-lotta commented Aug 12, 2024

peppi-lotta commented Aug 12, 2024

adilGhaffarDev commented Aug 12, 2024

adilGhaffarDev commented Aug 12, 2024

metal3-io-bot commented Aug 23, 2024

peppi-lotta commented Jun 13, 2024 •

edited

Loading

metal3-io-bot commented Aug 12, 2024 •

edited

Loading