Ginkgo2 #543

moleske · 2022-05-06T22:33:33Z

Are you submitting this PR against the develop branch?

All PR's to CATs should be submitted to develop and will be merged to main once they've passed acceptance.

What is this change about?

Describe the change and why it's needed.
bump to ginkgo2 to stay on supported versions

Please provide contextual information.

Include any links to other PRs, stories, slack discussions, etc... that will help establish context.
~~cloudfoundry/clock#31~~
~~cloudfoundry/cf-test-helpers#58~~
~~and a future pr to https://github.com/cloudfoundry/credhub-cli to get it to ginkgo2~~ (not actually necessary, just nice to have to remove all ginkgo v1 references)
are required before this can get further

What version of cf-deployment have you run this cf-acceptance-test change against?

Please check all that apply for this PR:

introduces a new test --- Are you sure everyone should be running this test?
changes an existing test
requires an update to a CATs integration-config

Did you update the README as appropriate for this change?

YES
N/A

If you are introducing a new acceptance test, what is your rationale for including it CATs rather than your own acceptance test suite?

CATs should validate common operator workflows.
CATs is not a regression test suite.
CATs is run by every component team to validate their releases before promotion.

How many more (or fewer) seconds of runtime will this change introduce to CATs?

🤷‍♀️

What is the level of urgency for publishing this change?

Urgent - unblocks current or future work
Slightly Less than Urgent

Tag your pair, your PM, and/or team!

It's helpful to tag a few other folks on your team or your team alias in case we need to follow up later.
just a random person bumping ginkgo2

cf-rel-int-status-bot · 2022-05-06T22:34:49Z

Hello friend, it looks like your pull request has failed one or more of our checks. Please take a look! 👀

cf-rel-int-status-bot · 2022-05-13T21:32:17Z

Hello friend, it looks like your pull request has failed one or more of our checks. Please take a look! 👀

cf-rel-int-status-bot · 2022-06-03T17:27:23Z

Hello friend, it looks like your pull request has failed one or more of our checks. Please take a look! 👀

moleske · 2022-06-03T23:26:55Z

Marking this ready not because I think it should be merged as is, but that it is in a state that could now have reasonable feedback. Tried running on a bosh lite, but I hit a timeout issue on test, so didn't finish the suite. There is some reporter stuff that I'd like some feed back on ~~and I think I didn't put back colors~~ (apparently I did put them back)

moleske · 2022-06-03T23:30:35Z

cats_suite_test.go

+	//rs := []Reporter{}
+	//
+	//if validationError == nil {
+	//	if Config.GetArtifactsDirectory() != "" {
+	//		helpers.EnableCFTrace(Config, "CATS")
+	//		rs = append(rs, helpers.NewJUnitReporter(Config, "CATS"))
+	//	}
+	//}
+	//
+	//reporterConfig := Config.GetReporterConfig()
+	//
+	//if reporterConfig.HoneyCombDataset != "" && reporterConfig.HoneyCombWriteKey != "" {
+	//	honeyCombClient := client.New(libhoney.Config{
+	//		WriteKey: reporterConfig.HoneyCombWriteKey,
+	//		Dataset:  reporterConfig.HoneyCombDataset,
+	//	})
+	//
+	//	globalTags := map[string]interface{}{
+	//		"run_id":  os.Getenv("RUN_ID"),
+	//		"env_api": Config.GetApiEndpoint(),
+	//	}
+	//
+	//	honeyCombReporter := honeycomb.New(honeyCombClient)
+	//	honeyCombReporter.SetGlobalTags(globalTags)
+	//	honeyCombReporter.SetCustomTags(reporterConfig.CustomTags)
+	//
+	//	rs = append(rs, honeyCombReporter)
+	//}


The honeyCombReporter I think needs to be rewritten to be properly used. The junit reporter seems to be a flag on the command line now based on https://onsi.github.io/ginkgo/MIGRATING_TO_V2#improved-reporting-infrastructure. Curious on folks thoughts about this

I'd previously made an issue about this: #528. I'm in favor of removing it, but just in case I've started a conversation in slack.

- removed some color stuff cause not available in ginkgo2 Authored-by: Michael Oleske <moleske@pivotal.io>

ginkgo2 Authored-by: Michael Oleske <moleske@pivotal.io>

- still broken, need cf-test-helpers updated first - might have future problem with honeycomb and junit reporters Authored-by: Michael Oleske <moleske@pivotal.io>

Authored-by: Michael Oleske <moleske@pivotal.io>

- the done async style shouldn't be used, so replaced with eventually timeout (probably done correctly 🤷‍♀️) Authored-by: Michael Oleske <moleske@pivotal.io>

- at least for now from bin/test, you need to specify a file location for it Authored-by: Michael Oleske <moleske@pivotal.io>

Authored-by: Michael Oleske <moleske@pivotal.io>

moleske · 2022-06-10T21:19:43Z

finally got around to running on a bosh lite. For what my bosh lite was configured to support, it was passing (120 passed, 0 failed, 3 pending, 105 skipped). So feeling pretty good about the ginkgo v2 minus the above about reporters

below is the configuration of what tests were included. I didn't have a windows cell set up to test anything there

  "include_apps": true,
  "include_capi_experimental": true,
  "include_detect": true,
  "include_security_groups": true,
  "include_services": true,
  "include_v3": true,
  "include_tasks": true,
  "include_backend_compatibility": false,
  "include_capi_no_bridge": false,
  "include_container_networking": false,
  "include_credhub" : false,
  "include_docker": false,
  "include_internet_dependent": false,
  "include_isolation_segments": false,
  "include_persistent_app": false,
  "include_private_docker_registry": false,
  "include_privileged_container_support": false,
  "include_route_services": false,
  "include_routing": false,
  "include_routing_isolation_segments": false,
  "include_service_discovery": false,
  "include_service_instance_sharing": false,
  "include_ssh": false,
  "include_sso": false,
  "include_zipkin": false

Authored-by: Michael Oleske <moleske@pivotal.io>

ctlong

There is an ask to keep the Honeycomb reporter, but I don't think that should stop up this PR. I do think we should add the JUnit and cf trace stuff back in though as that's pretty easy to do.

Once that's done I think this is good to go, thanks @moleske !

assets/credhub-service-broker/go.mod

moleske marked this pull request as ready for review June 3, 2022 23:25

moleske commented Jun 3, 2022

View reviewed changes

moleske added 9 commits June 3, 2022 16:45

First pass at bumping ginkgo

9260afb

- removed some color stuff cause not available in ginkgo2 Authored-by: Michael Oleske <moleske@pivotal.io>

remove reporters cause I don't like them and they're pretty dead in

a17fcc1

ginkgo2 Authored-by: Michael Oleske <moleske@pivotal.io>

put back report config

40ba3e0

- still broken, need cf-test-helpers updated first - might have future problem with honeycomb and junit reporters Authored-by: Michael Oleske <moleske@pivotal.io>

update clock to get ginkgo2 only in catnip

229be5c

Authored-by: Michael Oleske <moleske@pivotal.io>

bump cf-test-helpers to get ginkgo2

1a6f4bb

Authored-by: Michael Oleske <moleske@pivotal.io>

Remove unknown decorators

ea3f9a0

- the done async style shouldn't be used, so replaced with eventually timeout (probably done correctly 🤷‍♀️) Authored-by: Michael Oleske <moleske@pivotal.io>

Remove junit-reporter

bb1757d

- at least for now from bin/test, you need to specify a file location for it Authored-by: Michael Oleske <moleske@pivotal.io>

comment out reporters to see if everything else is ok

ffa8634

Authored-by: Michael Oleske <moleske@pivotal.io>

forgot go mod tidy and vendor

600e73f

Authored-by: Michael Oleske <moleske@pivotal.io>

ctlong mentioned this pull request Jun 6, 2022

Re-enable .NET core buildpack test #563

Merged

7 tasks

Fix missed deprecation

121ffc3

Authored-by: Michael Oleske <moleske@pivotal.io>

ctlong mentioned this pull request Jun 13, 2022

Feat: upgrade to ginkgo v2 cloudfoundry/cf-smoke-tests#75

Merged

4 tasks

ctlong mentioned this pull request Jun 24, 2022

Branch for cf21.3? #569

Closed

ctlong reviewed Jun 25, 2022

View reviewed changes

assets/credhub-service-broker/go.mod Show resolved Hide resolved

Add JUnitReporter and CFTrace back

2976960

ctlong merged commit e0cc782 into cloudfoundry:develop Jun 25, 2022

moleske deleted the ginkgo2 branch June 25, 2022 06:34

davewalter mentioned this pull request Jul 1, 2022

Remove the Ginkgo CLI cloudfoundry/cf-deployment-concourse-tasks#127

Merged

8 tasks

davewalter mentioned this pull request Jul 19, 2022

Update run-cats to use the new Ginkgo v2 CLI flags cloudfoundry/cf-deployment-concourse-tasks#129

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ginkgo2 #543

Ginkgo2 #543

moleske commented May 6, 2022 •

edited

Loading

cf-rel-int-status-bot commented May 6, 2022

cf-rel-int-status-bot commented May 13, 2022

cf-rel-int-status-bot commented Jun 3, 2022

moleske commented Jun 3, 2022 •

edited

Loading

moleske Jun 3, 2022

ctlong Jun 15, 2022

moleske commented Jun 10, 2022

ctlong left a comment

Ginkgo2 #543

Ginkgo2 #543

Conversation

moleske commented May 6, 2022 • edited Loading

Are you submitting this PR against the develop branch?

What is this change about?

Please provide contextual information.

What version of cf-deployment have you run this cf-acceptance-test change against?

Please check all that apply for this PR:

Did you update the README as appropriate for this change?

If you are introducing a new acceptance test, what is your rationale for including it CATs rather than your own acceptance test suite?

How many more (or fewer) seconds of runtime will this change introduce to CATs?

What is the level of urgency for publishing this change?

Tag your pair, your PM, and/or team!

cf-rel-int-status-bot commented May 6, 2022

cf-rel-int-status-bot commented May 13, 2022

cf-rel-int-status-bot commented Jun 3, 2022

moleske commented Jun 3, 2022 • edited Loading

moleske Jun 3, 2022

Choose a reason for hiding this comment

ctlong Jun 15, 2022

Choose a reason for hiding this comment

moleske commented Jun 10, 2022

ctlong left a comment

Choose a reason for hiding this comment

moleske commented May 6, 2022 •

edited

Loading

moleske commented Jun 3, 2022 •

edited

Loading