404 when accessing debug logs posted onto PRs

[SarahFrench]

Background

The link to the debug logs for the passing test in the screenshot above/this PR comment has a 404 error.

What kind of contribution is this issue about?

general contributions

Related PR(s), if any:

• Add google_cloud_identity_group_lookup data source GoogleCloudPlatform/magic-modules#9163 (comment)

Details

The TestAccCloudIdentityGroup test loops through multiple tests and is structured differently to typical acceptance tests, so this might mean the URL of the log file is formed differently than normal?

terraform-provider-google/google/services/cloudidentity/resource_cloud_identity_group_test.go

Lines 18 to 46 in 52d037c

	// Intended to fix https://github.com/hashicorp/terraform-provider-google/issues/10001
	// These are all of the tests that use a cloud_identity_group, except for
	// testAccAccessContextManagerGcpUserAccessBinding_basicTest. The theory is that they sometimes
	// fail with a 409 because of concurrent roster mutations, so running them serially should prevent
	// the error.
	func TestAccCloudIdentityGroup(t *testing.T) {
	testCases := map[string]func(t *testing.T){
	"basic": testAccCloudIdentityGroup_cloudIdentityGroupsBasicExampleTest,
	"update": testAccCloudIdentityGroup_updateTest,
	"membership_basic": testAccCloudIdentityGroupMembership_cloudIdentityGroupMembershipExampleTest,
	"membership_update": testAccCloudIdentityGroupMembership_updateTest,
	"membership_import": testAccCloudIdentityGroupMembership_importTest,
	"membership_dne": testAccCloudIdentityGroupMembership_membershipDoesNotExistTest,
	"membership_user": testAccCloudIdentityGroupMembership_cloudIdentityGroupMembershipUserExampleTest,
	"data_source_basic": testAccDataSourceCloudIdentityGroups_basicTest,
	"data_source_membership_basic": testAccDataSourceCloudIdentityGroupMemberships_basicTest,
	}
	for name, tc := range testCases {
	// shadow the tc variable into scope so that when
	// the loop continues, if t.Run hasn't executed tc(t)
	// yet, we don't have a race condition
	// see https://github.com/golang/go/wiki/CommonMistakes#using-goroutines-on-loop-iterator-variables
	tc := tc
	t.Run(name, func(t *testing.T) {
	tc(t)
	})
	}
	}

[rileykarson]

You can access those logs directly through the GCS bucket from the bottom link! You've got it, the logs are split out by subtest and we're just reporting on the main test. TC has a similar output, actually- the only test in its eyes is the top-level one and the artifacts are subtest-specific.

Given that most of these are whole packages that need to run in serial, I wonder if we could stop using subtests by just omitting t.Parallel since I think that applies at the package level. We'd likely want a way to catch inadvertent additions of parallel tests to those packages, though, since reviewers of smaller PRs would easily miss deviations.

[SarahFrench]

I'll close this issue and open an issue about refactoring the test, thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.