Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test(s): TestAccContainerCluster_failedCreation #13711

Comments

@melinath
Copy link
Collaborator

melinath commented Feb 10, 2023

Failure rates

  • 100% since 2023-02-04

Impacted tests:

  • TestAccContainerCluster_failedCreation

Affected Resource(s)

  • google_container_cluster

Nightly builds

Message(s)

bootstrap_utils_test.go:490: Error setting billing account for project "tf-fail-cluster-test" to "*******": googleapi: Error 403: The caller does not have permission, forbidden

b/299601076

@roaks3
Copy link
Collaborator

roaks3 commented Feb 13, 2023

Note: I'm fairly certain this was a result of using the new billing accounts when running this test on the new GA project. We will need a solution for handling the project space as part of that effort, but for now I've updated the billing account to the old one and hoping it will pass again.

@rileykarson rileykarson added this to the Near-Term Goals milestone Feb 13, 2023
@shuyama1
Copy link
Collaborator

Update 02/17: Test still failed in GA due to a different error, while test passed in beta after 02/13:

bootstrap_utils_test.go:499: Error listing services for project "tf-fail-cluster-test": Failed to list enabled services for project tf-fail-cluster-test: googleapi: Error 403: Project 'tf-fail-cluster-test' not found or permission denied.
Help Token: AVUsNs0Tl_DcVYjbtNaqtyie6DYCeuXungrlNLROjzZuEtDCmvvYEUPqoA6ePDxNkFZ8c-KTgyanZMHG7mHGmeLWkhcmXX09tEkbkfVANNY70eLJ
Details:
[
{
"@type": "type.googleapis.com/google.rpc.PreconditionFailure",
"violations": [
{
"subject": "?error_code=210002\u0026type=Project\u0026resource_id=tf-fail-cluster-test",
"type": "googleapis.com"
}
]
},
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"domain": "serviceusage.googleapis.com",
"metadata": {
"resource_id": "tf-fail-cluster-test",
"type": "Project"
},
"reason": "RESOURCES_NOT_FOUND"
}
]

@trodge
Copy link
Collaborator

trodge commented Feb 17, 2023

I'm not sure why we're getting a 403 listing the services in tf-fail-cluster-test, I've already granted hashicorp-test-runner@ci-test-project-188019.iam.gserviceaccount.com the owner role in that project. Is it possible the request to list services is coming from a different account?

I also tried adding the service usage admin role to the hashicorp test runner just now, but I'm not sure if that will help.

@trodge
Copy link
Collaborator

trodge commented Mar 21, 2023

This test is still failing and I'm currently investigating why.

@trodge trodge reopened this Mar 21, 2023
@trodge trodge self-assigned this Mar 21, 2023
@roaks3 roaks3 added the test-failure-100 100% fail rate label Apr 4, 2023
@trodge
Copy link
Collaborator

trodge commented Apr 19, 2023

It looks like the test fails because cluster creation is no longer failing with the permission changes we make, which means the cluster can't be deleted because it is still being created.

@github-actions github-actions bot added service/container forward/review In review; remove label to forward labels Aug 17, 2023
@melinath melinath removed the forward/review In review; remove label to forward label Sep 7, 2023
@melinath
Copy link
Collaborator Author

melinath commented Sep 7, 2023

If the cluster creation can no longer fail in this way we should potentially remove this test. If it can fail like this, we should update the test to trigger the failure correctly.

Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.