Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry for internal 160009 errors #14727

Conversation

modular-magician
Copy link
Collaborator

Fixes #14691

This does a bit of refactoring- I did that to wrap my head around the existing logic, and hopefully make it more clear. Sorry for the extra noise!

I've confirmed our existing tests pass (TestAccProjectService_, TestAccProject_deleteDefaultNetwork), but was unable to reproduce the error so far to confirm the fix works.

Our logs show some escape characters in the sequence (the relevant bit is "[service pubsub.googleapis.com encountered internal error: type: \"googleapis.com\" subject: \"160009\" ] with failed services [pubsub.googleapis.com]",) so I looked for a few different parts of the string in the body rather than one long string- without a repro, I'm not 100% certain what the correct characters would be.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
  • Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

serviceusage: added retries to handle internal error: type: "googleapis.com" subject: "160009" when activating services
cloudresourcemanager: added retries to handle internal error: type: "googleapis.com" subject: "160009" when activating "compute.googleapis.com" to destroy the default network when `auto_create_network` is `false`

Derived from GoogleCloudPlatform/magic-modules#8017

Signed-off-by: Modular Magician <magic-modules@google.com>
@modular-magician modular-magician merged commit b220eab into hashicorp:main May 25, 2023
ScottSuarez pushed a commit that referenced this pull request May 25, 2023
Signed-off-by: Modular Magician <magic-modules@google.com>
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 25, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

google_project_service: 160009 issues when activating services
1 participant