Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GCP-8479] Bigtable: 'test_bigtable_list_tables' snippet flakes with '504 Deadline Exceeded'. #15

Closed
wants to merge 2 commits into from

Conversation

sangramql
Copy link

@sangramql sangramql commented Jul 30, 2019

Addresses #8479

Preliminary discussion can be found here

@sangramql
Copy link
Author

sangramql commented Jul 30, 2019

@mf2199 you have already reviewed, but for further updates plz review.
Opened PR on qlogic repository.

Copy link
Collaborator

@mf2199 mf2199 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks fine but the questions are whether the flakes were reproduced, how positive are we about this way of solving them, and if enough testing has been done afterwards to ensure they're gone.

instance = client.instance(INSTANCE_ID)
table = instance.table("to_list")
max_versions_rule = column_family.MaxVersionsGCRule(2)
table.create(column_families={"cf1": max_versions_rule})

# [START bigtable_list_tables]
from google.cloud.bigtable import Client
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, not sure why this duplicate import was there before, in the first place.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, not sure why this duplicate import was there before, in the first place.

These are snippets which are included in doc string of actual functions, so for example codes we need to have imports for users to understand.

cluster = instance.cluster(
"clus-my-123",
"clus-to-delete" + UNIQUE_SUFFIX,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nit-picking, but I'd use somewhat more informative names, such as "snippet-instance", "cluster-to-delete" etc.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This instance is created for only 'instance delete' test, hence the names.
Already tried naming that, but got string length error.

from_value = <_Rendezvous of RPC that terminated with: status = StatusCode.INVALID_ARGUMENT details = "Error in field 'clusters' ...lection clusters : Length should be between [6,30], but found 31 'cluster-to-delete-1564545637619'","grpc_status":3}" ???
E google.api_core.exceptions.InvalidArgument: 400 Error in field 'clusters' : Cluster Id : Invalid id for collection clusters : Length should be between [6,30], but found 31 'cluster-to-delete-1564545637619'

@mf2199 mf2199 requested review from emar-kar and sumit-ql July 31, 2019 01:53
@mf2199 mf2199 changed the title Bigtable: Create table in list_table test, deadline exceed error fix [GCP-8479] Bigtable: 'test_bigtable_list_tables' snippet flakes with '504 Deadline Exceeded'. Jul 31, 2019
@sangramql
Copy link
Author

The code looks fine but the questions are whether the flakes were reproduced, how positive are we about this way of solving them, and if enough testing has been done afterwards to ensure they're gone.

This issue is not consistent in production i suppose. And I was not able to reproduce even once.
I tried different approach to hit this issue,

  1. Created multiple tables(>100) in test function.
  2. Created instance and immediately created multiple tables (> 100) in class setup.
  3. Tried to created multiple instances (>10) and table in an instance, but got the following error
    after creating 6 instances. But didn’t get ‘deadline exceeded’ error though.
  4. Executed parallel threads of snippets.py from nox.

@sangramql
Copy link
Author

Closing this PR as this opened in googleapis#8879

@sangramql sangramql closed this Aug 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants