Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: Fix schema recognition of struct field types #9001

Merged
merged 2 commits into from
Aug 14, 2019

Conversation

plamut
Copy link
Contributor

@plamut plamut commented Aug 9, 2019

A struct field can be referred to as "RECORD" or "STRUCT", and this commit assures that the to_api_repr() logic is correct. It also moves STRUCT_TYPES from _pandas_helpers to schema where it belongs more naturally IMO.

Discovered this while writing the "all types" system test, and tried to create a STRUCT field using the standard SQL type name "STRUCT".

How to test

  • Using the BigQuery client, try to create a table with a "STRUCT" column (using that particular type name):
    from google.cloud import bigquery
    
    bq_client = bigquery.Client()
    
    schema = [
        bigquery.SchemaField(
            "person_struct_field",
            "STRUCT",
            fields=(
                bigquery.SchemaField("name", "STRING"),
                bigquery.SchemaField("age", "INT64"),
            )
        ),
    ]
    bq_table = bigquery.table.Table(
        table_ref="{}.{}.active_users".format(PROJECT_ID, DATASET_ID),
        schema=schema,
    )
    
    bq_client.create_table(bq_table)

Actual result (before the fix):
The server responds with an error:

Field <field_name> is type RECORD but has no schema

Expected result (after the fix):
The table is successfully created.

A struct field can be referred to as "RECORD" or "STRUCT", and this
commit assures that the to_api_repr() logic is correct.
@plamut plamut added the api: bigquery Issues related to the BigQuery API. label Aug 9, 2019
@plamut plamut requested review from tswast and a team August 9, 2019 13:16
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 9, 2019
@plamut plamut requested a review from tswast August 13, 2019 21:47
@tswast tswast merged commit 66fe46e into googleapis:master Aug 14, 2019
emar-kar pushed a commit to MaxxleLLC/google-cloud-python that referenced this pull request Sep 18, 2019
* Fix schema recognition of struct field types

A struct field can be referred to as "RECORD" or "STRUCT", and this
commit assures that the to_api_repr() logic is correct.

* Mark STRUCT_TYPES as private in schema.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants