Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: query_and_wait drops unknown properties in the QueryJobConfig #1792

Closed
tswast opened this issue Jan 19, 2024 · 2 comments · Fixed by #1793
Closed

bug: query_and_wait drops unknown properties in the QueryJobConfig #1792

tswast opened this issue Jan 19, 2024 · 2 comments · Fixed by #1793
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@tswast
Copy link
Contributor

tswast commented Jan 19, 2024

In implementing pandas-gbq integration with the new, faster query_and_wait method, a test failure raised the issue that some request body details in a custom job configuration are being dropped. See: https://github.com/googleapis/python-bigquery-pandas/pull/722/files/7739f41989c5d0effbaf66070c6b94b6c3840506#r1459296431

Steps to reproduce

  1. Create a QueryJobConfig with unknown or invalid parameters via from_api_repr()
  2. Pass the job_config to query_and_wait().
  3. Observe that the extra parameters are not sent.

Code example

import google.cloud.bigquery
client = google.cloud.bigquery.Client()
project_id = "swast-scratch"

job_config = google.cloud.bigquery.QueryJobConfig.from_api_repr(
        {
            "copy": {
                "sourceTable": {
                    "projectId": project_id,
                    "datasetId": "publicdata:samples",
                    "tableId": "wikipedia",
                },
                "destinationTable": {
                    "projectId": project_id,
                    "datasetId": "publicdata:samples",
                    "tableId": "wikipedia_copied",
                },
            }
        })
client.query_and_wait("select 1", job_config=job_config)

Stack trace

No stack trace is produced, even though one should be due to an invalid configuration.

@tswast tswast self-assigned this Jan 19, 2024
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jan 19, 2024
@tswast tswast added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Jan 19, 2024
@Linchin
Copy link
Contributor

Linchin commented Jan 19, 2024

I also wonder to which extent clients should verify the validity of requests. I think we intentionally avoid it in many cases, in order to reduce redundancy and keep the client lightweight.

@tswast
Copy link
Contributor Author

tswast commented Jan 22, 2024

Yeah, we do indeed. I was surprised not to see a server-side error for this, so I limited my client-side validation to only the most common mistakes (wrong type of config object).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants