Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: fix swallowed error message #6168

Merged
merged 2 commits into from
Oct 10, 2018

Conversation

mistercrunch
Copy link
Contributor

When getting a Cannot query over table (...) without a filter over column(s) (...), the re-raising happening gobbles up the error message
and the context is lost.

This may or may not be the right fix, but clearly highlights the issue
of self._query_job.errors being None in that particular context.
When that is the case, the only context available is that of
DatabaseError without any details which clearly isn't enough detail.

After this fix is applied (wrapping the exception into another exception) both stack traces are shown:
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/job.py", line 2685, in result
    super(QueryJob, self).result(timeout=timeout)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/job.py", line 697, in result
    return super(_AsyncJob, self).result(timeout=timeout)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/future/polling.py", line 115, in result
    self._blocking_poll(timeout=timeout)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/job.py", line 2659, in _blocking_poll
    super(QueryJob, self)._blocking_poll(timeout=timeout)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/future/polling.py", line 94, in _blocking_poll
    retry_(self._done_or_raise)()
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
    on_error=on_error,
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/retry.py", line 177, in retry_target
    return target()
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/future/polling.py", line 73, in _done_or_raise
    if not self.done():
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/job.py", line 2647, in done
    location=self.location)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/client.py", line 584, in _get_query_results
    retry, method='GET', path=path, query_params=extra_params)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/client.py", line 336, in _call_api
    return call()
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
    on_error=on_error,
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/api_core/retry.py", line 177, in retry_target
    return target()
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/_http.py", line 293, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.BadRequest: 400 GET https://www.googleapis.com/bigquery/v2/projects/fsql-production/queries/6ed337a4-e23c-4634-ba94-15254648bdfb?maxResults=0&location=US: Cannot query over table 'fsql-production.records.routes_stream' without a filter over column(s) 'created_at' that can be used for partition elimination

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mbeauchemin/incubator-superset/superset/sql_lab.py", line 185, in execute_sql
    db_engine_spec.execute(cursor, query.executed_sql, async_=True)
  File "/home/mbeauchemin/incubator-superset/superset/db_engine_specs.py", line 376, in execute
    cursor.execute(query)
  File "/home/mbeauchemin/incubator-superset/env/lib/python3.4/site-packages/google/cloud/bigquery/dbapi/cursor.py", line 157, in execute
    raise exceptions.DatabaseError(self._query_job.errors or e)
google.cloud.bigquery.dbapi.exceptions.DatabaseError: 400 GET https://www.googleapis.com/bigquery/v2/projects/fsql-production/queries/6ed337a4-e23c-4634-ba94-15254648bdfb?maxResults=0&location=US: Cannot query over table 'fsql-production.records.routes_stream' without a filter over column(s) 'created_at' that can be used for partition elimination

When getting a "Cannot query over table (...) without a filter over
column(s) (...)", the re-raising happening gobbles up the error message
and the context is lost.

This may or may not be the right fix, but clearly highlights the issue
of `self._query_job.errors` being `None` in that particular context.
When that is the case, the only context available is that of
`DatabaseError` without any details which clearly isn't enough detail.

It seems like wrapping the exception into another exception works as
expected and shows both stack traces as shown here:
@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

@googlebot googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label Oct 4, 2018
@sumedhsakdeo
Copy link

Looks great. CC: @tswast

@tseaver tseaver changed the title [bigquery] fix swallowed error message BigQuery: fix swallowed error message Oct 5, 2018
@tseaver tseaver added the api: bigquery Issues related to the BigQuery API. label Oct 5, 2018
except google.cloud.exceptions.GoogleCloudError:
raise exceptions.DatabaseError(self._query_job.errors)
except google.cloud.exceptions.GoogleCloudError as e:
raise exceptions.DatabaseError(self._query_job.errors or e)

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

The errors on the job resource are mostly redundant with the exception information.
@googlebot
Copy link

CLAs look good, thanks!

@googlebot googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Oct 10, 2018
@tseaver tseaver merged commit 6fda29b into googleapis:master Oct 10, 2018
@mistercrunch
Copy link
Contributor Author

🔥 🕺 🔥

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants