Support for sharded BigQuery tables

[martin-wiebusch-thg]

Is your feature request related to a problem?

I would like to use Ibis to query sharded BigQuery tables via the wildcard syntax. Ibis seems to accept wildcards in table names, e.g.

t = con.table('my_sharded_table*')

works and gives me the correct schema of the sharded table. However, it seems that I cannot limit the range of the shards via _TABLE_SUFFIX. With

t.filter(_._TABLE_SUFFIX >= '20240613')

I get AttributeError: 'Table' object has no attribute '_TABLE_SUFFIX'

What is the motivation behind your request?

Sharded tables are (still) commonplace in BigQuery and to query them we need support for the _TABLE_SUFFIX pseudo-column.

Describe the solution you'd like

I would like the following code to work:

import ibis
from ibis import _

con = ibis.bigquery.connect()
t = con.table('some_sharded_table*')
t.filter(_._TABLE_SUFFIX >= '20240613')

What version of ibis are you running?

9.0.0

What backend(s) are you using, if any?

BigQuery

Code of Conduct

• I agree to follow this project's Code of Conduct

[cpcloud]

@martin-wiebusch-thg Thanks for the issue, looks like a good feature to add!