Skip to content

Commit

Permalink
feat(api): make create_table uniform
Browse files Browse the repository at this point in the history
This commit addresses most outstanding DDL API discrepancy issues including:

- `create_table`/`create_view` for pandas, dask and polars
- making the various DDL APIs as uniform as possible (see clickhouse for
  an example of divergence)
- deprecation of `load_data` (except on impala, since it's significantly
  different from the others)
- add clickhouse implementations of `create_table`/`create_view`/`create_database`
- standardization of APIs for creating tables

During the process of getting all of this to work, I uncovered multiple
issues with `snowflake-sqlalchemy`'s quoting behavior and had to monkey
patch in `normalize_name` to avoid the broken heuristic they are using.

Additionally, to avoid having to solve the "which case should I use?"
problem in multiple places, I decided to remove our backend-scoped
use of `sqlalchemy.MetaData`. Without removing it, we'd have to deal
with identifiers' case not matching. It's possible there's a performance
hit, but removing this maintenance burden until someone comes along
saying it's slow is worth it IMO.

BREAKING CHANGE: Snowflake identifiers are now kept **as is** from the database. Many table names and column names may now be in SHOUTING CASE. Adjust code accordingly.
  • Loading branch information
cpcloud committed Mar 15, 2023
1 parent 60c7382 commit 833c698
Show file tree
Hide file tree
Showing 27 changed files with 557 additions and 326 deletions.
44 changes: 24 additions & 20 deletions ibis/backends/base/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -769,17 +769,19 @@ def create_database(self, name: str, force: bool = False) -> None:
f'Backend "{self.name}" does not implement "create_database"'
)

@abc.abstractmethod
def create_table(
self,
name: str,
obj: pd.DataFrame | ir.Table | None = None,
*,
schema: ibis.Schema | None = None,
database: str | None = None,
temp: bool = False,
overwrite: bool = False,
) -> ir.Table:
"""Create a new table.
Not all backends implement this method.
Parameters
----------
name
Expand All @@ -794,19 +796,22 @@ def create_table(
database
Name of the database where the table will be created, if not the
default.
temp
Whether a table is temporary or not
overwrite
Whether to clobber existing data
Returns
-------
Table
The table that was created.
"""
raise NotImplementedError(
f'Backend "{self.name}" does not implement "create_table"'
)

@abc.abstractmethod
def drop_table(
self,
name: str,
*,
database: str | None = None,
force: bool = False,
) -> None:
Expand All @@ -825,36 +830,38 @@ def drop_table(
f'Backend "{self.name}" does not implement "drop_table"'
)

@abc.abstractmethod
def create_view(
self,
name: str,
expr: ir.Table,
obj: ir.Table,
*,
database: str | None = None,
overwrite: bool = False,
) -> ir.Table:
"""Create a view.
"""Create a new view from an expression.
Parameters
----------
name
Name for the new view.
expr
An Ibis table expression that will be used to extract the query
of the view.
Name of the new view.
obj
An Ibis table expression that will be used to create the view.
database
Name of the database where the view will be created, if not the
default.
Name of the database where the view will be created, if not
provided the database's default is used.
overwrite
Whether to clobber an existing view with the same name
Returns
-------
Table
The view that was created.
"""
raise NotImplementedError(
f'Backend "{self.name}" does not implement "create_view"'
)

@abc.abstractmethod
def drop_view(
self, name: str, database: str | None = None, force: bool = False
self, name: str, *, database: str | None = None, force: bool = False
) -> None:
"""Drop a view.
Expand All @@ -867,9 +874,6 @@ def drop_view(
force
If `False`, an exception is raised if the view does not exist.
"""
raise NotImplementedError(
f'Backend "{self.name}" does not implement "drop_view"'
)

@classmethod
def has_operation(cls, operation: type[ops.Value]) -> bool:
Expand Down
Loading

0 comments on commit 833c698

Please sign in to comment.