diff --git a/docs/_freeze/backends/clickhouse/execute-results/html.json b/docs/_freeze/backends/clickhouse/execute-results/html.json new file mode 100644 index 000000000000..6a98cef50304 --- /dev/null +++ b/docs/_freeze/backends/clickhouse/execute-results/html.json @@ -0,0 +1,15 @@ +{ + "hash": "d7d652378e73534de8400a7133016093", + "result": { + "markdown": "---\nexecute:\n freeze: auto\ntitle: ClickHouse\n---\n\n\n\n\n\n[https://clickhouse.com](https://clickhouse.com)\n\n![](https://img.shields.io/badge/memtables-native-green?style=flat-square) ![](https://img.shields.io/badge/inputs-ClickHouse tables-blue?style=flat-square) ![](https://img.shields.io/badge/outputs-ClickHouse tables | CSV | pandas | Parquet | PyArrow-orange?style=flat-square)\n\n## Install\n\nInstall Ibis and dependencies for the ClickHouse backend:\n\n::: {.panel-tabset}\n\n## `pip`\n\nInstall with the `clickhouse` extra:\n\n```{.bash}\npip install 'ibis-framework[clickhouse]'\n```\n\nAnd connect:\n\n```{.python}\nimport ibis\n\ncon = ibis.clickhouse.connect() # <1>\n```\n\n1. Adjust connection parameters as needed.\n\n## `conda`\n\nInstall for ClickHouse:\n\n```{.bash}\nconda install -c conda-forge ibis-clickhouse\n```\n\nAnd connect:\n\n```{.python}\nimport ibis\n\ncon = ibis.clickhouse.connect() # <1>\n```\n\n1. Adjust connection parameters as needed.\n\n## `mamba`\n\nInstall for ClickHouse:\n\n```{.bash}\nmamba install -c conda-forge ibis-clickhouse\n```\n\nAnd connect:\n\n```{.python}\nimport ibis\n\ncon = ibis.clickhouse.connect() # <1>\n```\n\n1. Adjust connection parameters as needed.\n\n:::\n\n## Connect\n\n### `ibis.clickhouse.connect`\n\n```python\ncon = ibis.clickhouse.connect(\n user=\"username\",\n password=\"password\",\n host=\"hostname\",\n)\n```\n\n::: {.callout-note}\n`ibis.clickhouse.connect` is a thin wrapper around [`ibis.backends.clickhouse.Backend.do_connect`](#ibis.backends.clickhouse.Backend.do_connect).\n:::\n\n### Connection Parameters\n\n\n\n\n#### do_connect { #ibis.backends.clickhouse.Backend.do_connect }\n\n\n`do_connect(self, host='localhost', port=None, database='default', user='default', password='', client_name='ibis', secure=None, compression=True, **kwargs)`\n\n\nCreate a ClickHouse client for use with Ibis.\n\n##### Parameters\n\n| Name | Type | Description | Default |\n|---------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------|---------------|\n| `host` | str | Host name of the clickhouse server | `'localhost'` |\n| `port` | int \\| None | ClickHouse HTTP server's port. If not passed, the value depends on whether `secure` is `True` or `False`. | `None` |\n| `database` | str | Default database when executing queries | `'default'` |\n| `user` | str | User to authenticate with | `'default'` |\n| `password` | str | Password to authenticate with | `''` |\n| `client_name` | str | Name of client that will appear in clickhouse server logs | `'ibis'` |\n| `secure` | bool \\| None | Whether or not to use an authenticated endpoint | `None` |\n| `compression` | str \\| bool | The kind of compression to use for requests. See https://clickhouse.com/docs/en/integrations/python#compression for more information. | `True` |\n| `kwargs` | Any | Client specific keyword arguments | `{}` |\n\n##### Examples\n\n```python\n>>> import ibis\n>>> client = ibis.clickhouse.connect()\n>>> client\n\n```\n\n\n### `ibis.connect` URL format\n\nIn addition to `ibis.clickhouse.connect`, you can also connect to ClickHouse by\npassing a properly formatted ClickHouse connection URL to `ibis.connect`\n\n```python\ncon = ibis.connect(f\"clickhouse://{user}:{password}@{host}:{port}?secure={secure}\")\n```\n\n## ClickHouse playground\n\nClickHouse provides a free playground with several datasets that you can connect to using Ibis:\n\n::: {#7d05ccde .cell execution_count=2}\n``` {.python .cell-code}\nfrom ibis.interactive import *\n\ncon = ibis.connect(\"clickhouse://play:clickhouse@play.clickhouse.com:443?secure=True\")\nactors = con.table(\"actors\")\nactors\n```\n\n::: {.cell-output .cell-output-display execution_count=22}\n```{=html}\n
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓\n┃ login             type     site_admin  name          company              blog                 location           email                      hireable  bio                                        twitter_username  public_repos  public_gists  followers  following  created_at           updated_at          ┃\n┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩\n│ !string!string!boolean!string!string!string!string!string!boolean!string!string!int64!int64!int64!int64!timestamp(0)!timestamp(0)       │\n├──────────────────┼─────────┼────────────┼──────────────┼─────────────────────┼─────────────────────┼───────────────────┼───────────────────────────┼──────────┼───────────────────────────────────────────┼──────────────────┼──────────────┼──────────────┼───────────┼───────────┼─────────────────────┼─────────────────────┤\n│ 0000Blaze       User    │ False      │ RohanChhetryPulchowk Campus,IOErohanchhetry.com.npSanepa , Lalitpur~                         │ True     │ ~rohanchhetry9   56057832019-02-24 02:31:212023-07-30 11:30:14 │\n│ 007developforfunUser    │ False      │ ~~~~~                         │ False    │ ~~00002015-08-07 11:28:012022-08-12 08:45:30 │\n│ 00arthur00      User    │ False      │ Arthur      ~~Beijing, China   yang_yapo@126.com         │ False    │ ~~7215612017-04-01 13:37:012023-06-15 14:50:12 │\n│ 010001          User    │ False      │ ~~~~1025394547@qq.com         │ False    │ ~~150112015-02-05 03:11:592023-03-17 06:07:01 │\n│ 01001101ilad    User    │ False      │ Milad       ~~~~                         │ False    │ Programmer, Writer and Full-Time Learner.~100002016-10-31 19:12:552023-07-24 11:43:03 │\n│ 010227leo       User    │ False      │ zucker      trip.com           ~Shanghai, China  ~                         │ False    │ ~~217112012-01-11 06:23:152023-07-24 03:35:26 │\n│ 010ric          User    │ False      │ Mario Turic ~~Munich           ~                         │ True     │ Enthusiast and Maker                     ~19023682017-10-27 14:00:072023-08-04 18:44:35 │\n│ 01egen          User    │ False      │ ~~~~~                         │ False    │ ~~10022019-02-27 08:59:002023-06-20 04:02:51 │\n│ 0400H           User    │ False      │ 0400H       ~~Shanghai         git@0400h.cn              │ True     │ HPC & MLSys & PPML                       ~1703422015-12-20 17:38:002023-07-21 11:28:22 │\n│ 0442A403        User    │ False      │ Damir Petrov~~Moscow           petrovdamir2235@gmail.com │ False    │ HSE student                              ~19015302016-11-05 18:59:382023-06-22 06:08:50 │\n│                    │\n└──────────────────┴─────────┴────────────┴──────────────┴─────────────────────┴─────────────────────┴───────────────────┴───────────────────────────┴──────────┴───────────────────────────────────────────┴──────────────────┴──────────────┴──────────────┴───────────┴───────────┴─────────────────────┴─────────────────────┘\n
\n```\n:::\n:::\n\n\n\n\n## `clickhouse.Backend` { #ibis.backends.clickhouse.Backend }\n\n\n### close { #ibis.backends.clickhouse.Backend.close }\n\n\n`close(self)`\n\n\nClose ClickHouse connection.\n\n\n### compile { #ibis.backends.clickhouse.Backend.compile }\n\n\n`compile(self, expr, limit=None, params=None, **kwargs)`\n\n\nCompile an Ibis expression to a ClickHouse SQL string.\n\n\n### create_database { #ibis.backends.clickhouse.Backend.create_database }\n\n\n`create_database(self, name, *, force=False, engine='Atomic')`\n\n\nCreate a new database.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|---------|--------|--------------------------------------------------------------------|------------|\n| `name` | str | Name of the new database. | _required_ |\n| `force` | bool | If `False`, an exception is raised if the database already exists. | `False` |\n\n\n### create_table { #ibis.backends.clickhouse.Backend.create_table }\n\n\n`create_table(self, name, obj=None, *, schema=None, database=None, temp=False, overwrite=False, engine='MergeTree', order_by=None, partition_by=None, sample_by=None, settings=None)`\n\n\nCreate a table in a ClickHouse database.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|----------------|----------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|\n| `name` | str | Name of the table to create | _required_ |\n| `obj` | pd.DataFrame \\| pa.Table \\| ir.Table \\| None | Optional data to create the table with | `None` |\n| `schema` | ibis.Schema \\| None | Optional names and types of the table | `None` |\n| `database` | str \\| None | Database to create the table in | `None` |\n| `temp` | bool | Create a temporary table. This is not yet supported, and exists for API compatibility. | `False` |\n| `overwrite` | bool | Whether to overwrite the table | `False` |\n| `engine` | str | The table engine to use. See [ClickHouse's `CREATE TABLE` documentation](https://clickhouse.com/docs/en/sql-reference/statements/create/table) for specifics. Defaults to [`MergeTree`](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree) with `ORDER BY tuple()` because `MergeTree` is the most feature-complete engine. | `'MergeTree'` |\n| `order_by` | Iterable\\[str\\] \\| None | String column names to order by. Required for some table engines like `MergeTree`. | `None` |\n| `partition_by` | Iterable\\[str\\] \\| None | String column names to partition by | `None` |\n| `sample_by` | str \\| None | String column names to sample by | `None` |\n| `settings` | Mapping\\[str, Any\\] \\| None | Key-value pairs of settings for table creation | `None` |\n\n#### Returns\n\n| Type | Description |\n|--------|---------------|\n| Table | The new table |\n\n\n### create_view { #ibis.backends.clickhouse.Backend.create_view }\n\n\n`create_view(self, name, obj, *, database=None, overwrite=False)`\n\n\nCreate a new view from an expression.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|-------------|-------------|------------------------------------------------------------------------------------------------------|------------|\n| `name` | str | Name of the new view. | _required_ |\n| `obj` | ir.Table | An Ibis table expression that will be used to create the view. | _required_ |\n| `database` | str \\| None | Name of the database where the view will be created, if not provided the database's default is used. | `None` |\n| `overwrite` | bool | Whether to clobber an existing view with the same name | `False` |\n\n#### Returns\n\n| Type | Description |\n|--------|----------------------------|\n| Table | The view that was created. |\n\n\n### drop_database { #ibis.backends.clickhouse.Backend.drop_database }\n\n\n`drop_database(self, name, *, force=False)`\n\n\nDrop a database with name `name`.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|---------|--------|--------------------------------------------------------------------|------------|\n| `name` | str | Database to drop. | _required_ |\n| `force` | bool | If `False`, an exception is raised if the database does not exist. | `False` |\n\n\n### drop_table { #ibis.backends.clickhouse.Backend.drop_table }\n\n\n`drop_table(self, name, database=None, force=False)`\n\n\nDrop a table.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|------------|-------------|------------------------------------------------------------------|------------|\n| `name` | str | Name of the table to drop. | _required_ |\n| `database` | str \\| None | Name of the database where the table exists, if not the default. | `None` |\n| `force` | bool | If `False`, an exception is raised if the table does not exist. | `False` |\n\n\n### drop_view { #ibis.backends.clickhouse.Backend.drop_view }\n\n\n`drop_view(self, name, *, database=None, force=False)`\n\n\nDrop a view.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|------------|-------------|-----------------------------------------------------------------|------------|\n| `name` | str | Name of the view to drop. | _required_ |\n| `database` | str \\| None | Name of the database where the view exists, if not the default. | `None` |\n| `force` | bool | If `False`, an exception is raised if the view does not exist. | `False` |\n\n\n### execute { #ibis.backends.clickhouse.Backend.execute }\n\n\n`execute(self, expr, limit='default', external_tables=None, **kwargs)`\n\n\nExecute an expression.\n\n\n### get_schema { #ibis.backends.clickhouse.Backend.get_schema }\n\n\n`get_schema(self, table_name, database=None)`\n\n\nReturn a Schema object for the indicated table and database.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|--------------|-------------|---------------------------------------------------------------------------------------|------------|\n| `table_name` | str | May **not** be fully qualified. Use `database` if you want to qualify the identifier. | _required_ |\n| `database` | str \\| None | Database name | `None` |\n\n#### Returns\n\n| Type | Description |\n|------------|---------------|\n| sch.Schema | Ibis schema |\n\n\n### has_operation { #ibis.backends.clickhouse.Backend.has_operation }\n\n\n`has_operation(cls, operation)`\n\n\nReturn whether the backend implements support for `operation`.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|-------------|-------------------|----------------------------------------|------------|\n| `operation` | type\\[ops.Value\\] | A class corresponding to an operation. | _required_ |\n\n#### Returns\n\n| Type | Description |\n|--------|-----------------------------------------------|\n| bool | Whether the backend implements the operation. |\n\n#### Examples\n\n```python\n>>> import ibis\n>>> import ibis.expr.operations as ops\n>>> ibis.sqlite.has_operation(ops.ArrayIndex)\nFalse\n>>> ibis.postgres.has_operation(ops.ArrayIndex)\nTrue\n```\n\n\n### insert { #ibis.backends.clickhouse.Backend.insert }\n\n\n`insert(self, name, obj, settings=None, **kwargs)`\n\n\n\n\n\n### list_databases { #ibis.backends.clickhouse.Backend.list_databases }\n\n\n`list_databases(self, like=None)`\n\n\nList existing databases in the current connection.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|--------|-------------|-----------------------------------------------------------------------|-----------|\n| `like` | str \\| None | A pattern in Python's regex format to filter returned database names. | `None` |\n\n#### Returns\n\n| Type | Description |\n|-------------|-----------------------------------------------------------------------------------------------------|\n| list\\[str\\] | The database names that exist in the current connection, that match the `like` pattern if provided. |\n\n\n### list_tables { #ibis.backends.clickhouse.Backend.list_tables }\n\n\n`list_tables(self, like=None, database=None)`\n\n\nReturn the list of table names in the current database.\n\nFor some backends, the tables may be files in a directory,\nor other equivalent entities in a SQL database.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|------------|-------------|----------------------------------------------------------------------------------------|-----------|\n| `like` | str \\| None | A pattern in Python's regex format. | `None` |\n| `database` | str \\| None | The database from which to list tables. If not provided, the current database is used. | `None` |\n\n#### Returns\n\n| Type | Description |\n|-------------|------------------------------------------------------------|\n| list\\[str\\] | The list of the table names that match the pattern `like`. |\n\n\n### raw_sql { #ibis.backends.clickhouse.Backend.raw_sql }\n\n\n`raw_sql(self, query, external_tables=None, **kwargs)`\n\n\nExecute a SQL string `query` against the database.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|-------------------|--------------------------------------|-----------------------------------------------------------------------------------------|------------|\n| `query` | str \\| sg.exp.Expression | Raw SQL string | _required_ |\n| `external_tables` | Mapping\\[str, pd.DataFrame\\] \\| None | Mapping of table name to pandas DataFrames providing external datasources for the query | `None` |\n| `kwargs` | | Backend specific query arguments | `{}` |\n\n#### Returns\n\n| Type | Description |\n|--------|-------------------|\n| Cursor | Clickhouse cursor |\n\n\n### read_csv { #ibis.backends.clickhouse.Backend.read_csv }\n\n\n`read_csv(self, path, table_name=None, engine='MergeTree', **kwargs)`\n\n\nRegister a CSV file as a table in the current backend.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|--------------|-------------|------------------------------------------------------------------------------------------------|------------|\n| `path` | str \\| Path | The data source. A string or Path to the CSV file. | _required_ |\n| `table_name` | str \\| None | An optional name to use for the created table. This defaults to a sequentially generated name. | `None` |\n| `**kwargs` | Any | Additional keyword arguments passed to the backend loading function. | `{}` |\n\n#### Returns\n\n| Type | Description |\n|----------|---------------------------|\n| ir.Table | The just-registered table |\n\n\n### read_parquet { #ibis.backends.clickhouse.Backend.read_parquet }\n\n\n`read_parquet(self, path, table_name=None, engine='MergeTree', **kwargs)`\n\n\nRegister a parquet file as a table in the current backend.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|--------------|-------------|------------------------------------------------------------------------------------------------|------------|\n| `path` | str \\| Path | The data source. | _required_ |\n| `table_name` | str \\| None | An optional name to use for the created table. This defaults to a sequentially generated name. | `None` |\n| `**kwargs` | Any | Additional keyword arguments passed to the backend loading function. | `{}` |\n\n#### Returns\n\n| Type | Description |\n|----------|---------------------------|\n| ir.Table | The just-registered table |\n\n\n### sql { #ibis.backends.clickhouse.Backend.sql }\n\n\n`sql(self, query, schema=None, dialect=None)`\n\n\n\n\n\n### table { #ibis.backends.clickhouse.Backend.table }\n\n\n`table(self, name, database=None)`\n\n\nConstruct a table expression.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|------------|-------------|---------------|------------|\n| `name` | str | Table name | _required_ |\n| `database` | str \\| None | Database name | `None` |\n\n#### Returns\n\n| Type | Description |\n|--------|------------------|\n| Table | Table expression |\n\n\n### to_pyarrow { #ibis.backends.clickhouse.Backend.to_pyarrow }\n\n\n`to_pyarrow(self, expr, *, params=None, limit=None, external_tables=None, **kwargs)`\n\n\nExecute expression and return results in as a pyarrow table.\n\nThis method is eager and will execute the associated expression\nimmediately.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|----------|-----------------------------------|--------------------------------------------------------------------------------------------------------------------|------------|\n| `expr` | ir.Expr | Ibis expression to export to pyarrow | _required_ |\n| `params` | Mapping\\[ir.Scalar, Any\\] \\| None | Mapping of scalar parameter expressions to value. | `None` |\n| `limit` | int \\| str \\| None | An integer to effect a specific row limit. A value of `None` means \"no limit\". The default is in `ibis/config.py`. | `None` |\n| `kwargs` | Any | Keyword arguments | `{}` |\n\n#### Returns\n\n| Type | Description |\n|--------|-----------------------------------------------------------------|\n| Table | A pyarrow table holding the results of the executed expression. |\n\n\n### to_pyarrow_batches { #ibis.backends.clickhouse.Backend.to_pyarrow_batches }\n\n\n`to_pyarrow_batches(self, expr, *, limit=None, params=None, external_tables=None, chunk_size=1000000, **_)`\n\n\nExecute expression and return an iterator of pyarrow record batches.\n\nThis method is eager and will execute the associated expression\nimmediately.\n\n#### Parameters\n\n| Name | Type | Description | Default |\n|-------------------|-----------------------------------|--------------------------------------------------------------------------------------------------------------------|------------|\n| `expr` | ir.Expr | Ibis expression to export to pyarrow | _required_ |\n| `limit` | int \\| str \\| None | An integer to effect a specific row limit. A value of `None` means \"no limit\". The default is in `ibis/config.py`. | `None` |\n| `params` | Mapping\\[ir.Scalar, Any\\] \\| None | Mapping of scalar parameter expressions to value. | `None` |\n| `external_tables` | Mapping\\[str, Any\\] \\| None | External data | `None` |\n| `chunk_size` | int | Maximum number of row to return in a single chunk | `1000000` |\n\n#### Returns\n\n| Type | Description |\n|---------|-------------------|\n| results | RecordBatchReader |\n\n#### Notes\n\nThere are a variety of ways to implement clickhouse -> record batches.\n\n1. FORMAT ArrowStream -> record batches via raw_query\n This has the same type conversion problem(s) as `to_pyarrow`.\n It's harder to address due to lack of `cast` on `RecordBatch`.\n However, this is a ClickHouse problem: we should be able to get\n string data out without a bunch of settings/permissions rigmarole.\n2. Native -> Python objects -> pyarrow batches\n This is what is implemented, using `query_column_block_stream`.\n3. Native -> Python objects -> DataFrame chunks -> pyarrow batches\n This is not implemented because it adds an unnecessary pandas step in\n between Python object -> arrow. We can go directly to record batches\n without pandas in the middle.\n\n\n### truncate_table { #ibis.backends.clickhouse.Backend.truncate_table }\n\n\n`truncate_table(self, name, database=None)`\n\n\n\n\n\n", + "supporting": [ + "clickhouse_files" + ], + "filters": [], + "includes": { + "include-in-header": [ + "\n\n\n" + ] + } + } +} \ No newline at end of file diff --git a/docs/backends/clickhouse.qmd b/docs/backends/clickhouse.qmd index 57d15f708f1e..1832feffa9d8 100644 --- a/docs/backends/clickhouse.qmd +++ b/docs/backends/clickhouse.qmd @@ -1,3 +1,7 @@ +--- +execute: + freeze: auto +--- # ClickHouse [https://clickhouse.com](https://clickhouse.com) @@ -103,25 +107,14 @@ con = ibis.connect(f"clickhouse://{user}:{password}@{host}:{port}?secure={secure ## ClickHouse playground -ClickHouse provides a free playground with several datasets that you can connect to using `ibis`: +ClickHouse provides a free playground with several datasets that you can connect to using Ibis: ```{python} from ibis.interactive import * -con = ibis.clickhouse.connect( - host="play.clickhouse.com", - secure=True, - user="play", - password="clickhouse", -) -con.table("actors") -``` - -or - -```{python} con = ibis.connect("clickhouse://play:clickhouse@play.clickhouse.com:443?secure=True") -con.table("opensky") +actors = con.table("actors") +actors ``` ```{python}