docs: restructure and consistency

fix extending operations fix prettier update per feedback
ibis-project · Jun 27, 2023 · 351d424 · 351d424
1 parent e14662e
commit 351d424
Show file tree

Hide file tree

Showing 62 changed files with 197 additions and 197 deletions.
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -1,40 +1,51 @@
 * [Home](index.md)
 * [Install](install.md)
-* [Docs](docs/index.md)
-    * [Getting Started](getting_started.md)
-    * [How To Guides](how_to/)
-    * [Execution Backends](backends/)
-    * [Supported Python Versions](supported_python_versions.md)
-    * [User Guide](user_guide/)
-    * [Ibis for SQL Programmers](ibis-for-sql-programmers.ipynb)
-    * [Ibis for pandas Users](ibis-for-pandas-users.ipynb)
-    * [Ibis for dplyr Users](ibis-for-dplyr-users.ipynb)
-    * [Backend Operations Matrix](backends/support_matrix.md)
-    * [Why Ibis?](why_ibis.md)
-* API Reference
-    * [Expressions](api/expressions/index.md)
-        * [Top Level](api/expressions/top_level.md)
-        * [Tables](api/expressions/tables.md)
-        * [Generic Values](api/expressions/generic.md)
-        * [Numeric + Boolean](api/expressions/numeric.md)
-        * [Strings](api/expressions/strings.md)
-        * [Timestamps + Dates + Times](api/expressions/timestamps.md)
-        * [Collections](api/expressions/collections.md)
-        * [Geospatial](api/expressions/geospatial.md)
-    * [Column Selectors](api/selectors.md)
-    * [Data Types](api/datatypes.md)
-    * [Schemas](api/schemas.md)
-    * [Backend Interfaces](api/backends/)
-    * [Configuration](api/config.md)
-* [Releases](release_notes.md)
+* Concepts
+    * [Why Ibis?](concept/why_ibis.md)
+    * [Design](concept/design.md)
+    * [Backends](concept/backends.md)
+* [Backends](backends/)
+* [Tutorials](tutorial/index.md)
+    * [Getting started with Ibis](tutorial/getting_started.md)
+    * [Ibis for SQL users](tutorial/ibis-for-sql-users.ipynb)
+    * [Ibis for pandas users](tutorial/ibis-for-pandas-users.ipynb)
+    * [Ibis for dplyr users](tutorial/ibis-for-dplyr-users.ipynb)
+* How-to guides
+    * [Configure Ibis](how_to/configuration.md)
+    * [Chain expressions with the underscore API](how_to/chain_expressions.md)
+    * [Fill data using window functions](how_to/ffill_bfill_w_window.md)
+    * [Perform self joins](how_to/self_joins.md)
+    * [Sessionize a log of events](how_to/sessionize.md)
+    * [Compute the top K records](how_to/topk.md)
+    * [Join an in-memory DataFrame to a TableExpression](how_to/memtable_join.md)
+    * [Load external data files with the DuckDB backend](how_to/duckdb_register.md)
+    * [Write a Streamlit app with Ibis](how_to/streamlit.md)
+    * [Extend with custom operations](how_to/extending/)
+* Reference
+    * [Expressions](reference/expressions/index.md)
+        * [Top level](reference/expressions/top_level.md)
+        * [Tables](reference/expressions/tables.md)
+        * [Generic Values](reference/expressions/generic.md)
+        * [Numeric and boolean](reference/expressions/numeric.md)
+        * [Strings](reference/expressions/strings.md)
+        * [Timestamps, dates, and times](reference/expressions/timestamps.md)
+        * [Collections](reference/expressions/collections.md)
+        * [Geospatial](reference/expressions/geospatial.md)
+    * [Column selectors](reference/selectors.md)
+    * [Data types](reference/datatypes.md)
+    * [Schemas](reference/schemas.md)
+    * [Backend interfaces](reference/backends/)
+    * [Configuration](reference/config.md)
+    * [Supported Python versions](supported_python_versions.md)
+    * [Release notes](release_notes.md)
 * Blog
-    * [Campaign Finance Analysis with Ibis](blog/rendered/campaign-finance.ipynb)
-    * [Ibis Sneak Peek: Writing to Files](blog/ibis-to-file.md)
-    * [Ibis Sneak Peek: Examples](blog/ibis-examples.md)
-    * [Maximizing Productivity with Selectors](blog/selectors.md)
+    * [Campaign finance analysis with Ibis](blog/rendered/campaign-finance.ipynb)
+    * [Ibis sneak peek: writing to files](blog/ibis-to-file.md)
+    * [Ibis sneak peek: examples](blog/ibis-examples.md)
+    * [Maximizing productivity with selectors](blog/selectors.md)
     * [Ibis + DuckDB + Substrait](blog/ibis_substrait_to_duckdb.md)
     * [Ibis v4.0.0](blog/ibis-version-4.0.0-release.md)
-    * [Analyzing Ibis's CI Data with Ibis](blog/rendered/ci-analysis.ipynb)
+    * [Analyzing Ibis's CI data with Ibis](blog/rendered/ci-analysis.ipynb)
     * [ffill and bfill using ibis](blog/ffill-and-bfill-using-ibis.md)
     * [Ibis v3.1.0](blog/Ibis-version-3.1.0-release.md)
     * [Ibis v3.0.0](blog/Ibis-version-3.0.0-release.md)

diff --git a/docs/backends/support_matrix.md → docs/backends/_support_matrix.md b/docs/backends/support_matrix.md → docs/backends/_support_matrix.md
@@ -3,7 +3,7 @@ hide:
   - toc
 ---
 
-# Operation Support Matrix
+# Operation support matrix
 
 Backends are shown in descending order of the number of supported operations.
 

diff --git a/docs/backends/BigQuery.md → docs/backends/bigquery.md b/docs/backends/BigQuery.md → docs/backends/bigquery.md
@@ -4,6 +4,8 @@ backend_url: https://cloud.google.com/bigquery
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# BigQuery
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/ClickHouse.md → docs/backends/clickhouse.md b/docs/backends/ClickHouse.md → docs/backends/clickhouse.md
@@ -5,6 +5,8 @@ backend_module: clickhouse
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# ClickHouse
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/Dask.md → docs/backends/dask.md b/docs/backends/Dask.md → docs/backends/dask.md
diff --git a/docs/backends/Datafusion.md → docs/backends/datafusion.md b/docs/backends/Datafusion.md → docs/backends/datafusion.md
@@ -7,6 +7,8 @@ exports: ["PyArrow", "Parquet", "Delta Lake", "CSV", "Pandas"]
 imports: ["CSV", "Parquet", "Delta Lake"]
 ---
 
+# DataFusion
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/Druid.md → docs/backends/druid.md b/docs/backends/Druid.md → docs/backends/druid.md
@@ -5,6 +5,8 @@ backend_module: druid
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# Druid
+
 {% include 'backends/badges.md' %}
 
 !!! experimental "Introduced in v5.0"

diff --git a/docs/backends/DuckDB.md → docs/backends/duckdb.md b/docs/backends/DuckDB.md → docs/backends/duckdb.md
@@ -16,6 +16,8 @@ imports:
   ]
 ---
 
+# DuckDB
+
 {% include 'backends/badges.md' %}
 
 ??? danger "`duckdb` >= 0.5.0 requires `duckdb-engine` >= 0.6.2"

diff --git a/docs/backends/Impala.md → docs/backends/impala.md b/docs/backends/Impala.md → docs/backends/impala.md
diff --git a/docs/backends/index.md b/docs/backends/index.md
@@ -1,47 +1,9 @@
 # Backends
 
-See the [configuration guide](../user_guide/configuration.md#default-backend)
-to inspect or reconfigure the backend used by default.
+A backend is where execution of Ibis table expressions occur after compiling into some intermediate representation. A backend is often a database and the intermediate representation often SQL, but several types of backends exist.
 
-## String Generating Backends
+See the [configuration guide](../how_to/configuration.md#default-backend)
+to inspect or reconfigure the backend used by default. View the [operation support matrix](_support_matrix.md) to see which operations
+are supported by each backend.
 
-The first category of backend translate Ibis table expressions into query strings.
-
-The compiler turns each table expression into a query string and passes that query
-to the database through a driver API for execution.
-
-- [Apache Impala](Impala.md)
-- [ClickHouse](ClickHouse.md)
-- [Google BigQuery](BigQuery.md)
-- [HeavyAI](https://github.com/heavyai/ibis-heavyai)
-
-## Expression Generating Backends
-
-The next category of backends translates Ibis table expressions into another
-system's table expression objects, for example, SQLAlchemy.
-
-Instead of generating a query string for each table expression, these backends
-produce another kind of table expression object and typically have high-level APIs
-for execution.
-
-- [Apache Arrow Datafusion](Datafusion.md)
-- [Apache Druid](Druid.md)
-- [Apache PySpark](PySpark.md)
-- [Dask](Dask.md)
-- [DuckDB](DuckDB.md)
-- [MS SQL Server](MSSQL.md)
-- [MySQL](MySQL.md)
-- [Oracle](Oracle.md)
-- [Polars](Polars.md)
-- [PostgreSQL](PostgreSQL.md)
-- [SQLite](SQLite.md)
-- [Snowflake](Snowflake.md)
-- [Trino](Trino.md)
-
-## Direct Execution Backends
-
-The pandas backend is the only direct execution backend. A full description
-of the implementation can be found in the module docstring of the pandas
-backend located in `ibis/backends/pandas/core.py`.
-
-- [pandas](pandas.md)
+Each backend has its own configuration options documented here.
diff --git a/docs/backends/MSSQL.md → docs/backends/mssql.md b/docs/backends/MSSQL.md → docs/backends/mssql.md
@@ -7,6 +7,8 @@ version_added: "4.0"
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# MSSQL
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/MySQL.md → docs/backends/mysql.md b/docs/backends/MySQL.md → docs/backends/mysql.md
@@ -6,6 +6,8 @@ backend_param_style: a SQLAlchemy-style URI
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# MySQL
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/Oracle.md → docs/backends/oracle.md b/docs/backends/Oracle.md → docs/backends/oracle.md
@@ -9,6 +9,8 @@ version_added: "6.0"
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# Oracle
+
 {% include 'backends/badges.md' %}
 
 !!! experimental "Introduced in v6.0"

diff --git a/docs/backends/Polars.md → docs/backends/polars.md b/docs/backends/Polars.md → docs/backends/polars.md
@@ -8,6 +8,8 @@ exports: ["PyArrow", "Parquet", "Delta Lake", "CSV", "Pandas"]
 imports: ["CSV", "Parquet", "Delta Lake", "Pandas"]
 ---
 
+# Polars
+
 {% include 'backends/badges.md' %}
 
 !!! experimental "Introduced in v4.0"

diff --git a/docs/backends/PostgreSQL.md → docs/backends/postgresql.md b/docs/backends/PostgreSQL.md → docs/backends/postgresql.md
@@ -6,6 +6,8 @@ backend_param_style: a SQLAlchemy-style URI
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# PostgreSQL
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/PySpark.md → docs/backends/pyspark.md b/docs/backends/PySpark.md → docs/backends/pyspark.md
@@ -7,6 +7,8 @@ exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 imports: ["CSV", "Parquet"]
 ---
 
+# PySpark
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/Snowflake.md → docs/backends/snowflake.md b/docs/backends/Snowflake.md → docs/backends/snowflake.md
@@ -4,6 +4,8 @@ backend_url: https://snowflake.com/
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# Snowflake
+
 {% include 'backends/badges.md' %}
 
 !!! experimental "Introduced in v4.0"

diff --git a/docs/backends/SQLite.md → docs/backends/sqlite.md b/docs/backends/SQLite.md → docs/backends/sqlite.md
@@ -5,6 +5,8 @@ backend_module: sqlite
 imports: ["CSV", "Parquet", "JSON", "PyArrow", "Pandas", "SQLite", "Postgres"]
 ---
 
+# SQLite
+
 {% include 'backends/badges.md' %}
 
 ## Install

diff --git a/docs/backends/template.md b/docs/backends/template.md
@@ -15,7 +15,7 @@
 
 ## Install
 
-Install ibis and dependencies for the {{ backend_name }} backend:
+Install `ibis` and dependencies for the {{ backend_name }} backend:
 
 === "pip"
 

diff --git a/docs/backends/Trino.md → docs/backends/trino.md b/docs/backends/Trino.md → docs/backends/trino.md
@@ -5,6 +5,8 @@ backend_module: trino
 exports: ["PyArrow", "Parquet", "CSV", "Pandas"]
 ---
 
+# Trino
+
 {% include 'backends/badges.md' %}
 
 !!! experimental "Introduced in v4.0"

diff --git a/docs/concept/backends.md b/docs/concept/backends.md
@@ -0,0 +1,46 @@
+# Backends
+
+A backend is where execution of Ibis table expressions occur after compiling into some intermediate representation. A backend is often a database and the intermediate representation often SQL, but several types of backends exist. See the [backends page](/backends/) for specific documentation on each.
+
+## String generating backends
+
+The first category of backends translate Ibis table expressions into query strings.
+
+The compiler turns each table expression into a query string and passes that query
+to the database through a driver API for execution.
+
+- [Apache Impala](/backends/impala/)
+- [ClickHouse](/backends/clickhouse/)
+- [Google BigQuery](/backends/bigquery/)
+- [HeavyAI](https://github.com/heavyai/ibis-heavyai)
+
+## Expression generating backends
+
+The next category of backends translates Ibis table expressions into another
+system's table expression objects, for example, SQLAlchemy.
+
+Instead of generating a query string for each table expression, these backends
+produce another kind of table expression object and typically have high-level APIs
+for execution.
+
+- [Apache Arrow Datafusion](/backends/datafusion/)
+- [Apache Druid](/backends/druid/)
+- [Apache PySpark](/backends/pyspark/)
+- [Dask](/backends/dask/)
+- [DuckDB](/backends/duckdb/)
+- [MS SQL Server](/backends/mssql/)
+- [MySQL](/backends/mysql/)
+- [Oracle](/backends/oracle/)
+- [Polars](/backends/polars/)
+- [PostgreSQL](/backends/postgresql/)
+- [SQLite](/backends/sqlite/)
+- [Snowflake](/backends/snowflake/)
+- [Trino](/backends/trino/)
+
+## Direct execution backends
+
+The pandas backend is the only direct execution backend. A full description
+of the implementation can be found in the module docstring of the pandas
+backend located in `ibis/backends/pandas/core.py`.
+
+- [pandas](/backends/pandas/)
diff --git a/docs/user_guide/design.md → docs/concept/design.md b/docs/user_guide/design.md → docs/concept/design.md
diff --git a/docs/why_ibis.md → docs/concept/why_ibis.md b/docs/why_ibis.md → docs/concept/why_ibis.md
@@ -1,16 +1,11 @@
----
-hide:
-  - footer
----
-
-# Why try Ibis?
+# Why Ibis?
 
 Ibis is a dataframe interface to execution engines with support for [15+
-backends](./backends/index.md). Ibis doesn't replace your existing execution
+backends](/backends/). Ibis doesn't replace your existing execution
 engine, it _extends_ it with powerful abstractions and intuitive syntax.
 
 Ibis works with what you already have, so why not check out our [getting started
-guide](./getting_started.md)?
+guide](/tutorial/getting_started/)?
 
 # How does Ibis compare to...
 
@@ -39,7 +34,7 @@ we can summarize some key points:
 - Ibis lets you use SQL when you want to (for our SQL-based backends)
 
 If your SQL-fu is strong, we might not convince you to leave it all behind, but
-check out our [Ibis for SQL Programmers guide](./ibis-for-sql-programmers.ipynb)
+check out our [Ibis for SQL users guide](/tutorial/ibis-for-sql-users/)
 and see if it whets your appetite.
 
 ## `pandas`
@@ -58,7 +53,7 @@ Ibis to quickly and easily switch to a different backend that supports
 out-of-core execution.
 
 Ibis syntax is similar to `pandas` syntax, but it isn't a drop-in replacement.
-Check out our [Ibis for pandas Users guide](./ibis-for-pandas-users.ipynb) if
+Check out our [Ibis for pandas Users guide](/tutorial/ibis-for-pandas-users/) if
 you'd like to give Ibis a try!
 
 ## `sqlalchemy` and `sqlglot`
@@ -77,4 +72,4 @@ using SQLGlot.
 
 If you are looking for a dataframe API to construct and execute your analytics
 queries against a large collection of powerful execution engines, then allow us
-point you at the [Ibis Getting Started guide](./getting_started.md).
+point you at the [Ibis Getting Started guide](/tutorial/getting_started/).
diff --git a/docs/docs/index.md b/docs/docs/index.md
-Original file line number
+Diff line change
@@ Expand Up / @@ -3,7 +3,7 @@ hide: @@
       - toc
     ---
-    # Operation Support Matrix
+    # Operation support matrix
     Backends are shown in descending order of the number of supported operations.
@@ Expand Down @@