Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: rework why Ibis article to explain what Ibis is and other updates #8490

Merged
merged 19 commits into from
Mar 1, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/_tabsets/install.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ backends = [
{"name": "Polars", "module": "polars"},
{"name": "PostgreSQL", "module": "postgres"},
{"name": "PySpark", "module": "pyspark"},
{"name": "RisingWave", "module": "risingwave"},
{"name": "Snowflake", "module": "snowflake"},
{"name": "SQLite", "module": "sqlite"},
{"name": "Trino", "module": "trino"},
Expand Down
85 changes: 85 additions & 0 deletions docs/backends_sankey.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
from __future__ import annotations

import plotly.graph_objects as go

backend_categories = {
"SQL-generating": [
"BigQuery",
"ClickHouse",
"DataFusion",
"Druid",
"DuckDB",
"Exasol",
"Flink",
"Impala",
"MSSQL",
"MySQL",
"Oracle",
"PostgreSQL",
"PySpark",
"RisingWave",
"Snowflake",
"SQLite",
"Trino",
],
"Expression-generating": ["Dask", "Polars"],
"Naïve execution": ["pandas"],
}

category_colors = {
"Ibis API": "#999999",
"Naïve execution": "#FF8C00",
"Expression-generating": "#6A5ACD",
"SQL-generating": "#3CB371",
}

nodes, links = [], []
node_index = {}

nodes.append({"label": "Ibis API", "color": category_colors["Ibis API"]})
node_index["Ibis API"] = 0


idx = 1
for category, backends in backend_categories.items():
nodes.append({"label": category, "color": category_colors[category]})
node_index[category] = idx
links.append({"source": 0, "target": idx, "value": len(backends)})
idx += 1

for backend in backends:
if backend not in node_index:
nodes.append({"label": backend, "color": category_colors[category]})
node_index[backend] = idx
idx += 1
links.append(
{
"source": node_index[category],
"target": node_index[backend],
"value": 1,
}
)


fig = go.Figure(
data=[
go.Sankey(
node=dict(
pad=20,
thickness=20,
line=dict(color="black", width=0.5),
label=[node["label"] for node in nodes],
color=[node["color"] for node in nodes],
),
link=dict(
source=[link["source"] for link in links],
target=[link["target"] for link in links],
value=[link["value"] for link in links],
),
)
]
)

fig.update_layout(
title_text="Ibis backend types", font_size=14, margin=dict(l=30, r=30, t=80, b=30)
)
30 changes: 23 additions & 7 deletions docs/concepts/who.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,30 @@ most of the core development team employed there. As of writing, this includes
five full-time developers, one technical product manager, and other staff who
lostmygithubaccount marked this conversation as resolved.
Show resolved Hide resolved
contribute to Ibis.

::: {.callout-tip title="Why does Voltron Data support Ibis?"}
Check out the [blog post on why Voltron Data supports
Ibis](../posts/why-voda-supports-ibis/index.qmd)
:::

## Other companies

Ibis is used by many other companies, with various tools built on top of it.
Some include:

- [Google BigQuery DataFrames](https://github.com/googleapis/python-bigquery-dataframes), a clone of the pandas API built on Ibis
- [Starburst Galaxy Python DataFrames](https://www.starburst.io/blog/introducing-python-dataframes/), with support for Ibis
- [Claypot AI's contribution of the Flink backend](https://github.com/claypotai/ibis-flink-example), working in collaboration with Voltron Data
- [Microsoft's Magpie project](https://www.microsoft.com/en-us/research/project/magpie-2/), built on top of Ibis
- [SuperDuperDB](https://github.com/SuperDuperDB/superduperdb), bringing AI to any backend Ibis supports
- [Google BigQuery
DataFrames](https://github.com/googleapis/python-bigquery-dataframes), a clone
of the pandas API built on Ibis
- [Starburst Galaxy Python
DataFrames](https://www.starburst.io/blog/introducing-python-dataframes/), with
support for Ibis
- [Claypot AI's contribution of the Flink
backend](https://github.com/claypotai/ibis-flink-example), working in
collaboration with Voltron Data
- [Microsoft's Magpie
project](https://www.microsoft.com/en-us/research/project/magpie-2/), built on
top of Ibis
- [SuperDuperDB](https://github.com/SuperDuperDB/superduperdb), bringing AI to
any backend Ibis supports

Ibis is also contributed to by other companies. You can [look through the full
list of contributors on
Expand All @@ -41,8 +55,10 @@ Wes, Voltron Data, and others to solve problems seen throughout the space that
are compounding as data volume and AI complexity increase. Some good background
material on the composable data ecosystem and Ibis can be found at:

- ["Apache Arrow and the '10 Things I Hate About pandas'" by Wes](https://wesmckinney.com/blog/apache-arrow-pandas-internals/)
- ["The Road to Composable Data Systems: Thoughts on the Last 15 Years and the Future" by Wes](https://wesmckinney.com/blog/looking-back-15-years/)
- ["Apache Arrow and the '10 Things I Hate About pandas'" by
Wes](https://wesmckinney.com/blog/apache-arrow-pandas-internals/)
- ["The Road to Composable Data Systems: Thoughts on the Last 15 Years and the
Future" by Wes](https://wesmckinney.com/blog/looking-back-15-years/)
- ["The Composable Codex" by Voltron Data](https://voltrondata.com/codex)

## Support for production workloads
Expand Down
Loading
Loading