From 1527655b9cbf455ecc82757d19fa78184415db3a Mon Sep 17 00:00:00 2001 From: Phillip Cloud <417981+cpcloud@users.noreply.github.com> Date: Thu, 31 Aug 2023 09:39:09 -0400 Subject: [PATCH] docs(blog): add tags to blog posts --- .../posts/ibis-examples/index/execute-results/html.json | 4 ++-- .../posts/selectors/index/execute-results/html.json | 8 +++++--- docs/posts/campaign-finance/index.qmd | 4 ++++ docs/posts/ci-analysis/index.qmd | 4 ++++ docs/posts/ffill-and-bfill-using-ibis/index.qmd | 2 ++ docs/posts/ibis-analytics/index.qmd | 4 +++- docs/posts/ibis-examples/index.qmd | 2 ++ docs/posts/ibis-to-file/index.qmd | 3 +++ docs/posts/ibis_substrait_to_duckdb/index.qmd | 3 +++ docs/posts/selectors/index.qmd | 3 +++ docs/posts/torch/index.qmd | 4 ++++ 11 files changed, 35 insertions(+), 6 deletions(-) diff --git a/docs/_freeze/posts/ibis-examples/index/execute-results/html.json b/docs/_freeze/posts/ibis-examples/index/execute-results/html.json index 7c7a6dce63c0..7f240b300b33 100644 --- a/docs/_freeze/posts/ibis-examples/index/execute-results/html.json +++ b/docs/_freeze/posts/ibis-examples/index/execute-results/html.json @@ -1,7 +1,7 @@ { - "hash": "8b6f5258426c33dcdd642ed7e29b303b", + "hash": "bf567943230271e22d981edfc997116e", "result": { - "markdown": "---\ntitle: \"Ibis sneak peek: examples\"\nauthor: Kae Suarez\ndate: 2023-03-08\ncategories:\n - blog\n---\n\nIbis has been moving quickly to provide a powerful but easy-to-use interface for interacting with analytical engines. However, as we’re approaching the 5.0 release of Ibis, we’ve realized that moving from not knowing Ibis to writing a first expression is not trivial.\n\nAs is, in our tutorial structure, work must be done on the user’s part — though we do provide the commands — to download a SQLite database onto disk, which can only be used with said backend. We feel that this put too much emphasis on a single backend, and added too much effort into picking the right backend for the first tutorial. We want minimal steps between users and learning the Ibis API.\n\nThis is why we’ve added the `ibis.examples` module.\n\n## Getting Started with Examples\n\nThis module offers in-Ibis access to multiple small tables (the largest is around only 30k rows), which are downloaded when requested and immediately read into the backend upon completion. We worked to keep pulling in examples simple, so it looks like this:\n\n::: {#eec00f31 .cell execution_count=1}\n``` {.python .cell-code}\nimport ibis\nimport ibis.examples as ex\n\nibis.options.interactive = True\n\nt = ex.penguins.fetch()\nt\n```\n\n::: {.cell-output .cell-output-display execution_count=1}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nAnother advantage of this new method is that we were able to register all of them so you can tab-complete, as you can see here:\n\n![Tab Complete](./tab_complete.png)\n\nOnce you’ve retrieved an example table, you can get straight to learning and experimenting, instead of struggling with just getting the data itself.\n\nIn the future, our tutorials will use the _examples_ module to to help speed up learning of the Ibis framework.\n\nInterested in Ibis? Docs are available on this very website, at:\n\n- [Ibis Docs](https://ibis-project.org/)\n\nand the repo is always at:\n\n- [Ibis GitHub](https://github.com/ibis-project/ibis)\n\nPlease feel free to reach out on GitHub!\n\n", + "markdown": "---\ntitle: \"Ibis sneak peek: examples\"\nauthor: Kae Suarez\ndate: 2023-03-08\ncategories:\n - blog\n - new feature\n - sneak peek\n---\n\nIbis has been moving quickly to provide a powerful but easy-to-use interface for interacting with analytical engines. However, as we’re approaching the 5.0 release of Ibis, we’ve realized that moving from not knowing Ibis to writing a first expression is not trivial.\n\nAs is, in our tutorial structure, work must be done on the user’s part — though we do provide the commands — to download a SQLite database onto disk, which can only be used with said backend. We feel that this put too much emphasis on a single backend, and added too much effort into picking the right backend for the first tutorial. We want minimal steps between users and learning the Ibis API.\n\nThis is why we’ve added the `ibis.examples` module.\n\n## Getting Started with Examples\n\nThis module offers in-Ibis access to multiple small tables (the largest is around only 30k rows), which are downloaded when requested and immediately read into the backend upon completion. We worked to keep pulling in examples simple, so it looks like this:\n\n::: {#f1971251 .cell execution_count=1}\n``` {.python .cell-code}\nimport ibis\nimport ibis.examples as ex\n\nibis.options.interactive = True\n\nt = ex.penguins.fetch()\nt\n```\n\n::: {.cell-output .cell-output-display execution_count=1}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nAnother advantage of this new method is that we were able to register all of them so you can tab-complete, as you can see here:\n\n![Tab Complete](./tab_complete.png)\n\nOnce you’ve retrieved an example table, you can get straight to learning and experimenting, instead of struggling with just getting the data itself.\n\nIn the future, our tutorials will use the _examples_ module to to help speed up learning of the Ibis framework.\n\nInterested in Ibis? Docs are available on this very website, at:\n\n- [Ibis Docs](https://ibis-project.org/)\n\nand the repo is always at:\n\n- [Ibis GitHub](https://github.com/ibis-project/ibis)\n\nPlease feel free to reach out on GitHub!\n\n", "supporting": [ "index_files" ], diff --git a/docs/_freeze/posts/selectors/index/execute-results/html.json b/docs/_freeze/posts/selectors/index/execute-results/html.json index 40446b2874b4..3f88ec9b722d 100644 --- a/docs/_freeze/posts/selectors/index/execute-results/html.json +++ b/docs/_freeze/posts/selectors/index/execute-results/html.json @@ -1,8 +1,10 @@ { - "hash": "912e7803cf9a47bd95646d5c669a05fb", + "hash": "332826e0cfba3cf42ba1c48417205cc4", "result": { - "markdown": "---\ntitle: \"Maximizing productivity with selectors\"\nauthor: Phillip Cloud\ndate: 2023-02-27\ncategories:\n - blog\n---\n\nBefore Ibis 5.0 it's been challenging to concisely express whole-table\noperations with ibis. Happily this is no longer the case in ibis 5.0.\n\nLet's jump right in!\n\nWe'll look at selectors examples using the [`palmerpenguins` data\nset](https://allisonhorst.github.io/palmerpenguins/) with the [DuckDB\nbackend](https://ibis-project.org/backends/DuckDB/).\n\n## Setup\n\n::: {#7ea414d5 .cell execution_count=1}\n``` {.python .cell-code}\nfrom ibis.interactive import *\n\nt = ex.penguins.fetch()\nt\n```\n\n::: {.cell-output .cell-output-display execution_count=1}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n## Examples\n\n### Normalization\n\nLet's say you want to compute the\n[z-score](https://en.wikipedia.org/wiki/Standard_score) of every numeric column\nand replace the existing data with that normalized value. Here's how you'd do\nthat with selectors:\n\n::: {#98645d71 .cell execution_count=2}\n``` {.python .cell-code}\nt.mutate(s.across(s.numeric(), (_ - _.mean()) / _.std()))\n```\n\n::: {.cell-output .cell-output-display execution_count=2}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year      ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ float64   │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────────┤\n│ Adelie  │ Torgersen │      -0.883205 │      0.784300 │         -1.416272 │   -0.563317 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.809939 │      0.126003 │         -1.060696 │   -0.500969 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.663408 │      0.429833 │         -0.420660 │   -1.186793 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │ -1.257484 │\n│ Adelie  │ Torgersen │      -1.322799 │      1.088129 │         -0.562890 │   -0.937403 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.846572 │      1.746426 │         -0.776236 │   -0.688012 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.919837 │      0.328556 │         -1.416272 │   -0.719186 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.864888 │      1.240044 │         -0.420660 │    0.590115 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -1.799025 │      0.480471 │         -0.562890 │   -0.906229 │ NULL   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.352029 │      1.543873 │         -0.776236 │    0.060160 │ NULL   │ -1.257484 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │         … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────────┘\n
\n```\n:::\n:::\n\n\n### What's Up With the `year` Column?\n\nWhoops, looks like we included `year` in our normalization because it's an\n`int64` column (and therefore numeric) but normalizing the year doesn't make\nsense.\n\nWe can exclude `year` from the normalization using another selector:\n\n::: {#8beef7e9 .cell execution_count=3}\n``` {.python .cell-code}\nt.mutate(s.across(s.numeric() & ~s.c(\"year\"), (_ - _.mean()) / _.std()))\n```\n\n::: {.cell-output .cell-output-display execution_count=3}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │      -0.883205 │      0.784300 │         -1.416272 │   -0.563317 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -0.809939 │      0.126003 │         -1.060696 │   -0.500969 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.663408 │      0.429833 │         -0.420660 │   -1.186793 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      -1.322799 │      1.088129 │         -0.562890 │   -0.937403 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.846572 │      1.746426 │         -0.776236 │   -0.688012 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -0.919837 │      0.328556 │         -1.416272 │   -0.719186 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.864888 │      1.240044 │         -0.420660 │    0.590115 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -1.799025 │      0.480471 │         -0.562890 │   -0.906229 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      -0.352029 │      1.543873 │         -0.776236 │    0.060160 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n`c` is short for \"column\" and the `~` means \"negate\". Combining those we get \"not the year column\"!\n\nPretty neat right?\n\n### Composable Group By\n\nThe power of this approach comes in when you want the grouped version. Perhaps\nwe think some of these columns vary by species.\n\nWith selectors, all you need to do is slap a `.group_by(\"species\")` onto `t`:\n\n::: {#4a01494a .cell execution_count=4}\n``` {.python .cell-code}\nt.group_by(\"species\").mutate(\n s.across(s.numeric() & ~s.c(\"year\"), (_ - _.mean()) / _.std())\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=4}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │       0.791697 │     -1.270997 │          0.160007 │   -0.001444 │ female │  2008 │\n│ Adelie  │ Biscoe    │       1.467524 │     -0.038103 │          0.924596 │    0.816322 │ male   │  2009 │\n│ Adelie  │ Dream     │       0.378692 │      0.619441 │         -0.910418 │    2.070231 │ male   │  2007 │\n│ Adelie  │ Dream     │      -0.860324 │     -0.284681 │         -1.216254 │   -1.200835 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.785232 │      0.783827 │          0.465843 │   -0.546622 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.190962 │      1.852335 │          0.007089 │   -0.110480 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.040778 │     -0.449067 │         -1.369172 │   -0.164997 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.153416 │      1.030405 │          0.771678 │    2.124749 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -1.761426 │     -0.202489 │          0.465843 │   -0.492104 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │       1.204702 │      1.523563 │          0.007089 │    1.197947 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nSince ibis translates this into a run-of-the-mill selection as if you had\ncalled `select` or `mutate` without selectors, nothing special is needed for a\nbackend to work with these new constructs.\n\nLet's look at some more examples.\n\n### Min-max Normalization\n\nGrouped min/max normalization? Easy:\n\n::: {#fa8fb365 .cell execution_count=5}\n``` {.python .cell-code}\nt.group_by(\"species\").mutate(\n s.across(s.numeric() & ~s.c(\"year\"), (_ - _.min()) / (_.max() - _.min()))\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=5}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │       0.633094 │      0.216667 │          0.500000 │    0.441558 │ female │  2008 │\n│ Adelie  │ Biscoe    │       0.762590 │      0.466667 │          0.631579 │    0.636364 │ male   │  2009 │\n│ Adelie  │ Dream     │       0.553957 │      0.600000 │          0.315789 │    0.935065 │ male   │  2007 │\n│ Adelie  │ Dream     │       0.316547 │      0.416667 │          0.263158 │    0.155844 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.330935 │      0.633333 │          0.552632 │    0.311688 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.517986 │      0.850000 │          0.473684 │    0.415584 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.489209 │      0.383333 │          0.236842 │    0.402597 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.510791 │      0.683333 │          0.605263 │    0.948052 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.143885 │      0.433333 │          0.552632 │    0.324675 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │       0.712230 │      0.783333 │          0.473684 │    0.727273 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n### Casting and Munging\n\nHow about casting every column whose name ends with any of the strings `\"mm\"`\nor `\"g\"` to a `float32`? No problem!\n\n::: {#3ee1a042 .cell execution_count=6}\n``` {.python .cell-code}\nt.mutate(s.across(s.endswith((\"mm\", \"g\")), _.cast(\"float32\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=6}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float32        │ float32       │ float32           │ float32     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │      39.099998 │     18.700001 │             181.0 │      3750.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      39.500000 │     17.400000 │             186.0 │      3800.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      40.299999 │     18.000000 │             195.0 │      3250.0 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      36.700001 │     19.299999 │             193.0 │      3450.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      39.299999 │     20.600000 │             190.0 │      3650.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      38.900002 │     17.799999 │             181.0 │      3625.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      39.200001 │     19.600000 │             195.0 │      4675.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      34.099998 │     18.100000 │             193.0 │      3475.0 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      42.000000 │     20.200001 │             190.0 │      4250.0 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nWe can make all string columns have the same case too!\n\n::: {#fe2957b8 .cell execution_count=7}\n``` {.python .cell-code}\nt.mutate(s.across(s.of_type(\"string\"), _.lower()))\n```\n\n::: {.cell-output .cell-output-display execution_count=7}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ adelie  │ torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ adelie  │ torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ adelie  │ torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ adelie  │ torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ adelie  │ torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ adelie  │ torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ adelie  │ torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ adelie  │ torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ adelie  │ torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ adelie  │ torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n### Multiple Computations per Column\n\nWhat if I want to compute multiple things? Heck yeah!\n\n::: {#c6a0de34 .cell execution_count=8}\n``` {.python .cell-code}\nt.group_by(\"sex\").mutate(\n s.across(\n s.numeric() & ~s.c(\"year\"),\n dict(centered=_ - _.mean(), zscore=(_ - _.mean()) / _.std()),\n )\n).select(\"sex\", s.endswith((\"_centered\", \"_zscore\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=8}\n```{=html}\n
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ sex    ┃ bill_length_mm_centered ┃ bill_depth_mm_centered ┃ flipper_length_mm_centered ┃ body_mass_g_centered ┃ bill_length_mm_zscore ┃ bill_depth_mm_zscore ┃ flipper_length_mm_zscore ┃ body_mass_g_zscore ┃\n┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string │ float64                 │ float64                │ float64                    │ float64              │ float64               │ float64              │ float64                  │ float64            │\n├────────┼─────────────────────────┼────────────────────────┼────────────────────────────┼──────────────────────┼───────────────────────┼──────────────────────┼──────────────────────────┼────────────────────┤\n│ female │                 4.10303 │              -1.925455 │                  11.636364 │           937.727273 │              0.836760 │            -1.072270 │                 0.930851 │           1.407635 │\n│ female │                 1.20303 │              -2.425455 │                  10.636364 │           712.727273 │              0.245342 │            -1.350716 │                 0.850856 │           1.069885 │\n│ female │                -8.09697 │               0.674545 │                 -12.363636 │          -462.272727 │             -1.651271 │             0.375649 │                -0.989030 │          -0.693924 │\n│ female │                -5.89697 │               0.874545 │                 -10.363636 │          -562.272727 │             -1.202610 │             0.487027 │                -0.829039 │          -0.844035 │\n│ female │                -0.99697 │               1.174545 │                 -15.363636 │          -662.272727 │             -0.203319 │             0.654095 │                -1.229015 │          -0.994147 │\n│ female │                -5.49697 │               1.374545 │                 -12.363636 │          -162.272727 │             -1.121035 │             0.765473 │                -0.989030 │          -0.243590 │\n│ female │                -3.39697 │               2.574545 │                  -2.363636 │          -412.272727 │             -0.692768 │             1.433743 │                -0.189079 │          -0.618868 │\n│ female │                -7.69697 │               1.974545 │                 -13.363636 │          -537.272727 │             -1.569697 │             1.099608 │                -1.069025 │          -0.806507 │\n│ female │                -4.29697 │               1.874545 │                 -23.363636 │          -462.272727 │             -0.876311 │             1.043919 │                -1.868975 │          -0.693924 │\n│ female │                -6.19697 │               2.774545 │                  -8.363636 │           -62.272727 │             -1.263791 │             1.545122 │                -0.669049 │          -0.093478 │\n│ …      │                       … │                      … │                          … │                    … │                     … │                    … │                        … │                  … │\n└────────┴─────────────────────────┴────────────────────────┴────────────────────────────┴──────────────────────┴───────────────────────┴──────────────────────┴──────────────────────────┴────────────────────┘\n
\n```\n:::\n:::\n\n\nDon't like the naming convention?\n\nPass a function to make your own name!\n\n::: {#1b283810 .cell execution_count=9}\n``` {.python .cell-code}\nt.select(s.startswith(\"bill\")).mutate(\n s.across(\n s.all(),\n dict(x=_ - _.mean(), y=_.max()),\n names=lambda col, fn: f\"{col}_{fn}_improved\",\n )\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=9}\n```{=html}\n
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ bill_length_mm ┃ bill_depth_mm ┃ bill_length_mm_x_improved ┃ bill_depth_mm_x_improved ┃ bill_length_mm_y_improved ┃ bill_depth_mm_y_improved ┃\n┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n│ float64        │ float64       │ float64                   │ float64                  │ float64                   │ float64                  │\n├────────────────┼───────────────┼───────────────────────────┼──────────────────────────┼───────────────────────────┼──────────────────────────┤\n│           39.1 │          18.7 │                  -4.82193 │                  1.54883 │                      59.6 │                     21.5 │\n│           39.5 │          17.4 │                  -4.42193 │                  0.24883 │                      59.6 │                     21.5 │\n│           40.3 │          18.0 │                  -3.62193 │                  0.84883 │                      59.6 │                     21.5 │\n│            nan │           nan │                       nan │                      nan │                      59.6 │                     21.5 │\n│           36.7 │          19.3 │                  -7.22193 │                  2.14883 │                      59.6 │                     21.5 │\n│           39.3 │          20.6 │                  -4.62193 │                  3.44883 │                      59.6 │                     21.5 │\n│           38.9 │          17.8 │                  -5.02193 │                  0.64883 │                      59.6 │                     21.5 │\n│           39.2 │          19.6 │                  -4.72193 │                  2.44883 │                      59.6 │                     21.5 │\n│           34.1 │          18.1 │                  -9.82193 │                  0.94883 │                      59.6 │                     21.5 │\n│           42.0 │          20.2 │                  -1.92193 │                  3.04883 │                      59.6 │                     21.5 │\n│              … │             … │                         … │                        … │                         … │                        … │\n└────────────────┴───────────────┴───────────────────────────┴──────────────────────────┴───────────────────────────┴──────────────────────────┘\n
\n```\n:::\n:::\n\n\nDon't like lambda functions? We support a format string too!\n\n::: {#5dfcfe9e .cell execution_count=10}\n``` {.python .cell-code}\nt.select(s.startswith(\"bill\")).mutate(\n s.across(\n s.all(),\n func=dict(x=_ - _.mean(), y=_.max()),\n names=\"{col}_{fn}_improved\",\n )\n).head(2)\n```\n\n::: {.cell-output .cell-output-display execution_count=10}\n```{=html}\n
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ bill_length_mm ┃ bill_depth_mm ┃ bill_length_mm_x_improved ┃ bill_depth_mm_x_improved ┃ bill_length_mm_y_improved ┃ bill_depth_mm_y_improved ┃\n┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n│ float64        │ float64       │ float64                   │ float64                  │ float64                   │ float64                  │\n├────────────────┼───────────────┼───────────────────────────┼──────────────────────────┼───────────────────────────┼──────────────────────────┤\n│           39.1 │          18.7 │                  -4.82193 │                  1.54883 │                      59.6 │                     21.5 │\n│           39.5 │          17.4 │                  -4.42193 │                  0.24883 │                      59.6 │                     21.5 │\n└────────────────┴───────────────┴───────────────────────────┴──────────────────────────┴───────────────────────────┴──────────────────────────┘\n
\n```\n:::\n:::\n\n\n### Working with other Ibis APIs\n\nWe've seen lots of mutate use, but selectors also work with `.agg`:\n\n::: {#27630eb6 .cell execution_count=11}\n``` {.python .cell-code}\nt.group_by(\"year\").agg(s.across(s.numeric() & ~s.c(\"year\"), _.mean())).order_by(\"year\")\n```\n\n::: {.cell-output .cell-output-display execution_count=11}\n```{=html}\n
┏━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n┃ year  ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃\n┡━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n│ int64 │ float64        │ float64       │ float64           │ float64     │\n├───────┼────────────────┼───────────────┼───────────────────┼─────────────┤\n│  2007 │      43.740367 │     17.427523 │        196.880734 │ 4124.541284 │\n│  2008 │      43.541228 │     16.914035 │        202.798246 │ 4266.666667 │\n│  2009 │      44.452941 │     17.125210 │        202.806723 │ 4210.294118 │\n└───────┴────────────────┴───────────────┴───────────────────┴─────────────┘\n
\n```\n:::\n:::\n\n\nNaturally, selectors work in grouping keys too, for even more convenience:\n\n::: {#49f1501d .cell execution_count=12}\n``` {.python .cell-code}\nt.group_by(~s.numeric() | s.c(\"year\")).mutate(\n s.across(s.numeric() & ~s.c(\"year\"), dict(centered=_ - _.mean(), std=_.std()))\n).select(\"species\", s.endswith((\"_centered\", \"_std\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=12}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n┃ species ┃ bill_length_mm_centered ┃ bill_depth_mm_centered ┃ flipper_length_mm_centered ┃ body_mass_g_centered ┃ bill_length_mm_std ┃ bill_depth_mm_std ┃ flipper_length_mm_std ┃ body_mass_g_std ┃\n┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n│ string  │ float64                 │ float64                │ float64                    │ float64              │ float64            │ float64           │ float64               │ float64         │\n├─────────┼─────────────────────────┼────────────────────────┼────────────────────────────┼──────────────────────┼────────────────────┼───────────────────┼───────────────────────┼─────────────────┤\n│ Adelie  │                   -1.46 │               0.400000 │                  -1.600000 │          -170.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                   -0.96 │              -0.200000 │                   3.400000 │           180.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                   -0.36 │              -1.100000 │                  -1.600000 │            30.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                    1.44 │               0.300000 │                   1.400000 │          -220.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                    1.34 │               0.600000 │                  -1.600000 │           180.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Gentoo  │                    1.00 │               0.935294 │                  11.117647 │           147.058824 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                    1.00 │              -0.164706 │                  -0.882353 │           147.058824 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -1.40 │              -0.864706 │                  -3.882353 │          -152.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -2.30 │              -0.064706 │                   0.117647 │          -352.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -2.20 │               0.035294 │                  -3.882353 │          -402.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ …       │                       … │                      … │                          … │                    … │                  … │                 … │                     … │               … │\n└─────────┴─────────────────────────┴────────────────────────┴────────────────────────────┴──────────────────────┴────────────────────┴───────────────────┴───────────────────────┴─────────────────┘\n
\n```\n:::\n:::\n\n\n### Filtering Selectors\n\nYou can also express complex filters more concisely.\n\nLet's say we only want to keep rows where all the bill size z-score related\ncolumns' absolute values are greater than 2.\n\n::: {#d62a82d1 .cell execution_count=13}\n``` {.python .cell-code}\nt.drop(\"year\").group_by(\"species\").mutate(\n s.across(s.numeric(), dict(zscore=(_ - _.mean()) / _.std()))\n).filter(s.if_all(s.startswith(\"bill\") & s.endswith(\"_zscore\"), _.abs() > 2))\n```\n\n::: {.cell-output .cell-output-display execution_count=13}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ bill_length_mm_zscore ┃ bill_depth_mm_zscore ┃ flipper_length_mm_zscore ┃ body_mass_g_zscore ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ float64               │ float64              │ float64                  │ float64            │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────────────────────┼──────────────────────┼──────────────────────────┼────────────────────┤\n│ Adelie  │ Torgersen │           46.0 │          21.5 │               194 │        4200 │ male   │              2.706539 │             2.592071 │                 0.618760 │           1.088911 │\n│ Adelie  │ Dream     │           32.1 │          15.5 │               188 │        3050 │ female │             -2.512345 │            -2.339505 │                -0.298747 │          -1.418906 │\n│ Gentoo  │ Biscoe    │           55.9 │          17.0 │               228 │        5600 │ male   │              2.724046 │             2.056508 │                 1.667394 │           1.039411 │\n│ Gentoo  │ Biscoe    │           59.6 │          17.0 │               230 │        6050 │ male   │              3.924621 │             2.056508 │                 1.975799 │           1.932062 │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────────────────────┴──────────────────────┴──────────────────────────┴────────────────────┘\n
\n```\n:::\n:::\n\n\n### Bonus: Generated SQL\n\nThe SQL for that last expression is pretty gnarly:\n\n::: {#118b13de .cell execution_count=14}\n``` {.python .cell-code}\nibis.to_sql(\n t.drop(\"year\")\n .group_by(\"species\")\n .mutate(s.across(s.numeric(), dict(zscore=(_ - _.mean()) / _.std())))\n .filter(s.if_all(s.startswith(\"bill\") & s.endswith(\"_zscore\"), _.abs() > 2))\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=14}\n```sql\nWITH t0 AS (\n SELECT\n t2.species AS species,\n t2.island AS island,\n t2.bill_length_mm AS bill_length_mm,\n t2.bill_depth_mm AS bill_depth_mm,\n t2.flipper_length_mm AS flipper_length_mm,\n t2.body_mass_g AS body_mass_g,\n t2.sex AS sex\n FROM main._ibis_examples_penguins_zkpeihb5b5aw7al5hilxgkenfe AS t2\n), t1 AS (\n SELECT\n t0.species AS species,\n t0.island AS island,\n t0.bill_length_mm AS bill_length_mm,\n t0.bill_depth_mm AS bill_depth_mm,\n t0.flipper_length_mm AS flipper_length_mm,\n t0.body_mass_g AS body_mass_g,\n t0.sex AS sex,\n (\n t0.bill_length_mm - AVG(t0.bill_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.bill_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS bill_length_mm_zscore,\n (\n t0.bill_depth_mm - AVG(t0.bill_depth_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.bill_depth_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS bill_depth_mm_zscore,\n (\n t0.flipper_length_mm - AVG(t0.flipper_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.flipper_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS flipper_length_mm_zscore,\n (\n t0.body_mass_g - AVG(t0.body_mass_g) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.body_mass_g) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS body_mass_g_zscore\n FROM t0\n)\nSELECT\n t1.species,\n t1.island,\n t1.bill_length_mm,\n t1.bill_depth_mm,\n t1.flipper_length_mm,\n t1.body_mass_g,\n t1.sex,\n t1.bill_length_mm_zscore,\n t1.bill_depth_mm_zscore,\n t1.flipper_length_mm_zscore,\n t1.body_mass_g_zscore\nFROM t1\nWHERE\n ABS(t1.bill_length_mm_zscore) > CAST(2 AS TINYINT)\n AND ABS(t1.bill_depth_mm_zscore) > CAST(2 AS TINYINT)\n```\n:::\n:::\n\n\nGood thing you didn't have to write that by hand!\n\n## Summary\n\nThis blog post illustrates the ability to apply computations to many columns at\nonce and the power of ibis as a composable, expressive library for analytics.\n\n- [Get involved!](https://ibis-project.org/community/contribute/)\n- [Report issues!](https://github.com/ibis-project/ibis/issues/new/choose)\n\n", - "supporting": ["index_files"], + "markdown": "---\ntitle: \"Maximizing productivity with selectors\"\nauthor: Phillip Cloud\ndate: 2023-02-27\ncategories:\n - blog\n - new feature\n - productivity\n - duckdb\n---\n\nBefore Ibis 5.0 it's been challenging to concisely express whole-table\noperations with ibis. Happily this is no longer the case in ibis 5.0.\n\nLet's jump right in!\n\nWe'll look at selectors examples using the [`palmerpenguins` data\nset](https://allisonhorst.github.io/palmerpenguins/) with the [DuckDB\nbackend](https://ibis-project.org/backends/DuckDB/).\n\n## Setup\n\n::: {#bc7b7606 .cell execution_count=1}\n``` {.python .cell-code}\nfrom ibis.interactive import *\n\nt = ex.penguins.fetch()\nt\n```\n\n::: {.cell-output .cell-output-display execution_count=1}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ Adelie  │ Torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ Adelie  │ Torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ Adelie  │ Torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ Adelie  │ Torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ Adelie  │ Torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n## Examples\n\n### Normalization\n\nLet's say you want to compute the\n[z-score](https://en.wikipedia.org/wiki/Standard_score) of every numeric column\nand replace the existing data with that normalized value. Here's how you'd do\nthat with selectors:\n\n::: {#475b3449 .cell execution_count=2}\n``` {.python .cell-code}\nt.mutate(s.across(s.numeric(), (_ - _.mean()) / _.std()))\n```\n\n::: {.cell-output .cell-output-display execution_count=2}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year      ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ float64   │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────────┤\n│ Adelie  │ Torgersen │      -0.883205 │      0.784300 │         -1.416272 │   -0.563317 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.809939 │      0.126003 │         -1.060696 │   -0.500969 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.663408 │      0.429833 │         -0.420660 │   -1.186793 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │ -1.257484 │\n│ Adelie  │ Torgersen │      -1.322799 │      1.088129 │         -0.562890 │   -0.937403 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.846572 │      1.746426 │         -0.776236 │   -0.688012 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.919837 │      0.328556 │         -1.416272 │   -0.719186 │ female │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.864888 │      1.240044 │         -0.420660 │    0.590115 │ male   │ -1.257484 │\n│ Adelie  │ Torgersen │      -1.799025 │      0.480471 │         -0.562890 │   -0.906229 │ NULL   │ -1.257484 │\n│ Adelie  │ Torgersen │      -0.352029 │      1.543873 │         -0.776236 │    0.060160 │ NULL   │ -1.257484 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │         … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────────┘\n
\n```\n:::\n:::\n\n\n### What's Up With the `year` Column?\n\nWhoops, looks like we included `year` in our normalization because it's an\n`int64` column (and therefore numeric) but normalizing the year doesn't make\nsense.\n\nWe can exclude `year` from the normalization using another selector:\n\n::: {#01f0665f .cell execution_count=3}\n``` {.python .cell-code}\nt.mutate(s.across(s.numeric() & ~s.c(\"year\"), (_ - _.mean()) / _.std()))\n```\n\n::: {.cell-output .cell-output-display execution_count=3}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │      -0.883205 │      0.784300 │         -1.416272 │   -0.563317 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -0.809939 │      0.126003 │         -1.060696 │   -0.500969 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.663408 │      0.429833 │         -0.420660 │   -1.186793 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      -1.322799 │      1.088129 │         -0.562890 │   -0.937403 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.846572 │      1.746426 │         -0.776236 │   -0.688012 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -0.919837 │      0.328556 │         -1.416272 │   -0.719186 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.864888 │      1.240044 │         -0.420660 │    0.590115 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -1.799025 │      0.480471 │         -0.562890 │   -0.906229 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      -0.352029 │      1.543873 │         -0.776236 │    0.060160 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n`c` is short for \"column\" and the `~` means \"negate\". Combining those we get \"not the year column\"!\n\nPretty neat right?\n\n### Composable Group By\n\nThe power of this approach comes in when you want the grouped version. Perhaps\nwe think some of these columns vary by species.\n\nWith selectors, all you need to do is slap a `.group_by(\"species\")` onto `t`:\n\n::: {#dc862f0a .cell execution_count=4}\n``` {.python .cell-code}\nt.group_by(\"species\").mutate(\n s.across(s.numeric() & ~s.c(\"year\"), (_ - _.mean()) / _.std())\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=4}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │       0.791697 │     -1.270997 │          0.160007 │   -0.001444 │ female │  2008 │\n│ Adelie  │ Biscoe    │       1.467524 │     -0.038103 │          0.924596 │    0.816322 │ male   │  2009 │\n│ Adelie  │ Dream     │       0.378692 │      0.619441 │         -0.910418 │    2.070231 │ male   │  2007 │\n│ Adelie  │ Dream     │      -0.860324 │     -0.284681 │         -1.216254 │   -1.200835 │ female │  2007 │\n│ Adelie  │ Torgersen │      -0.785232 │      0.783827 │          0.465843 │   -0.546622 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.190962 │      1.852335 │          0.007089 │   -0.110480 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.040778 │     -0.449067 │         -1.369172 │   -0.164997 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.153416 │      1.030405 │          0.771678 │    2.124749 │ male   │  2007 │\n│ Adelie  │ Torgersen │      -1.761426 │     -0.202489 │          0.465843 │   -0.492104 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │       1.204702 │      1.523563 │          0.007089 │    1.197947 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nSince ibis translates this into a run-of-the-mill selection as if you had\ncalled `select` or `mutate` without selectors, nothing special is needed for a\nbackend to work with these new constructs.\n\nLet's look at some more examples.\n\n### Min-max Normalization\n\nGrouped min/max normalization? Easy:\n\n::: {#da876bd8 .cell execution_count=5}\n``` {.python .cell-code}\nt.group_by(\"species\").mutate(\n s.across(s.numeric() & ~s.c(\"year\"), (_ - _.min()) / (_.max() - _.min()))\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=5}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ float64           │ float64     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │       0.633094 │      0.216667 │          0.500000 │    0.441558 │ female │  2008 │\n│ Adelie  │ Biscoe    │       0.762590 │      0.466667 │          0.631579 │    0.636364 │ male   │  2009 │\n│ Adelie  │ Dream     │       0.553957 │      0.600000 │          0.315789 │    0.935065 │ male   │  2007 │\n│ Adelie  │ Dream     │       0.316547 │      0.416667 │          0.263158 │    0.155844 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.330935 │      0.633333 │          0.552632 │    0.311688 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.517986 │      0.850000 │          0.473684 │    0.415584 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.489209 │      0.383333 │          0.236842 │    0.402597 │ female │  2007 │\n│ Adelie  │ Torgersen │       0.510791 │      0.683333 │          0.605263 │    0.948052 │ male   │  2007 │\n│ Adelie  │ Torgersen │       0.143885 │      0.433333 │          0.552632 │    0.324675 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │       0.712230 │      0.783333 │          0.473684 │    0.727273 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n### Casting and Munging\n\nHow about casting every column whose name ends with any of the strings `\"mm\"`\nor `\"g\"` to a `float32`? No problem!\n\n::: {#c48e4798 .cell execution_count=6}\n``` {.python .cell-code}\nt.mutate(s.across(s.endswith((\"mm\", \"g\")), _.cast(\"float32\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=6}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float32        │ float32       │ float32           │ float32     │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ Adelie  │ Torgersen │      39.099998 │     18.700001 │             181.0 │      3750.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      39.500000 │     17.400000 │             186.0 │      3800.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      40.299999 │     18.000000 │             195.0 │      3250.0 │ female │  2007 │\n│ Adelie  │ Torgersen │            nan │           nan │               nan │         nan │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      36.700001 │     19.299999 │             193.0 │      3450.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      39.299999 │     20.600000 │             190.0 │      3650.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      38.900002 │     17.799999 │             181.0 │      3625.0 │ female │  2007 │\n│ Adelie  │ Torgersen │      39.200001 │     19.600000 │             195.0 │      4675.0 │ male   │  2007 │\n│ Adelie  │ Torgersen │      34.099998 │     18.100000 │             193.0 │      3475.0 │ NULL   │  2007 │\n│ Adelie  │ Torgersen │      42.000000 │     20.200001 │             190.0 │      4250.0 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\nWe can make all string columns have the same case too!\n\n::: {#46d8930e .cell execution_count=7}\n``` {.python .cell-code}\nt.mutate(s.across(s.of_type(\"string\"), _.lower()))\n```\n\n::: {.cell-output .cell-output-display execution_count=7}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ year  ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ int64 │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────┤\n│ adelie  │ torgersen │           39.1 │          18.7 │               181 │        3750 │ male   │  2007 │\n│ adelie  │ torgersen │           39.5 │          17.4 │               186 │        3800 │ female │  2007 │\n│ adelie  │ torgersen │           40.3 │          18.0 │               195 │        3250 │ female │  2007 │\n│ adelie  │ torgersen │            nan │           nan │              NULL │        NULL │ NULL   │  2007 │\n│ adelie  │ torgersen │           36.7 │          19.3 │               193 │        3450 │ female │  2007 │\n│ adelie  │ torgersen │           39.3 │          20.6 │               190 │        3650 │ male   │  2007 │\n│ adelie  │ torgersen │           38.9 │          17.8 │               181 │        3625 │ female │  2007 │\n│ adelie  │ torgersen │           39.2 │          19.6 │               195 │        4675 │ male   │  2007 │\n│ adelie  │ torgersen │           34.1 │          18.1 │               193 │        3475 │ NULL   │  2007 │\n│ adelie  │ torgersen │           42.0 │          20.2 │               190 │        4250 │ NULL   │  2007 │\n│ …       │ …         │              … │             … │                 … │           … │ …      │     … │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────┘\n
\n```\n:::\n:::\n\n\n### Multiple Computations per Column\n\nWhat if I want to compute multiple things? Heck yeah!\n\n::: {#f1208079 .cell execution_count=8}\n``` {.python .cell-code}\nt.group_by(\"sex\").mutate(\n s.across(\n s.numeric() & ~s.c(\"year\"),\n dict(centered=_ - _.mean(), zscore=(_ - _.mean()) / _.std()),\n )\n).select(\"sex\", s.endswith((\"_centered\", \"_zscore\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=8}\n```{=html}\n
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ sex    ┃ bill_length_mm_centered ┃ bill_depth_mm_centered ┃ flipper_length_mm_centered ┃ body_mass_g_centered ┃ bill_length_mm_zscore ┃ bill_depth_mm_zscore ┃ flipper_length_mm_zscore ┃ body_mass_g_zscore ┃\n┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string │ float64                 │ float64                │ float64                    │ float64              │ float64               │ float64              │ float64                  │ float64            │\n├────────┼─────────────────────────┼────────────────────────┼────────────────────────────┼──────────────────────┼───────────────────────┼──────────────────────┼──────────────────────────┼────────────────────┤\n│ female │                 4.10303 │              -1.925455 │                  11.636364 │           937.727273 │              0.836760 │            -1.072270 │                 0.930851 │           1.407635 │\n│ female │                 1.20303 │              -2.425455 │                  10.636364 │           712.727273 │              0.245342 │            -1.350716 │                 0.850856 │           1.069885 │\n│ female │                -8.09697 │               0.674545 │                 -12.363636 │          -462.272727 │             -1.651271 │             0.375649 │                -0.989030 │          -0.693924 │\n│ female │                -5.89697 │               0.874545 │                 -10.363636 │          -562.272727 │             -1.202610 │             0.487027 │                -0.829039 │          -0.844035 │\n│ female │                -0.99697 │               1.174545 │                 -15.363636 │          -662.272727 │             -0.203319 │             0.654095 │                -1.229015 │          -0.994147 │\n│ female │                -5.49697 │               1.374545 │                 -12.363636 │          -162.272727 │             -1.121035 │             0.765473 │                -0.989030 │          -0.243590 │\n│ female │                -3.39697 │               2.574545 │                  -2.363636 │          -412.272727 │             -0.692768 │             1.433743 │                -0.189079 │          -0.618868 │\n│ female │                -7.69697 │               1.974545 │                 -13.363636 │          -537.272727 │             -1.569697 │             1.099608 │                -1.069025 │          -0.806507 │\n│ female │                -4.29697 │               1.874545 │                 -23.363636 │          -462.272727 │             -0.876311 │             1.043919 │                -1.868975 │          -0.693924 │\n│ female │                -6.19697 │               2.774545 │                  -8.363636 │           -62.272727 │             -1.263791 │             1.545122 │                -0.669049 │          -0.093478 │\n│ …      │                       … │                      … │                          … │                    … │                     … │                    … │                        … │                  … │\n└────────┴─────────────────────────┴────────────────────────┴────────────────────────────┴──────────────────────┴───────────────────────┴──────────────────────┴──────────────────────────┴────────────────────┘\n
\n```\n:::\n:::\n\n\nDon't like the naming convention?\n\nPass a function to make your own name!\n\n::: {#ca542835 .cell execution_count=9}\n``` {.python .cell-code}\nt.select(s.startswith(\"bill\")).mutate(\n s.across(\n s.all(),\n dict(x=_ - _.mean(), y=_.max()),\n names=lambda col, fn: f\"{col}_{fn}_improved\",\n )\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=9}\n```{=html}\n
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ bill_length_mm ┃ bill_depth_mm ┃ bill_length_mm_x_improved ┃ bill_depth_mm_x_improved ┃ bill_length_mm_y_improved ┃ bill_depth_mm_y_improved ┃\n┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n│ float64        │ float64       │ float64                   │ float64                  │ float64                   │ float64                  │\n├────────────────┼───────────────┼───────────────────────────┼──────────────────────────┼───────────────────────────┼──────────────────────────┤\n│           39.1 │          18.7 │                  -4.82193 │                  1.54883 │                      59.6 │                     21.5 │\n│           39.5 │          17.4 │                  -4.42193 │                  0.24883 │                      59.6 │                     21.5 │\n│           40.3 │          18.0 │                  -3.62193 │                  0.84883 │                      59.6 │                     21.5 │\n│            nan │           nan │                       nan │                      nan │                      59.6 │                     21.5 │\n│           36.7 │          19.3 │                  -7.22193 │                  2.14883 │                      59.6 │                     21.5 │\n│           39.3 │          20.6 │                  -4.62193 │                  3.44883 │                      59.6 │                     21.5 │\n│           38.9 │          17.8 │                  -5.02193 │                  0.64883 │                      59.6 │                     21.5 │\n│           39.2 │          19.6 │                  -4.72193 │                  2.44883 │                      59.6 │                     21.5 │\n│           34.1 │          18.1 │                  -9.82193 │                  0.94883 │                      59.6 │                     21.5 │\n│           42.0 │          20.2 │                  -1.92193 │                  3.04883 │                      59.6 │                     21.5 │\n│              … │             … │                         … │                        … │                         … │                        … │\n└────────────────┴───────────────┴───────────────────────────┴──────────────────────────┴───────────────────────────┴──────────────────────────┘\n
\n```\n:::\n:::\n\n\nDon't like lambda functions? We support a format string too!\n\n::: {#aeedd50b .cell execution_count=10}\n``` {.python .cell-code}\nt.select(s.startswith(\"bill\")).mutate(\n s.across(\n s.all(),\n func=dict(x=_ - _.mean(), y=_.max()),\n names=\"{col}_{fn}_improved\",\n )\n).head(2)\n```\n\n::: {.cell-output .cell-output-display execution_count=10}\n```{=html}\n
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ bill_length_mm ┃ bill_depth_mm ┃ bill_length_mm_x_improved ┃ bill_depth_mm_x_improved ┃ bill_length_mm_y_improved ┃ bill_depth_mm_y_improved ┃\n┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n│ float64        │ float64       │ float64                   │ float64                  │ float64                   │ float64                  │\n├────────────────┼───────────────┼───────────────────────────┼──────────────────────────┼───────────────────────────┼──────────────────────────┤\n│           39.1 │          18.7 │                  -4.82193 │                  1.54883 │                      59.6 │                     21.5 │\n│           39.5 │          17.4 │                  -4.42193 │                  0.24883 │                      59.6 │                     21.5 │\n└────────────────┴───────────────┴───────────────────────────┴──────────────────────────┴───────────────────────────┴──────────────────────────┘\n
\n```\n:::\n:::\n\n\n### Working with other Ibis APIs\n\nWe've seen lots of mutate use, but selectors also work with `.agg`:\n\n::: {#6b1950e6 .cell execution_count=11}\n``` {.python .cell-code}\nt.group_by(\"year\").agg(s.across(s.numeric() & ~s.c(\"year\"), _.mean())).order_by(\"year\")\n```\n\n::: {.cell-output .cell-output-display execution_count=11}\n```{=html}\n
┏━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓\n┃ year  ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃\n┡━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩\n│ int64 │ float64        │ float64       │ float64           │ float64     │\n├───────┼────────────────┼───────────────┼───────────────────┼─────────────┤\n│  2007 │      43.740367 │     17.427523 │        196.880734 │ 4124.541284 │\n│  2008 │      43.541228 │     16.914035 │        202.798246 │ 4266.666667 │\n│  2009 │      44.452941 │     17.125210 │        202.806723 │ 4210.294118 │\n└───────┴────────────────┴───────────────┴───────────────────┴─────────────┘\n
\n```\n:::\n:::\n\n\nNaturally, selectors work in grouping keys too, for even more convenience:\n\n::: {#da973a3f .cell execution_count=12}\n``` {.python .cell-code}\nt.group_by(~s.numeric() | s.c(\"year\")).mutate(\n s.across(s.numeric() & ~s.c(\"year\"), dict(centered=_ - _.mean(), std=_.std()))\n).select(\"species\", s.endswith((\"_centered\", \"_std\")))\n```\n\n::: {.cell-output .cell-output-display execution_count=12}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n┃ species ┃ bill_length_mm_centered ┃ bill_depth_mm_centered ┃ flipper_length_mm_centered ┃ body_mass_g_centered ┃ bill_length_mm_std ┃ bill_depth_mm_std ┃ flipper_length_mm_std ┃ body_mass_g_std ┃\n┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n│ string  │ float64                 │ float64                │ float64                    │ float64              │ float64            │ float64           │ float64               │ float64         │\n├─────────┼─────────────────────────┼────────────────────────┼────────────────────────────┼──────────────────────┼────────────────────┼───────────────────┼───────────────────────┼─────────────────┤\n│ Adelie  │                   -1.46 │               0.400000 │                  -1.600000 │          -170.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                   -0.96 │              -0.200000 │                   3.400000 │           180.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                   -0.36 │              -1.100000 │                  -1.600000 │            30.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                    1.44 │               0.300000 │                   1.400000 │          -220.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Adelie  │                    1.34 │               0.600000 │                  -1.600000 │           180.000000 │           1.327780 │          0.681909 │              2.302173 │      189.076704 │\n│ Gentoo  │                    1.00 │               0.935294 │                  11.117647 │           147.058824 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                    1.00 │              -0.164706 │                  -0.882353 │           147.058824 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -1.40 │              -0.864706 │                  -3.882353 │          -152.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -2.30 │              -0.064706 │                   0.117647 │          -352.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ Gentoo  │                   -2.20 │               0.035294 │                  -3.882353 │          -402.941176 │           3.056755 │          0.670766 │              4.973459 │      349.763576 │\n│ …       │                       … │                      … │                          … │                    … │                  … │                 … │                     … │               … │\n└─────────┴─────────────────────────┴────────────────────────┴────────────────────────────┴──────────────────────┴────────────────────┴───────────────────┴───────────────────────┴─────────────────┘\n
\n```\n:::\n:::\n\n\n### Filtering Selectors\n\nYou can also express complex filters more concisely.\n\nLet's say we only want to keep rows where all the bill size z-score related\ncolumns' absolute values are greater than 2.\n\n::: {#a9bc15e3 .cell execution_count=13}\n``` {.python .cell-code}\nt.drop(\"year\").group_by(\"species\").mutate(\n s.across(s.numeric(), dict(zscore=(_ - _.mean()) / _.std()))\n).filter(s.if_all(s.startswith(\"bill\") & s.endswith(\"_zscore\"), _.abs() > 2))\n```\n\n::: {.cell-output .cell-output-display execution_count=13}\n```{=html}\n
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓\n┃ species ┃ island    ┃ bill_length_mm ┃ bill_depth_mm ┃ flipper_length_mm ┃ body_mass_g ┃ sex    ┃ bill_length_mm_zscore ┃ bill_depth_mm_zscore ┃ flipper_length_mm_zscore ┃ body_mass_g_zscore ┃\n┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩\n│ string  │ string    │ float64        │ float64       │ int64             │ int64       │ string │ float64               │ float64              │ float64                  │ float64            │\n├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼────────┼───────────────────────┼──────────────────────┼──────────────────────────┼────────────────────┤\n│ Adelie  │ Torgersen │           46.0 │          21.5 │               194 │        4200 │ male   │              2.706539 │             2.592071 │                 0.618760 │           1.088911 │\n│ Adelie  │ Dream     │           32.1 │          15.5 │               188 │        3050 │ female │             -2.512345 │            -2.339505 │                -0.298747 │          -1.418906 │\n│ Gentoo  │ Biscoe    │           55.9 │          17.0 │               228 │        5600 │ male   │              2.724046 │             2.056508 │                 1.667394 │           1.039411 │\n│ Gentoo  │ Biscoe    │           59.6 │          17.0 │               230 │        6050 │ male   │              3.924621 │             2.056508 │                 1.975799 │           1.932062 │\n└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴────────┴───────────────────────┴──────────────────────┴──────────────────────────┴────────────────────┘\n
\n```\n:::\n:::\n\n\n### Bonus: Generated SQL\n\nThe SQL for that last expression is pretty gnarly:\n\n::: {#fdd76c58 .cell execution_count=14}\n``` {.python .cell-code}\nibis.to_sql(\n t.drop(\"year\")\n .group_by(\"species\")\n .mutate(s.across(s.numeric(), dict(zscore=(_ - _.mean()) / _.std())))\n .filter(s.if_all(s.startswith(\"bill\") & s.endswith(\"_zscore\"), _.abs() > 2))\n)\n```\n\n::: {.cell-output .cell-output-display execution_count=14}\n```sql\nWITH t0 AS (\n SELECT\n t2.species AS species,\n t2.island AS island,\n t2.bill_length_mm AS bill_length_mm,\n t2.bill_depth_mm AS bill_depth_mm,\n t2.flipper_length_mm AS flipper_length_mm,\n t2.body_mass_g AS body_mass_g,\n t2.sex AS sex\n FROM main._ibis_examples_penguins_wyios4y4nbfd5oqs53lzomk2ee AS t2\n), t1 AS (\n SELECT\n t0.species AS species,\n t0.island AS island,\n t0.bill_length_mm AS bill_length_mm,\n t0.bill_depth_mm AS bill_depth_mm,\n t0.flipper_length_mm AS flipper_length_mm,\n t0.body_mass_g AS body_mass_g,\n t0.sex AS sex,\n (\n t0.bill_length_mm - AVG(t0.bill_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.bill_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS bill_length_mm_zscore,\n (\n t0.bill_depth_mm - AVG(t0.bill_depth_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.bill_depth_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS bill_depth_mm_zscore,\n (\n t0.flipper_length_mm - AVG(t0.flipper_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.flipper_length_mm) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS flipper_length_mm_zscore,\n (\n t0.body_mass_g - AVG(t0.body_mass_g) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)\n ) / STDDEV_SAMP(t0.body_mass_g) OVER (PARTITION BY t0.species ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS body_mass_g_zscore\n FROM t0\n)\nSELECT\n t1.species,\n t1.island,\n t1.bill_length_mm,\n t1.bill_depth_mm,\n t1.flipper_length_mm,\n t1.body_mass_g,\n t1.sex,\n t1.bill_length_mm_zscore,\n t1.bill_depth_mm_zscore,\n t1.flipper_length_mm_zscore,\n t1.body_mass_g_zscore\nFROM t1\nWHERE\n ABS(t1.bill_length_mm_zscore) > CAST(2 AS TINYINT)\n AND ABS(t1.bill_depth_mm_zscore) > CAST(2 AS TINYINT)\n```\n:::\n:::\n\n\nGood thing you didn't have to write that by hand!\n\n## Summary\n\nThis blog post illustrates the ability to apply computations to many columns at\nonce and the power of ibis as a composable, expressive library for analytics.\n\n- [Get involved!](https://ibis-project.org/community/contribute/)\n- [Report issues!](https://github.com/ibis-project/ibis/issues/new/choose)\n\n", + "supporting": [ + "index_files" + ], "filters": [], "includes": { "include-in-header": [ diff --git a/docs/posts/campaign-finance/index.qmd b/docs/posts/campaign-finance/index.qmd index 6c0d46916940..35558b9566fe 100644 --- a/docs/posts/campaign-finance/index.qmd +++ b/docs/posts/campaign-finance/index.qmd @@ -4,6 +4,10 @@ author: "Nick Crews" date: "2023-03-24" categories: - blog + - data engineering + - case study + - duckdb + - performance --- Hi! My name is [Nick Crews](https://www.linkedin.com/in/nicholas-b-crews/), and I'm a data engineer that looks at public campaign finance data. diff --git a/docs/posts/ci-analysis/index.qmd b/docs/posts/ci-analysis/index.qmd index 7c734185fe4c..271161cecaa1 100644 --- a/docs/posts/ci-analysis/index.qmd +++ b/docs/posts/ci-analysis/index.qmd @@ -4,6 +4,10 @@ author: "Phillip Cloud" date: "2023-01-09" categories: - blog + - bigquery + - continuous integration + - data engineering + - dogfood --- ## Summary diff --git a/docs/posts/ffill-and-bfill-using-ibis/index.qmd b/docs/posts/ffill-and-bfill-using-ibis/index.qmd index b25e52eddbd3..c623cfea4e80 100644 --- a/docs/posts/ffill-and-bfill-using-ibis/index.qmd +++ b/docs/posts/ffill-and-bfill-using-ibis/index.qmd @@ -4,6 +4,8 @@ author: Patrick Clarke date: 2022-09-09 categories: - blog + - window functions + - time series --- Suppose you have a table of data mapping events and dates to values, and that this data contains gaps in values. diff --git a/docs/posts/ibis-analytics/index.qmd b/docs/posts/ibis-analytics/index.qmd index d110064b0ed7..5cf5d1754b78 100644 --- a/docs/posts/ibis-analytics/index.qmd +++ b/docs/posts/ibis-analytics/index.qmd @@ -6,7 +6,9 @@ image: "thumbnail.png" code-annotations: below categories: - blog - - demo + - duckdb + - case study + - dogfood draft: true --- diff --git a/docs/posts/ibis-examples/index.qmd b/docs/posts/ibis-examples/index.qmd index cdb574dcef2a..f27d54bbbfb1 100644 --- a/docs/posts/ibis-examples/index.qmd +++ b/docs/posts/ibis-examples/index.qmd @@ -4,6 +4,8 @@ author: Kae Suarez date: 2023-03-08 categories: - blog + - new feature + - sneak peek --- Ibis has been moving quickly to provide a powerful but easy-to-use interface for interacting with analytical engines. However, as we’re approaching the 5.0 release of Ibis, we’ve realized that moving from not knowing Ibis to writing a first expression is not trivial. diff --git a/docs/posts/ibis-to-file/index.qmd b/docs/posts/ibis-to-file/index.qmd index 71346c540960..5737e0ebf7e2 100644 --- a/docs/posts/ibis-to-file/index.qmd +++ b/docs/posts/ibis-to-file/index.qmd @@ -4,6 +4,9 @@ author: Kae Suarez date: 2023-03-09 categories: - blog + - io + - new feature + - sneak peek --- Ibis 5.0 is coming soon and will offer new functionality and fixes to users. To enhance clarity around this process, we’re sharing a sneak peek into what we’re working on. diff --git a/docs/posts/ibis_substrait_to_duckdb/index.qmd b/docs/posts/ibis_substrait_to_duckdb/index.qmd index 7b1fac3522bd..fcd8d2fd71bf 100644 --- a/docs/posts/ibis_substrait_to_duckdb/index.qmd +++ b/docs/posts/ibis_substrait_to_duckdb/index.qmd @@ -4,6 +4,9 @@ author: Gil Forsyth date: 2023-02-01 categories: - blog + - substrait + - ecosystem + - duckdb --- Ibis strives to provide a consistent interface for interacting with a multitude diff --git a/docs/posts/selectors/index.qmd b/docs/posts/selectors/index.qmd index bfae465b8061..6e57967315ab 100644 --- a/docs/posts/selectors/index.qmd +++ b/docs/posts/selectors/index.qmd @@ -4,6 +4,9 @@ author: Phillip Cloud date: 2023-02-27 categories: - blog + - new feature + - productivity + - duckdb --- Before Ibis 5.0 it's been challenging to concisely express whole-table diff --git a/docs/posts/torch/index.qmd b/docs/posts/torch/index.qmd index 2e706028b47f..ba23299d4e8c 100644 --- a/docs/posts/torch/index.qmd +++ b/docs/posts/torch/index.qmd @@ -4,6 +4,10 @@ author: "Phillip Cloud" date: "2023-06-27" categories: - blog + - case study + - machine learning + - ecosystem + - new feature --- In this blog post we show how to leverage ecosystem tools to build an end-to-end ML pipeline using Ibis, DuckDB and PyTorch.