-
Notifications
You must be signed in to change notification settings - Fork 609
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## Description of changes add a quick Kedro blog post, linking out to the main article
- Loading branch information
1 parent
03973d2
commit 1ffe435
Showing
1 changed file
with
27 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
--- | ||
title: "Building scalable data pipelines with Kedro" | ||
author: "Cody" | ||
date: "2024-01-31" | ||
categories: | ||
- blog | ||
- kedro | ||
- data engineering | ||
--- | ||
|
||
# Overview | ||
|
||
[Kedro](https://kedro.org) is a toolbox for production-ready data science. It is | ||
an open-source Python framework like Ibis, and together you can bring the | ||
portability and scale of Ibis to the production-ready pipelines of Kedro. | ||
|
||
> In your ~~Kedro~~ data journey, have you ever... | ||
> | ||
> ...slurped up large amounts of data into memory, instead of pushing execution down to the source database/engine? | ||
> | ||
> ...prototyped a node in pandas, and then rewritten it in PySpark/Snowpark/some other native dataframe API? | ||
> | ||
> ...implemented a proof-of-concept solution in 3-4 months on data extracts, and then struggled massively when you needed to move to running against the production databases and scale out? | ||
> ... | ||
If so, [read the full article on the Kedro | ||
blog](https://kedro.org/blog/building-scalable-data-pipelines-with-kedro-and-ibis)! |