Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Elyra extends JupyterLab with an AI centric approach.

A series of DAGs/Workflows to help maintain the operation of Airflow

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Dynamically generate Apache Airflow DAGs from YAML configuration files

Example end to end data engineering project.

A Data Engineering & Machine Learning Knowledge Hub

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

Apache Airflow integration for dbt

Turbine: the bare metals that gets you Airflow

Apache Airflow - OpenApi Client for Python

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Example DAGs using hooks and operators from Airflow Plugins

Auto-generated Diagrams from Airflow DAGs. 🔮 🪄

A plugin for Apache Airflow that exposes rest end points for the Command Line Interfaces

Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)