Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Apply Data Engineering to Personal Finance

🔥🔥🔥 Open Source Alternative to Hightouch, Census, and RudderStack.

Privacy and Security focused Segment-alternative, in Golang and React

Flink CDC is a streaming data integration tool

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Aqueduct Core is responsible for the core functionality of Aqueduct, an experiment management system.

DDE Workflow Utility Objects

⛅ Versatile Data Pipeline (VDP) console website

Watchmen Platform is a low code data platform for data pipeline, meta data management , analysis, indicator objective analysis and quality management

Docs for Chalk AI

A repository for the Methods of Advanced Data Engineering course at FAU

This repository hosts materials for the Docker for Data Engineers workshop, offering hands-on exercises and resources tailored for data engineering professionals.

A convenience tool for small-scale data pipelines in Python

Jayvee is a domain-specific language and runtime for automated processing of data pipelines

Feldera Continuous Analytics Platform

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

Memphis.dev is a highly scalable and effortless data streaming platform

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.