What is Prefect 2.0?
Prefect 2.0 is an open-source workflow management system designed to help data engineers and data scientists in automating data workflows. It features a modern API, a user-friendly interface, and advanced monitoring and debugging tools. Users can easily create, deploy, and manage complex data workflows across a variety of environments, including cloud and on-premises systems.
Part 1: Concept of The Decorators → Flow & Task
✨ Flows and tasks are the building blocks of workflows in Prefect 2.0
✨ A flow is a collection of tasks that represent a sequence of actions to be performed
✨ Tasks are individual units of work that can be composed into a flow
✨ Flows and tasks can be defined using Python code, making it easy to create complex workflows that integrate with a wide range of data sources and systems
Prefect Documentation
--> https://docs.prefect.io/tutorials/first-steps/
| No | Date | Update |
|---|---|---|
| 1 | 19/02/2023 | Update Prefect basic folder - Flow and task concept |
| 2 | 20/02/2023 | Update Prefect finance sample folder - Basic pipeline creation |
| 3 | 22/02/2023 | Update Prefect Nike sample folder - Scheduling |
| 4 | 24/02/2023 | Update Prefect GCP BQ query folder - Prefect 2.0 GCP Blocks and BigQuery |
| 5 | 24/02/2023 | Update Prefect Slack notification folder - Prefect 2.0 + Slack Notification via webhook |
| 6 | 25/02/2023 | Update Prefect Postgres data ingestion folder - Prefect 2.0 + ETL + Postgres SQL |