🚀 Apache Airflow: From Scratch to Advanced

This repository is a step-by-step Airflow learning guide that covers everything from beginner concepts to advanced features and real-world data engineering pipelines.

---

📌 What’s Inside?

• ✅ Airflow Basics (DAGs, Scheduling, Catchup, Backfill)
• ✅ Operators (Python, Bash, Branch, Dummy, Custom)
• ✅ Sensors (FileSensor, ExternalTaskSensor)
• ✅ Advanced Concepts (XComs, TaskGroups, Dynamic DAGs, SLAs)
• ✅ Airflow with Docker & Local Executor
• ✅ Real-World Projects (ETL, API ingestion, Google Trends, Twitter Data Pipeline)

---

📂 Repository Structure

airflow-from-scratch-to-advanced/ │── basics/ # Core concepts & simple DAGs │ ├── hello_world_dag.py │ ├── catchup_example.py │ ├── backfill_example.py │ │── operators/ # Python, Bash, Branch, Dummy, etc. │ ├── python_operator.py │ ├── bash_operator.py │ │── sensors/ # FileSensor, ExternalTaskSensor, etc. │ │── advanced/ # XComs, TaskGroups, Dynamic DAGs │ ├── xcom_example.py │ ├── dynamic_dag.py │── README.md