Skip to content

Commit e632fc8

Browse files
jgujerryjgujerry
committed
Update workflow & pipeline frameworks
1 parent 337a538 commit e632fc8

1 file changed

Lines changed: 41 additions & 41 deletions

File tree

‎index.html‎

Lines changed: 41 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -2494,7 +2494,7 @@ <h3 class="mt-4 display-6">Parallel and Distributed Computing</h3>
24942494
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
24952495
<div class="card">
24962496
<div class="card-body text-center">
2497-
<a href="https://www.ray.io/" target="_blank">
2497+
<a href="https://github.com/ray-project/ray" target="_blank">
24982498
<img class="card-img-top" src="static/image/logos/ray.png" alt="Ray">
24992499
</a>
25002500
<hr class="my-2">
@@ -2514,7 +2514,7 @@ <h3 class="mt-4 display-6">Parallel and Distributed Computing</h3>
25142514
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
25152515
<div class="card">
25162516
<div class="card-body">
2517-
<a href="https://www.dask.org/" target="_blank">
2517+
<a href="https://github.com/dask/dask" target="_blank">
25182518
<img class="card-img-top p-2" src="static/image/logos/dask.svg" style="background-color: #262326;" alt="Dask">
25192519
</a>
25202520
<p class="card-text mt-3">Parallel computing with task scheduling</p>
@@ -2530,7 +2530,7 @@ <h3 class="mt-4 display-6">Parallel and Distributed Computing</h3>
25302530
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
25312531
<div class="card text-center">
25322532
<div class="p-3 text-center" style="background-color: #297699;">
2533-
<a href="https://spark.apache.org/" target="_blank">
2533+
<a href="https://github.com/apache/spark/tree/master/python" target="_blank">
25342534
<img class="card-img-top" src="static/image/logos/spark.png" style="width: 70%; height: auto;" alt="ApacheSpark">
25352535
</a>
25362536
</div>
@@ -2546,7 +2546,7 @@ <h5 class="card-title"><a href="https://github.com/apache/spark/tree/master/pyth
25462546
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
25472547
<div class="card text-end">
25482548
<div class="p-3">
2549-
<a href="https://faust-streaming.github.io/faust/" target="_blank">
2549+
<a href="https://github.com/faust-streaming/faust" target="_blank">
25502550
<img class="card-img-top" src="static/image/logos/faust.png" alt="Faust">
25512551
</a>
25522552
</div>
@@ -2560,7 +2560,7 @@ <h5 class="card-title"><a href="https://github.com/apache/spark/tree/master/pyth
25602560
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
25612561
<div class="card">
25622562
<div class="card-header text-center">
2563-
<a href="https://streamparse.readthedocs.io/" target="_blank">
2563+
<a href="https://github.com/pystorm/streamparse" target="_blank">
25642564
<img class="card-img-top" src="static/image/logos/streamparse.png" style="width:100%;height:auto;" alt="Streamparse"/>
25652565
</a>
25662566
</div>
@@ -2590,7 +2590,7 @@ <h3 class="mt-4 display-6">Workflow and Pipelines</h3>
25902590

25912591
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
25922592
<div class="card">
2593-
<a href="https://airflow.apache.org/" target="_blank">
2593+
<a href="https://github.com/apache/airflow/" target="_blank">
25942594
<img class="card-img-top" src="static/image/logos/airflow.png" style="width: 100%; height: auto;" alt="Airflow" />
25952595
</a>
25962596
<div class="card-body">
@@ -2604,11 +2604,11 @@ <h3 class="mt-4 display-6">Workflow and Pipelines</h3>
26042604

26052605
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26062606
<div class="card text-center">
2607-
<a href="https://www.prefect.io/" target="_blank">
2607+
<a href="https://github.com/PrefectHQ/prefect" target="_blank">
26082608
<img class="card-img-top" src="static/image/logos/prefect.png" style="width: 100%; height: auto;" alt="Prefect" />
26092609
</a>
26102610
<div class="card-body">
2611-
<h5 class="card-title"><a href="https://www.prefect.io/" target="_blank">Prefect</a></h5>
2611+
<h5 class="card-title"><a href="https://github.com/PrefectHQ/prefect" target="_blank">Prefect</a></h5>
26122612
<p class="card-text"><small class="text-muted">Running Python functions as pipeline jobs</small></p>
26132613
<hr class="my-2">
26142614
<p class="card-text">Prefect is a workflow orchestration tool empowering developers to build, observe, and react to data pipelines</p>
@@ -2618,7 +2618,7 @@ <h5 class="card-title"><a href="https://www.prefect.io/" target="_blank">Prefect
26182618

26192619
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26202620
<div class="card p-3">
2621-
<a href="https://airbyte.com/" target="_blank">
2621+
<a href="https://github.com/airbytehq/airbyte" target="_blank">
26222622
<img class="card-img-top" src="static/image/logos/airbyte.svg" alt="Airbyte">
26232623
</a>
26242624
<figure class="p-3 mb-0 text-end">
@@ -2636,10 +2636,10 @@ <h5 class="card-title"><a href="https://www.prefect.io/" target="_blank">Prefect
26362636
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26372637
<div class="card">
26382638
<div class="card-body">
2639-
<a href="https://luigi.readthedocs.io/" target="_blank">
2639+
<a href="https://github.com/spotify/luigi" target="_blank">
26402640
<img class="card-img-top" src="static/image/logos/luigi.png" style="width: 70%; height: auto;" alt="Luigi">
26412641
</a>
2642-
<h4 class="card-title mt-3"><a href="https://luigi.readthedocs.io/" target="_blank">Luigi</a></h4>
2642+
<h4 class="card-title mt-3"><a href="https://github.com/spotify/luigi" target="_blank">Luigi</a></h4>
26432643
<hr class="my-2">
26442644
<p class="card-text">
26452645
Luigi is a Python package that helps you build complex pipelines of batch jobs.
@@ -2651,7 +2651,7 @@ <h4 class="card-title mt-3"><a href="https://luigi.readthedocs.io/" target="_bla
26512651
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26522652
<div class="card">
26532653
<div class="card-body">
2654-
<a href="https://www.getdbt.com/" target="_blank">
2654+
<a href="https://github.com/dbt-labs/dbt-core" target="_blank">
26552655
<img class="card-img-top p-2" src="static/image/logos/dbt.png" alt="dbt">
26562656
</a>
26572657
<p class="card-text mt-3">
@@ -2670,7 +2670,7 @@ <h4 class="card-title mt-3"><a href="https://luigi.readthedocs.io/" target="_bla
26702670

26712671
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26722672
<div class="card text-center">
2673-
<a href="https://www.mage.ai/" target="_blank">
2673+
<a href="https://github.com/mage-ai/mage-ai" target="_blank">
26742674
<img class="card-img-top" src="static/image/logos/mage-ai-mascots-shorter.jpeg" style="width:100%; height:auto" alt="Mage" />
26752675
</a>
26762676
<a href="https://github.com/mage-ai/mage-ai" target="_blank">
@@ -2679,14 +2679,14 @@ <h4 class="card-title mt-3"><a href="https://luigi.readthedocs.io/" target="_bla
26792679
<div class="card-body">
26802680
<p class="card-text"><small class="text-muted">Mage is an open-source, hybrid framework for transforming and integrating data.</small></p>
26812681
<hr class="my-2">
2682-
<p class="card-text text-start">The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.</p>
2682+
<p class="card-text text-start">The modern replacement for Airflow. Build modern data pipelines locally — fast, visual, and production-ready.</p>
26832683
</div>
26842684
</div>
26852685
</div>
26862686

26872687
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
26882688
<div class="card text-center">
2689-
<a href="https://joblib.readthedocs.io/" target="_blank">
2689+
<a href="https://github.com/joblib/joblib" target="_blank">
26902690
<img class="card-img-top" src="static/image/logos/joblib.svg" style="width:50%; height:auto" alt="joblib" />
26912691
</a>
26922692
<div class="card-body">
@@ -2699,25 +2699,7 @@ <h4 class="card-title mt-3"><a href="https://luigi.readthedocs.io/" target="_bla
26992699

27002700
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
27012701
<div class="card">
2702-
<div class="card-header">
2703-
<h2>
2704-
<a href="https://schedule.readthedocs.io/" target="_blank">
2705-
Schedule
2706-
</a>
2707-
</h2>
2708-
</div>
2709-
<div class="card-body">
2710-
<blockquote class="mb-0">
2711-
<p>Run Python functions (or any other callable) periodically using a friendly syntax.</p>
2712-
<footer class="blockquote-footer mt-2 text-end">Python job scheduling for humans.</footer>
2713-
</blockquote>
2714-
</div>
2715-
</div>
2716-
</div>
2717-
2718-
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
2719-
<div class="card">
2720-
<a href="https://mlflow.org/" target="_blank">
2702+
<a href="https://github.com/mlflow/mlflow" target="_blank">
27212703
<img class="card-img-top p-3" src="static/image/logos/mlflow.png" style="background-color: #092241; width:100%; height:auto;" alt="MLflow">
27222704
</a>
27232705
<div class="card-body">
@@ -2730,7 +2712,7 @@ <h2>
27302712
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
27312713
<div class="card">
27322714
<div class="card-body text-end">
2733-
<a href="https://spiffworkflow.readthedocs.io/" target="_blank">
2715+
<a href="https://github.com/sartography/SpiffWorkflow" target="_blank">
27342716
<img class="card-img-top p-1" src="static/image/logos/spiffworkflow.png" style="width:100%; height:auto;" alt="SpiffWorkflow">
27352717
</a>
27362718
<p class="card-text mt-3"><small>A powerful workflow engine implemented in pure Python.</small></p>
@@ -2742,7 +2724,7 @@ <h2>
27422724
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
27432725
<div class="card">
27442726
<div class="card-body text-center">
2745-
<a href="https://metaflow.org/" target="_blank">
2727+
<a href="https://github.com/Netflix/metaflow" target="_blank">
27462728
<img class="card-img-top" src="static/image/logos/metaflow.png" alt="Metaflow">
27472729
</a>
27482730
<hr class="my-2">
@@ -2759,7 +2741,7 @@ <h2>
27592741
<div class="card text-center">
27602742
<div class="card-header">
27612743
<h2>
2762-
<a href="https://botflow.readthedocs.io/en/latest/" target="_blank">
2744+
<a href="https://github.com/kkyon/botflow" target="_blank">
27632745
botflow
27642746
</a>
27652747
</h2>
@@ -2776,7 +2758,7 @@ <h2>
27762758
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
27772759
<div class="card">
27782760
<div class="card-body text-center">
2779-
<a href="https://kedro.org/" target="_blank">
2761+
<a href="https://github.com/kedro-org/kedro" target="_blank">
27802762
<img class="card-img-top" src="static/image/logos/kedro.webp" alt="Kedro">
27812763
</a>
27822764
<div style="background-color: #000;">
@@ -2795,7 +2777,7 @@ <h2>
27952777
<div class="card">
27962778
<div class="card-header" style="background-color:#2980B9;">
27972779
<h3 class="text-end">
2798-
<a style="color:#fff" href="https://apscheduler.readthedocs.io/" target="_blank">
2780+
<a style="color:#fff" href="https://github.com/agronholm/apscheduler" target="_blank">
27992781
APScheduler
28002782
</a>
28012783
</h3>
@@ -2811,7 +2793,7 @@ <h3 class="text-end">
28112793

28122794
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
28132795
<div class="card">
2814-
<a href="https://dagster.io/" target="_blank">
2796+
<a href="https://github.com/dagster-io/dagster" target="_blank">
28152797
<img class="card-img-top" src="static/image/logos/dagster.svg" style="width:100%; height:auto;" alt="Dagster">
28162798
</a>
28172799
<div class="card-body">
@@ -2827,7 +2809,7 @@ <h3 class="text-end">
28272809

28282810
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
28292811
<div class="card">
2830-
<a href="https://pydoit.org/" target="_blank">
2812+
<a href="https://github.com/pydoit/doit" target="_blank">
28312813
<img class="card-img-top p-3" src="static/image/logos/doit.png" style="width:100%; height:auto;" alt="Doit">
28322814
</a>
28332815
<div class="card-body">
@@ -2853,6 +2835,24 @@ <h3 class="text-end py-3">mara-pipelines</h3>
28532835
</div>
28542836
</div>
28552837

2838+
<div class="col-12 col-6 col-sm-6 col-lg-3 mb-4">
2839+
<div class="card">
2840+
<div class="card-header">
2841+
<h2>
2842+
<a href="https://github.com/dbader/schedule" target="_blank">
2843+
Schedule
2844+
</a>
2845+
</h2>
2846+
</div>
2847+
<div class="card-body">
2848+
<blockquote class="mb-0">
2849+
<p>Run Python functions (or any other callable) periodically using a friendly syntax.</p>
2850+
<footer class="blockquote-footer mt-2 text-end">Python job scheduling for humans.</footer>
2851+
</blockquote>
2852+
</div>
2853+
</div>
2854+
</div>
2855+
28562856
</div>
28572857
</div>
28582858
<!--End of Workflow and Pipelines-->

0 commit comments

Comments
 (0)