Join the Kedro community

A
M
M
M
D
Members
Guillaume Tauzin
G
Guillaume Tauzin
Offline, last seen 3 hours ago
Joined September 13, 2024

Hello Team!
So it's been a few months since we started using kedro and it's time to deploy some of the pipelines we have created.
We need to choose an orchestrator but this is not our field of expertise, so I wanted to ask for some help. We would like something simple to setup and use collaboratively. Also my company requires it is free (at least for now), our cloud provider is AWS and we already use mlflow. Here are the alternatives we found:

  • Prefect (open-source, seems nice to use, kedro support, but free tier imposes limitations)
  • Flyte (free?, open-source, seems nice to use, no kedro support)
  • MLRun (free and open-source, no kedro support? seems nice to use but a bit more than an orchestrator, requires python 3.9)
  • Kubeflow Pipelines (free and open-source, kedro plugin, and others seem to think it is complex to setup and maintain)
  • Airflow (free and open-source, kedro plugin)
  • Sagemaker (Amazon, kedro plugin, personally dislike its UI and how other AWS services are organized around it)

What would you recommend? What should we consider to make such a decision?

Thanks for your help :)

2 comments
J
G