It definitely feels like there is Airflow inspiration though poking through the site and docs, it seems that the devil is in the details with respect to the differences between the two... kinda like the relationship between Airflow and Prefect [1]. It looks like the barrier to entry is higher with Flyte but that there are benefits as well.
One example I see is that one can run Airflow simply without containers if desired with just simple Python functions whereas Flyte seems to be much more concerned with managing the execution environment for you (pros/cons to that).
Flyte also seems to be more "Kubernetes native" by default [2][3] vs with Airflow this is more of a choice amongst several executors.
I'd be curious to see a performance benchmark using comparable workflows vs Airflow with the Kubernetes Executor or Kubernetes Operator.
Thanks for the link! Looking at Prefect's comparison to Airflow (https://docs.prefect.io/core/welcome/why_not_airflow.html#wh...), I think it sums up a lot of the same areas we tried to address with Flyte! Particularly the bullet-points in the 'Overview' section were all things, to the word, that were paramount for us to solve at Lyft. Now perhaps we need to publish an article on comparison to Prefect ;)
One example I see is that one can run Airflow simply without containers if desired with just simple Python functions whereas Flyte seems to be much more concerned with managing the execution environment for you (pros/cons to that).
Flyte also seems to be more "Kubernetes native" by default [2][3] vs with Airflow this is more of a choice amongst several executors.
I'd be curious to see a performance benchmark using comparable workflows vs Airflow with the Kubernetes Executor or Kubernetes Operator.
[1]: https://www.prefect.io/
[2]: https://kccncna19.sched.com/event/UaYY/flyte-cloud-native-ma...
[3]: https://www.youtube.com/watch?v=KdUJGSP1h9U