I just started to learn Apache Airflow, doing some tests and yes, it's pretty cool, but this intro and considerations in terms of use cases, was eye-opening. The best video I found so far. Thanks for this, Bryan.
Hi, great and helpful video, is there a reliability strategy between the scheduler and the dag directory? It seems that's the signal point of the fail point. Thanks!!!
Hi guys! I am looking for Apache Aiflow study group. I have some intermediate knowledge on Kafka-KSQL, Spring-Batch, DataEngineering concepts etc. I really want to improove my skills.
I just started to learn Apache Airflow, doing some tests and yes, it's pretty cool, but this intro and considerations in terms of use cases, was eye-opening. The best video I found so far.
Thanks for this, Bryan.
You may want to look at Dagster too. I really like it. Very focused on data lineage.
The best intro regarding Airflow I found. Thank you...
I really appreciate the parsimony segment. Very helpful, thanks.
Hi, great and helpful video, is there a reliability strategy between the scheduler and the dag directory? It seems that's the signal point of the fail point. Thanks!!!
Not sure but I think the Airflow UI is where failures are expected to surfaced. But good question.
How do you feel about “Prefect”? They’re trying to replace Airflow.
A video on Prefect from you would be great too!
Do you need a WorkFlow engine?
@@BryanCafferky yes
How is DataBricks an alternative to Airflow? DataBricks isn't a workflow engine, is it?
Yes. Databricks has a workflow service that is powerful. See docs.databricks.com/en/jobs/index.html
would airflow be an alternative for ssis/ssdt?
Hi Nathan,
See my other vide to answer that.
Video: Don't Use Apache Airflow
ua-cam.com/video/YQ056EKzCyw/v-deo.html
Nice explanation
Hi guys! I am looking for Apache Aiflow study group. I have some intermediate knowledge on Kafka-KSQL, Spring-Batch, DataEngineering concepts etc. I really want to improove my skills.
Awesome!