Advancing Spark - Getting Started with MLFlow Pipelines

One of the big announcements coming from the Data + AI Summit was MLFlow pipelines, a new framework for building reliable, repeatable machine learning workflows. But what does that actually mean? How do you get started? Where does it fit into the existing ecosystem?
In this video, Simon is once again joined by principal data scientist Gavi to get the low down on the available MLFlow Pipeline template, walk through some example uses and see what it actually looks like!
The templates used can be found on the MLFlow site over at: www.mlflow.org...
And as always, if you need any help on your Lakehouse journey, give Advancing Analytics a call

[@RobertoMartin1]

Any examples on cross-validation; instead of using a dedicated validation set? As well as parameter search. I don’t see this as the best idea for model search. This is probably better used after you’ve discovered a good model, and you wanna retrain, validate and detect drift

[@CoopmanGreg]

Nice and clean. I like!😀 Great Presentation!

[@dangz90]

What wouldl be the best practices to incorporate both MLFlow Pipelines with a Feature Store? I was thinking that the ingestion section would rather call my Feature Tables.

[@xiangruimengdatabricks417]

We might add feature store abstraction to MLflow and then allow users to define feature joins/transforms declaratively in the transform step.