AWS re:Invent 2020: Building end-to-end ML workflows with Kubeflow Pipelines
Вставка
- Опубліковано 13 лип 2024
- Kubeflow is a popular open-source machine learning (ML) toolkit for Kubernetes users who want to build custom ML pipelines. Kubeflow Pipelines is an add-on to Kubeflow that lets you build and deploy portable and scalable end-to-end ML workflows. In this session, learn how to get started with Kubeflow Pipelines on AWS. See how you can integrate powerful Amazon SageMaker features such as data labeling, large-scale hyperparameter tuning, distributed training jobs, and secure and scalable model deployment using SageMaker Components for Kubeflow Pipelines.
Learn more about re:Invent 2020 at bit.ly/3c4NSdY
Subscribe:
More AWS videos bit.ly/2O3zS75
More AWS events videos bit.ly/316g9t4
#AWS #AWSEvents
This is the very good starting point to learn kubeflow
The best introduction to the kfp!
Amazing tutorial, thanks! I've got two questions.
1. @6:40 what are CreateClusterOp, AnalyzeOp, etc... ?
2. @12:28 "each components are individual docker" but aren't sum1(a+b), sum2(c+d), sum3 (sum1 + sum3) inside one python function in @dsl.pipeline generator? if not, @10:47 is creating 3 components instead of one which perform three addition?
but how to install kubeflow on aws, i tried, we dont have any good tutorial on internet!
did u got answer,
After creating a pipeline can we test it with different parameters w/o creating a new pipeline? for ex: @10:33 you've specified a,b,c,d but I'm wondering if I can try different a,b,c,d with one pipeline.
The "Experiement" funtionality is exactly what serves this purpose. You can run multiple iterations with different parameters under one Experiment ID. Also you may select what metrics needs to be tracked in each of the iteration.
a bit hard to follow with the audio out of sync with the presentation
Appreciate if we can get access to the jupyter notebook you using here.. Thnks
29:46
CONVERTING THE WHOLE CODE INTO .YAML , HOW TO DEBUG THAT .YAML THEN