Hi Mark, thank you for your clear video! I want to automate fetching data from an api that will be used for training and deployment at a particular time everyday? How can I achieve this?
Thanks for the comment. For automating fetching of data from an API on a given schedule, I suggest that you take a look at cloud.google.com/run/docs/execute/jobs-on-schedule. Cloud Run gives you significant flexibility and this topic shows how to schedule Cloud Run jobs.
Great vid. I do have some suggestions though. I think you should either link to previous or connected videos in description or create a playlist with all the connected videos for e.g. creating ML pipelines on Vertex AI.
Thanks very much for the recommendation. I have created a public playlist for my Vertex AI videos: ua-cam.com/play/PLDr6KjrVE4CU7gVu5znuNesTAG6wYU9d_.html&si=LtY8Vx04v0m75FRh
@markryan2475 how did you move your pipeline script and training script to cloud shell? i mean where did you keep those files? is it in you local file system or is it in cloud bucket?
Hi - I ran the pipeline script in Cloud Shell. That is, I just treated Cloud Shell as a Linux instance to run things from. You can upload files one at a time to Cloud Shell from a local system, or clone a repo to get files there. For the training script, it needs to be in the container.
@@AdhvaithG - I didn't go through the process of scheduling the pipeline script. However, if I were to do it, I would start with the procedure described here: cloud.google.com/vertex-ai/docs/pipelines/schedule-pipeline-run
![End-To-End: Pipeline Orchestration (KFP) - AutoML in Vertex AI for ML Operations [notebook 02c]](/img/n.gif)
How do I thank you for this very rare video on UA-cam.
Thanks for the comment. I'm glad you found the video helpful.
Hi Mark, thank you for your clear video! I want to automate fetching data from an api that will be used for training and deployment at a particular time everyday? How can I achieve this?
Thanks for the comment. For automating fetching of data from an API on a given schedule, I suggest that you take a look at cloud.google.com/run/docs/execute/jobs-on-schedule. Cloud Run gives you significant flexibility and this topic shows how to schedule Cloud Run jobs.
Great vid. I do have some suggestions though. I think you should either link to previous or connected videos in description or create a playlist with all the connected videos for e.g. creating ML pipelines on Vertex AI.
Thanks very much for the recommendation. I have created a public playlist for my Vertex AI videos: ua-cam.com/play/PLDr6KjrVE4CU7gVu5znuNesTAG6wYU9d_.html&si=LtY8Vx04v0m75FRh
@markryan2475 how did you move your pipeline script and training script to cloud shell? i mean where did you keep those files? is it in you local file system or is it in cloud bucket?
Hi - I ran the pipeline script in Cloud Shell. That is, I just treated Cloud Shell as a Linux instance to run things from. You can upload files one at a time to Cloud Shell from a local system, or clone a repo to get files there. For the training script, it needs to be in the container.
@@markryan2475 Thanks for quick response. In case if we want to schedule a pipeline script, how we can automate that? Please suggest your opinion.
@@AdhvaithG - I didn't go through the process of scheduling the pipeline script. However, if I were to do it, I would start with the procedure described here: cloud.google.com/vertex-ai/docs/pipelines/schedule-pipeline-run
I have a doubt regarding how many containers used in total.