🎯 Key Takeaways for quick navigation: 00:00 🤖 *Introducción a Azure Function Blob Trigger* - Pedro presenta el tema de Azure Function Blob Trigger, destacando su utilidad y la importancia de los disparadores (triggers) en las funciones de Azure. - Los disparadores son esenciales para el funcionamiento de las funciones de Azure, marcando el inicio de estas. - Ejemplos de escenarios de uso para el blob storage trigger, como el procesamiento de imágenes en aplicaciones web. 02:02 💻 *Configuración inicial y requisitos* - Descripción del entorno de desarrollo utilizado, incluyendo Visual Studio Code en WSL Ubuntu y la instalación de herramientas necesarias. - Proceso para crear una cuenta de almacenamiento en Azure y la importancia de la extensión de Azure en VS Code. - Preparativos previos al desarrollo, como la instalación de Python y la configuración de una cuenta de almacenamiento en Azure. 05:37 🛠️ *Creación y configuración de la función Blob Trigger* - Paso a paso para crear la función Blob Trigger, incluyendo la configuración del string de conexión y la especificación del path del contenido. - Estrategias para hacer dinámico el path de los archivos a procesar, usando comodines para filtrar por tipo de archivo. - Demostración de la creación de un contenedor y una carpeta dentro de la cuenta de almacenamiento de Azure para almacenar los archivos a procesar. 09:29 🚀 *Ejecución y prueba de la función* - Demostración de la ejecución local de la función, incluyendo la carga de archivos CSV y la observación de la activación automática de la función. - Implementación de un programa de prueba para generar y cargar múltiples registros de ventas a Azure Storage, demostrando la funcionalidad de la función Blob Trigger en tiempo real. - Conclusiones finales, invitación a suscribirse al canal y sugerencias para futuros temas relacionados con Azure Functions y storage triggers. Made with HARPA AI
Great Video. Azure Function Python V2 Programming model has very limited videos other than just basic. I will greatly appreciate more videos' for advanced use cases.
Could you please show in detail how you were able to create those records as new file in the blob storage and share your code, so I could try to imitate the process and take it from there. Thanks
@@pytalistaThank you. I'd really appreciate that. In my use case, I need Azure Function to fetch a file from blob storage, perform operations, save pdf pages as jpg, and then either rename the original file or save the renamed file in another folder on blob storage. I'm struggling with connecting to and writing back onto the blob storage, and the documentation hasn't been helpful.
if the function doesnt work and it keeps getting nothing, the solution might be going to your funtionApp on Azure, and go to environment variables , and then try to add the connection name as a key and the connection string of your storage account as a value , and it will work.
Hi, if you want explicitly is easy. Just put the full blob name. I uploaded the code in the repository github.com/pedrojunqueira/PytalistaYT/blob/master/Python/storage_trigger/function_app.py
A blob trigger is an event driven action. So whenever a blob arrive at a particular location a function will be triggered. Service principle are to grant access to resources. So a blob trigger is independent of service principle. Seu you have a service principal that you want access to your storage account you can do that for sure. 👍🏻
this video is great. Can you also do a video talk about the output? I am using a blob trigger templet and process the blob in openai , but I have a problem to store the openai response into blob storage. I am appreciated if you do a video about handling the output. The Azure documentation is unclear.
Hi @rosscortb you just need to change you small thing if the code instead of df.to_csv(f, index=False) you do a df.to_excel(f, index=False) Here is the code github.com/pedrojunqueira/load_table_data_lake/blob/master/load_data_to_folder.py look at line 89. Cheers
@@pytalista I've managed to get a function app that works for me. I have deployed and added it to a pipeline. It starts by copying an oracle query to csv and then the function app coverts it. When adding the function app to the pipeline, it asks for a method - Select API Method. I assume its Post but then it requires a body? Any ideas what to put here? thanks
@@rosscortb I need a bit more context. A POST request is usually when you want to change an object in your API. in the body you pass a "payload" in JSON format. I need to see the code to understand better. If you are using a blog trigger you do not need to put in a pipeline (you mean data factory?) it triggers when a new blob arrives in the folder you specify.
Thanks for the video, very nicely done! It did not work at first for me because I had to specify in my local.settings.json the connection string to AzureWebJobsStorage. Also, the path in function_app did not require the storage account name but only the blob container name. Which ended up to be "blob_container_name/{name}" (I wanted my function to get triggered no matter if I would receive a csv or not; otherwise it would have been "blob_container_name/{name}.csv"
Hi, the video was very helpful for my POC. Thanks for that. can I know how to give multiple file patterns for example pdf and csv for the blob trigger to happend. I am trying this one but its not working. Moreover are there any restrictions on the file name patters as one of my blob name contains a space between the words and a curl brackets. I would really appreciate if you can provide the solution. Thanks in advance
Hi check the blob name patterns documentation. learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-trigger?tabs=python-v2%2Cisolated-process%2Cnodejs-v4&pivots=programming-language-python#blob-name-patterns
What you can do is get the file name and then apply different actions depending on the extension. like if it is .csv do this if it is such and such do that... cheers
@@SAHITHTHATIPALLI You can give one name that is going to get the blob name or a pattern. As I said. If you want to manage multiple use cases. Get the name of the incoming blob and handle it separately in a function.
It is a very greate video. I was looking for very long time for a video like that. Thank you! My code works perfectly when i run it on Visual Studio. But it is not triggered on Azure. Is there any special thing I need to do more? Code is deployed on Azure. I can see the updated code on Azure. Connection strings in Environment Variables are correct. When I run code in Azure I get 202 response. But I get no logs. There are no invocations on Azure.
There is no question :). I have figured it out. My connection was connection="BlobStorageConnectionString" and i did not add it to Environments variables. Stupid error.
Hi Pedro, I have a question, are you a Data Engineer? Also, thank you so much for the video. There isn't alot of Azure videos out there, but Azure for a Data Engineer perspective can be one of the best cloud platforms to use. Keep it up!
@@pytalista same here! I'm building a data pipeline that scraps pdf files from an email using logic app and function app. I am using HTTP trigger for function app, but its not the best one for it. The blob trigger is best so that when the pdf file lands in the container, it gets triggered
Is there any specific reason why the blob size comes out as None everytime? I have a use case which extends blob trigger to call a search service on newly added/modified files, and I'm getting blob not found 404 error 😑 despite the blob being there in my container. Thank you for the video though, it confirmed I'm not doing something astronomically wrong 😂!
@@pytalista Oh yes, I had to implement custom logic for my use case, so the code was quite different. The storage account I was trying to access did not allow anonymous access and an incorrect request URL was being generated. All in all, Blob was not found. I was able to debug it. Your video set a great precedent, thank you for replying.
When pressing F5 button it is giving error in launch json with exit code 1" The terminal process "/bin/bash '-c', '. . venv/bin/activate && func host start' " terminated with exit code 1
hey there pytalista sir i am trying to run this code using vs code and python it is not similarly but it is not working for me i tried everything i am facing this issue py exited with code 1 (0x1). AttributeError: '_SixMetaPathImporter' object has no attribute '_path',AttributeError: '_SixMetaPathImporter' object has no attribute '_path',AttributeError: '_SixMetaPathImporter' object has no attribute '_path'. and my path is like this blobtrigger/test/{name} please help mee
![Consume Azure Queues with Azure Function Queue Trigger [Python] V2.](http://i.ytimg.com/vi/swrzXmyTtrk/mqdefault.jpg)
![Consume Azure Queues with Azure Function Queue Trigger [Python] V2.](/img/tr.png)
🎯 Key Takeaways for quick navigation:
00:00 🤖 *Introducción a Azure Function Blob Trigger*
- Pedro presenta el tema de Azure Function Blob Trigger, destacando su utilidad y la importancia de los disparadores (triggers) en las funciones de Azure.
- Los disparadores son esenciales para el funcionamiento de las funciones de Azure, marcando el inicio de estas.
- Ejemplos de escenarios de uso para el blob storage trigger, como el procesamiento de imágenes en aplicaciones web.
02:02 💻 *Configuración inicial y requisitos*
- Descripción del entorno de desarrollo utilizado, incluyendo Visual Studio Code en WSL Ubuntu y la instalación de herramientas necesarias.
- Proceso para crear una cuenta de almacenamiento en Azure y la importancia de la extensión de Azure en VS Code.
- Preparativos previos al desarrollo, como la instalación de Python y la configuración de una cuenta de almacenamiento en Azure.
05:37 🛠️ *Creación y configuración de la función Blob Trigger*
- Paso a paso para crear la función Blob Trigger, incluyendo la configuración del string de conexión y la especificación del path del contenido.
- Estrategias para hacer dinámico el path de los archivos a procesar, usando comodines para filtrar por tipo de archivo.
- Demostración de la creación de un contenedor y una carpeta dentro de la cuenta de almacenamiento de Azure para almacenar los archivos a procesar.
09:29 🚀 *Ejecución y prueba de la función*
- Demostración de la ejecución local de la función, incluyendo la carga de archivos CSV y la observación de la activación automática de la función.
- Implementación de un programa de prueba para generar y cargar múltiples registros de ventas a Azure Storage, demostrando la funcionalidad de la función Blob Trigger en tiempo real.
- Conclusiones finales, invitación a suscribirse al canal y sugerencias para futuros temas relacionados con Azure Functions y storage triggers.
Made with HARPA AI
Thanks for this ! Gracias
Great Video. Azure Function Python V2 Programming model has very limited videos other than just basic. I will greatly appreciate more videos' for advanced use cases.
Thanks, will do!
Great Video! I was stuck on the how to get the Path variable and you demonstrated it perfectly. The documentation still doesn't have it.
Thanks
Could you please show in detail how you were able to create those records as new file in the blob storage and share your code, so I could try to imitate the process and take it from there. Thanks
This is great idea. I will do a quick video about it and publish the code in github
@@pytalistaThank you. I'd really appreciate that.
In my use case, I need Azure Function to fetch a file from blob storage, perform operations, save pdf pages as jpg, and then either rename the original file or save the renamed file in another folder on blob storage. I'm struggling with connecting to and writing back onto the blob storage, and the documentation hasn't been helpful.
Here is the video ua-cam.com/video/LxEh2g6q92o/v-deo.htmlsi=RKin42UY0Mxe_QeE 😀
@@pytalista that was quick. Really appreciate it. Thanks !!!!!!!
👍🏻
You saved my life! thank you
😀
if the function doesnt work and it keeps getting nothing, the solution might be going to your funtionApp on Azure, and go to environment variables , and then try to add the connection name as a key and the connection string of your storage account as a value , and it will work.
Yes
Thank you this is exactly what i concluded after A/B testing creating the app from the portal or with vs code
I have encountered an error called "No job function found". Can I get some help in this regard please?
Make sure you follow all pre requisites. Installations and etc.
In local setup it is working but when i deploy its not working
Check if any environment variables in your code. Look at logs do debug
Great video, the only problem I have, after I have deployed my app it doesn't respond to new file uploaded to the blob
Have tested locally ? Did it work?
Hi, I am facing a small issue. I do want to explicitly mention the container name in the code. could you please help me with that..?
Hi, if you want explicitly is easy. Just put the full blob name. I uploaded the code in the repository github.com/pedrojunqueira/PytalistaYT/blob/master/Python/storage_trigger/function_app.py
@@pytalista i do not want to explicitly memtion it. Currently I am using environmental variables.
@@SAHITHTHATIPALLI Ok good.
hey, its really great video. can we achieve blob trigger using service principle?? if data get loaded in blob storage our blob trigger will run..
A blob trigger is an event driven action. So whenever a blob arrive at a particular location a function will be triggered. Service principle are to grant access to resources. So a blob trigger is independent of service principle. Seu you have a service principal that you want access to your storage account you can do that for sure. 👍🏻
this video is great. Can you also do a video talk about the output? I am using a blob trigger templet and process the blob in openai , but I have a problem to store the openai response into blob storage. I am appreciated if you do a video about handling the output. The Azure documentation is unclear.
Hi @cadencechen7081 I think this is a great idea. I will consider.
I'd really like to see this as well
Very informative video can you also make a same Blob trigger video using Azurite. It will be very helpful.
Sure. If you watch the durable function one it uses azurite
Actually it does not. Ok I will do a video.
Please would you consider extending this video to include the CSV file being converted to xlsx saved to an output container?
Hi @rosscortb
you just need to change you small thing if the code
instead of df.to_csv(f, index=False)
you do a df.to_excel(f, index=False)
Here is the code github.com/pedrojunqueira/load_table_data_lake/blob/master/load_data_to_folder.py
look at line 89.
Cheers
@@pytalista I've managed to get a function app that works for me. I have deployed and added it to a pipeline. It starts by copying an oracle query to csv and then the function app coverts it. When adding the function app to the pipeline, it asks for a method - Select API Method. I assume its Post but then it requires a body? Any ideas what to put here? thanks
@@rosscortb I need a bit more context. A POST request is usually when you want to change an object in your API. in the body you pass a "payload" in JSON format. I need to see the code to understand better. If you are using a blog trigger you do not need to put in a pipeline (you mean data factory?) it triggers when a new blob arrives in the folder you specify.
Thanks for the video, very nicely done!
It did not work at first for me because I had to specify in my local.settings.json the connection string to AzureWebJobsStorage.
Also, the path in function_app did not require the storage account name but only the blob container name. Which ended up to be "blob_container_name/{name}" (I wanted my function to get triggered no matter if I would receive a csv or not; otherwise it would have been "blob_container_name/{name}.csv"
Well done Roxane. Great it worked in the end. Life of a programmer 🧑💻😀
Hi, the video was very helpful for my POC. Thanks for that. can I know how to give multiple file patterns for example pdf and csv for the blob trigger to happend. I am trying this one but its not working. Moreover are there any restrictions on the file name patters as one of my blob name contains a space between the words and a curl brackets. I would really appreciate if you can provide the solution. Thanks in advance
Hi check the blob name patterns documentation. learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-trigger?tabs=python-v2%2Cisolated-process%2Cnodejs-v4&pivots=programming-language-python#blob-name-patterns
What you can do is get the file name and then apply different actions depending on the extension. like if it is .csv do this if it is such and such do that... cheers
Thanks for the response. Is it possible to give multiple blob names to trigger the function.?!
@@SAHITHTHATIPALLI You can give one name that is going to get the blob name or a pattern. As I said. If you want to manage multiple use cases. Get the name of the incoming blob and handle it separately in a function.
@@pytalista Thank you
It is a very greate video. I was looking for very long time for a video like that. Thank you! My code works perfectly when i run it on Visual Studio. But it is not triggered on Azure. Is there any special thing I need to do more? Code is deployed on Azure. I can see the updated code on Azure. Connection strings in Environment Variables are correct. When I run code in Azure I get 202 response. But I get no logs. There are no invocations on Azure.
There is no question :). I have figured it out. My connection was connection="BlobStorageConnectionString" and i did not add it to Environments variables. Stupid error.
Awesome. Great it worked
Great to hear! Below you sorted out. Cheers
Hi Pedro, I have a question, are you a Data Engineer? Also, thank you so much for the video. There isn't alot of Azure videos out there, but Azure for a Data Engineer perspective can be one of the best cloud platforms to use. Keep it up!
Hi. Yes I am a Data Engineer
@@pytalista same here! I'm building a data pipeline that scraps pdf files from an email using logic app and function app. I am using HTTP trigger for function app, but its not the best one for it. The blob trigger is best so that when the pdf file lands in the container, it gets triggered
@@antonietapintorebelo5020 Good job !
Is there any specific reason why the blob size comes out as None everytime? I have a use case which extends blob trigger to call a search service on newly added/modified files, and I'm getting blob not found 404 error 😑 despite the blob being there in my container. Thank you for the video though, it confirmed I'm not doing something astronomically wrong 😂!
No worries Thanks. Hard to tell without more context. 404 is a not found error. If you follow the tutorial should work exact the same.
@@pytalista Oh yes, I had to implement custom logic for my use case, so the code was quite different. The storage account I was trying to access did not allow anonymous access and an incorrect request URL was being generated. All in all, Blob was not found. I was able to debug it. Your video set a great precedent, thank you for replying.
@diyakhajuria2478 great work. Happy it helped.
When pressing F5 button it is giving error in launch json with exit code 1" The terminal process "/bin/bash '-c', '. . venv/bin/activate && func host start' " terminated with exit code 1
make sure you all the requisite covered. All the installations and your environment is set correctly.
@@pytalista I am working on codespace is that making difference??
@manavkumar1071 i don’t know what is code space. Probably it is. They to configure on your local computer.
did u reslove it?
hey there pytalista sir i am trying to run this code using vs code and python it is not similarly but it is not working for me i tried everything
i am facing this issue
py exited with code 1 (0x1). AttributeError: '_SixMetaPathImporter' object has no attribute '_path',AttributeError: '_SixMetaPathImporter' object has no attribute '_path',AttributeError: '_SixMetaPathImporter' object has no attribute '_path'.
and my path is like this blobtrigger/test/{name}
please help mee
Hi make sure you follow all pre requisites. Did you install everything you need on your environment?
@pytalista all pre requisites installed
The music is too loud in the background
Thanks for the feedback. Please like and subscribe
Sir your video is helpful and informative but please don't put background music 🙏
Only the intro. My best video has loud music all the way. Go figure.
Really helpful video, thanks!
Thanks 🙏🏻
so helpful, thanks!
Thanks
thanks!
Thanks 🤩
this music my dude LOL
Liked ?
@@pytalista nope, please remove the music on your videos
There is no music just intro
That intro is too loud and irritating
Irritating like you ?
Can you help me with one azure function in python?
i am not able to deploy it . @pytalista
Where are you stuck ?