I love the way you explain. No distracting or unnecessary information, the tone of voice makes it easy to listen, and the slides help to understand the complex concepts of Azure Event Hubs. 👍 Thanks
⚠ Attention ⚠ I was pointed out that there is a small mistake in dotnet package adding commands. Both Azure.Messaging.EventHubs.Producer and Azure.Messaging.EventHubs.Consumer are not separate packages as such they don't need to be added specifically. They are part of Azure.Messaging.EventHubs package. You will see during video error in the console when adding them. Github repo is updated. As per very good comment from user hellhax, please note that Auto-inflate feature is "auto-scaleUP" not auto-scaling (up and down). I oversimplified the explanation here. If you want automatic scaling up and down, you would have to add some custom logic!
Great video Adam, Thank you. However, there are some unclarity 1. at 5'15, Throughput units represent the number of 10 k events or 1mb messages per sec. We can get a max of 20k events per sec or 20mb msgs per sec 2. at 18', you can have up to 20 consumer groups per event hubs (NOT partitions). The partition is just for horizontal scaling of the data across multiple nodes in the event hubs 3. at 18.15' it is advised to have one reader per partition in a Single consumer group (if not the consumer application has to handle dups). However, multiple consumer groups (with different readers) can read the same partition of an event hub
Great comments. I checked those times and I see how I could be more precise at those times. For point 3 I did say it's not recommended to have more readers than partitions as per docs. You can have less, but you are 100% correct that ideally you should have 1 to 1 ratio of consumers to partitions. For point 2 I said 1 to 20 consumer groups, but I think I said mistakenly we've seen that when we created namespace which could be confusing. Thanks for the pointers :) Will make sure to keep an eye for the details.
So I have Udemy and Pluralsight both subscription - But man Hats Off. You make it some simple and crystal clear - Just what is needed and with needed details. Thanks a lot
Very informative thanks Adam .. just little comment to be considered as mentioned in the documentation "Auto-inflate does not automatically scale down the number of TUs when ingress or egress rates drop below the limits."
Awesome Adam!! At first I was not able to understand from where I should start but watching your video really helped me! Also, I was able to do hands-on for this! Thanks Adam for this video!! Hope to see more videos :)
this is such great content. structured very well and explained in very simple terms. It's perfect for anyone starting out with EventHub. Thank you for this! subscribed
IMO it is important to mention that there is a limit of the number of event hubs in the namespace. This essentially means that you can’t use event hubs in the same way as you would use kafka where you could create lots of topics.
Thanks Pawel. In general your remark is very import but not only to event hubs but any service in Azure. Always understand limitations of each service that you are planning to use. This page is great for that docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits?WT.mc_id=AZ-MVP-5003556
Thanks for the great Video. It would have taken several days if I want to learn from the documents from Microsoft site. But this one helped to learn in hours.
Just finished your Event Hub Tutorial and very much enjoyed it. Excellent. I wish you would've also dived into Autoscaling and Geo-replication with region pairing. You only mentioned them at the end but didn't address them. I was looking for a good tutorial on Event Hub and found this. Excellent job. I see you have some other, related videos as well which I will study as well. Thank you Adam.
few issues resolved like dotnet add package Azure.Messaging.EventHubs.Processor --version 5.2.0, requires a version number to install the packages and to build
@Adam Marczak, I am confused about which criteria should be used for the amount of partitions needed in case of maximal throughput. The official documentation states: "The number of partitions in an event hub directly relates to the number of concurrent readers you expect to have". later the documentation also states: "Choose number of partitions based on the downstream parallelism you want to achieve as well as your future throughput needs." I would think that the partition should be based on the incoming data. Not on the amount of readers (consumers) who are eventually reading the data. Maybe I am missing something here?
In general all those sentences mean that ideally you should have 1 separate process reading from 1 partition or you might get duplicates while processing. But partitions allow you to scale horizontally across nodes.
Very interesting video with clear explanation. I always recommend your video to my friend and colleague if they want to start learning Azure. Can you show a example in Event Hub like 1) Two consumer were reading the data in different time internal. 2) How do we can configure a message to publish the message to specific set of consumer.
Thanks Phil! Think of consumer groups like separate applications that need to read your entire stream of data from end to end. Consumers in a consumer group are processes within the same application. This design allows for maximum parallelization.
Adam, thanks for an awesome explanations! Quick question...what are the options if we run into situation when our consumers can't keep the pace with producers and need to scale out to multiple instances (a lot of i/o on consumer side, e.g.)? what are the options to make sure there are no duplicate hadlings? tnx!
Great presentation. I am new to this, still understood. What is the difference between Kafka topic vs event hub. Both has got partitions and consumer group concepts.
I have a question, I’m Sending the events to the event hub using json file and I turned on the capture and I noticed that the file gets stored in ADLS in the format Of .Avro. My first question is the output of event hub always in the format of .avro ? My second question I’m trying to read events from Databricks , should I write my pyspark code keeping in mind that the output is in .avro format ?
Well, it's up to you to decide what do you want to do with the data later on. Maybe use avro as input and output CSV/parquet or something else. It's entirely up to you. Each format has it's advantages and disadvantages. Best to read about them and then decide. You can also stream data directly to databricks instead of capturing as flat files docs.microsoft.com/en-us/azure/azure-databricks/databricks-stream-from-eventhubs
neelima wiyyapu hey Neelima, the events in the event hub will always be stored in ADLS in the .avro format when you turn the capture feature on and we can’t change the format of the file.
At 29:51 - When you have set the retention period 7 HRS and CAPTURE:ON and TIME WINDOWS: 1 min. Will it still save the event in Azure blob storage and why as our retention period was 7 hours
Great video.. I have one doubt if I want to handle 20k messages/ second. Do I need to add 20 partitions with 20 Throughput units? Is it one to one ratio?
It's a little bit more complex than this. Please check this doc as it nicely explains key things to note docs.microsoft.com/en-us/azure/event-hubs/event-hubs-scalability?WT.mc_id=AZ-MVP-5003556
One request ...don't know If I am asking more as you already have done a great job....May you please add some videos containing the business used cases and how these services interacting with each other in real time . Thanks in Advance.
Hi Adam, Nice presentation!! I have a question for you. How we can pull data from Server sent Events (SSE) to Azure ? ( Azure Eventhub or Blob storage)
Hey Please elaborate on your scenario as the description is too generic. If you are talking about azure web applications then you should use Azure Application Insights service.
I am just enjoying your presentations and Demo. they are so clear and to the point. could you help us by covering topics related to AZ 204 certifications as well?
That is a great question. I think it's simply using .option("checkpointLocation", "") when using readStream as per spark documentation. spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing
Hi Adam, I do not get one thing. While sending message you do not code anywhere for Storage & Blob. Why it is needed while Reading. Where is the connection between Sender app & StorageBlob
Hey, good question. At 18:20 I explain what is offset and checkpointing mechanism. When clients read data they need to save offset somewhere so that when they run again they won't start reading from the beginning of the stream, just from where they left off. SDK allows to save offset on blob storage. Remember that Event Hubs are not queues, they always have data for X amount of days, so if your program would fail then you would read everything from the last X days unless you saved checkpoints. When it comes to writing, it's just fire and forget so no need for blob.
thanks, Adam, for a concise and powerful tutorial! I would like to use Java to read the events from an event hub and to send them to an IoT Hub. Do you have something similar or could you recommend me some other source?
Very helpful tutorial sir Adam. Just wondering if there's a way to test event fetching without implementing and using the SDK. To test it via API call using postman for example?
Hi Adam, thank you for the great video. I wanted to ask you if you have a video about how to configure a VM to send OS logs to the Event Hub. I am struggling to see how to set it up so I can consume the VM's OS logs and send it to Splunk. thank you for your suggestions.
@@AdamMarczakYT you what I am trying to learn powershell scripting and after going thru many videos i finally remembered you and saw ur video and trust me you definitely have a place reserved in heaven.
Oh! Well spotted my friend. I somehow did this both for Producer but also Consumer namespace. I think I just grabbed using statements without thinking and put dotnet add package as a list. Unfortunately VSO doesn't show red for errors in console so I didn't notice my mistake. Too bad I can't add overlay to the existing video to correct this. Thanks for that. I already updated github repo samples! I added this as pinned comment on top so people hopefully take notice. As a multi-year .NET developer I'm ashamed of such silly mistake.
If i need to order message as per the order, then i need to use partition key . But if i use the partition , all data will land in same partition which could be a problem again Basically how do maintain order across the partitions
Hi, how can you send events from other Azure PasS services ( like DataLake, Data Factory, MySQL, Databricks, etc.) to the Event Hub? Is it possible to customise the events as well, for logging purposes?
Great video. Is it possible to set role base access for the consumer groups? I have basic tier so only have the default one and I can't set it for that.
I still don't understand consumer group. If I look at your code, the app is using blob storage to store a pointer to the location on event hub. If I have different apps, using different pointers, they can be at different locations in the stream. If I delete the files in the blob storage and rerun your code, it will read everything in the event hub from the start. So, what does consumer group give other than a label?
One app = one consumer group :) Blob stores checkpoints (offsets) for each app, if you delete it then it doesn't know where it ended last read hence reading from the start. docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features?WT.mc_id=AZ-MVP-5003556#consumer-groups
@@AdamMarczakYT What if my different apps used different checkpoints but the same consumer group? What if I shutdown the app for 8 days where the retention period of eventhub is 7 days, then what happens?
Hello Adam, nice video and very clear explanations. Can we send structured data in the event like JSON ? is there any limit of the size in byte for the event message ?
Hey, thanks! Yes you can send JSON. For limits and quotas for event hubs check out this doc docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quotas?WT.mc_id=AZ-MVP-5003556
Hi @Adam Marczak. I am new in Azure IoT Cloud. I am working with my app. When I test in Samples Examples I receive all data from simulated devices. When I add it in my application like a Plugin I receive the following message: 1836672 WARN [single-1] ConnectionHandler: onTransportError hostname[namespace.servicebus.windows.net:5671], connectionId[MF_561eee_1601915124635], error[Only authorized connections are allowed in development (myeventsnamespace.servicebus.windows.net denied)] . The same code works fine outside my plugin, but it does not work inside. Do you know what is happening?
Hi Adam, Great work, I just wanted to know that is it possible to send real time frames of a video to event hub? or does event hub deal with image data?
Your efforts in preparing the content are commendable (y). Which Softwares do you use or recommend recording and editing the videos? I also wish to prepare some videos.
I use Camtasia to record screen and Davinci resolve to combine recording from camera with screen and do extra effects. But in most cases camtasia is good enough.
@@AdamMarczakYT Thanks for letting me know. I'm inspired by the way you present and thought of creating the tutorial videos. Though I'm new in creating videos and I would need expert to advise me to make the perfect videos. I got Filmora and created a small video on Git Introduction. ua-cam.com/video/5Z9iD3jXa_Y/v-deo.html
Thanks. Some day in near future I plan to release small tips and tricks video of what I learned over the last year doing my videos. This could be helpful to you. Thanks for stopping by.
A 3hr Pluralsight video = 30 mins of your video. Well done.
The greatest complement! Thanks! :)
@@AdamMarczakYT True story. I thought the same watching your video about App insights
I love the way you explain. No distracting or unnecessary information, the tone of voice makes it easy to listen, and the slides help to understand the complex concepts of Azure Event Hubs. 👍 Thanks
⚠ Attention ⚠
I was pointed out that there is a small mistake in dotnet package adding commands. Both Azure.Messaging.EventHubs.Producer and Azure.Messaging.EventHubs.Consumer are not separate packages as such they don't need to be added specifically. They are part of Azure.Messaging.EventHubs package. You will see during video error in the console when adding them. Github repo is updated.
As per very good comment from user hellhax, please note that Auto-inflate feature is "auto-scaleUP" not auto-scaling (up and down). I oversimplified the explanation here. If you want automatic scaling up and down, you would have to add some custom logic!
Will update this in future if MS adds full auto scaling features to event hubs. :)
Great video Adam, Thank you. However, there are some unclarity
1. at 5'15, Throughput units represent the number of 10 k events or 1mb messages per sec. We can get a max of 20k events per sec or 20mb msgs per sec
2. at 18', you can have up to 20 consumer groups per event hubs (NOT partitions). The partition is just for horizontal scaling of the data across multiple nodes in the event hubs
3. at 18.15' it is advised to have one reader per partition in a Single consumer group (if not the consumer application has to handle dups). However, multiple consumer groups (with different readers) can read the same partition of an event hub
Great comments. I checked those times and I see how I could be more precise at those times. For point 3 I did say it's not recommended to have more readers than partitions as per docs. You can have less, but you are 100% correct that ideally you should have 1 to 1 ratio of consumers to partitions. For point 2 I said 1 to 20 consumer groups, but I think I said mistakenly we've seen that when we created namespace which could be confusing. Thanks for the pointers :) Will make sure to keep an eye for the details.
So I have Udemy and Pluralsight both subscription - But man Hats Off. You make it some simple and crystal clear - Just what is needed and with needed details. Thanks a lot
Thank you, glad to help out :)
Very informative thanks Adam .. just little comment to be considered as mentioned in the documentation "Auto-inflate does not automatically scale down the number of TUs when ingress or egress rates drop below the limits."
awesome. First I watch your video then I go to Microsoft Learn documentation. Best way to retain knowledge.
Fantastic presentation Adam. To the point without any unnecessary detail.
Glad it was helpful!
Great tutorial - Thanks
You're welcome!
Welldone ! Your video covers the key concepts and cuts all the noise ! Much appreciated
All of your videos are extremely helpful. Very informative. Clear presentation. Absolutely marvellous. Thanks a lot.
Glad you like them! Thank you :)
Very good work Adam. Thank you honestly.
My pleasure!
This is the best video I got on this topic. I am going bananas for last 2 days. This is simply awesome
Awesome Adam!! At first I was not able to understand from where I should start but watching your video really helped me! Also, I was able to do hands-on for this! Thanks Adam for this video!! Hope to see more videos :)
Glad I could help!
this is such great content. structured very well and explained in very simple terms. It's perfect for anyone starting out with EventHub. Thank you for this! subscribed
IMO it is important to mention that there is a limit of the number of event hubs in the namespace. This essentially means that you can’t use event hubs in the same way as you would use kafka where you could create lots of topics.
Thanks Pawel. In general your remark is very import but not only to event hubs but any service in Azure. Always understand limitations of each service that you are planning to use.
This page is great for that docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits?WT.mc_id=AZ-MVP-5003556
Thanks for the great Video. It would have taken several days if I want to learn from the documents from Microsoft site. But this one helped to learn in hours.
Compact and nicely explained. Please make a video for building micro services.
Thanks! Appreciated :) noted the idea.
Awesome, great presentation. Thanks Adam.
Glad you liked it!
Just finished your Event Hub Tutorial and very much enjoyed it. Excellent. I wish you would've also dived into Autoscaling and Geo-replication with region pairing. You only mentioned them at the end but didn't address them. I was looking for a good tutorial on Event Hub and found this. Excellent job. I see you have some other, related videos as well which I will study as well. Thank you Adam.
Very concise and effective video.
Great work
Much appreciated!
You are so clean in your description .
thankyou so much adam i cannot tell you how much this video help me , i am stuck in something and this video slove that. thanks man.
precise and crisp.. well narrated.
few issues resolved like dotnet add package Azure.Messaging.EventHubs.Processor --version 5.2.0, requires a version number to install the packages and to build
Cool! Thanks for letting me and others know :)
Thanks Adam very well presented and explain in simple language.
Nicely Explained...And very interesting and informative..Thanks a lot..You saved my time..
Glad to hear that
@Adam Marczak,
I am confused about which criteria should be used for the amount of partitions needed in case of maximal throughput.
The official documentation states: "The number of partitions in an event hub directly relates to the number of concurrent readers you expect to have".
later the documentation also states: "Choose number of partitions based on the downstream parallelism you want to achieve as well as your future throughput needs."
I would think that the partition should be based on the incoming data. Not on the amount of readers (consumers) who are eventually reading the data.
Maybe I am missing something here?
In general all those sentences mean that ideally you should have 1 separate process reading from 1 partition or you might get duplicates while processing. But partitions allow you to scale horizontally across nodes.
Very interesting video with clear explanation. I always recommend your video to my friend and colleague if they want to start learning Azure. Can you show a example in Event Hub like
1) Two consumer were reading the data in different time internal.
2) How do we can configure a message to publish the message to specific set of consumer.
what a talented guy - high quality content - excellent explanations!
Excellent explanation in short vedio.
This is very good and clear, congrats!
Thanks a lot!
Amazing video and great instructor. Keep it up!!
Thanks, will do!
Really nice overview of Event Hubs! Well done and informative! Subscribed! :)
This was very clear! Thank you
Awesome presentation... short and crisp 👍👍
Thanks a lot 😊
Great video! keep up the good work!
WHat a beautiful perfect introduction :), thank you!
Great content, very well presented Adam! Keep up the great work.
wonderful demo
Great demo, Thanks Adam
Great video Adam
Hi Adam, how are you?
just to say thank you! your azure video tutorials are really good!
Good one...nice clean detailing of topic..Thanks a lot!
Glad you liked it 😁
Great presentation! Much appreciated :)
My pleasure!
Azure has fixed the UI, now you have to pick at least two partitions.
Did they? I think it depends on event hub tier whether it's a standard or a basic one. I'll check later :)
Great overview, good balance of information and demos
Much appreciated!
Azure saviour thanks
Hehe thanks! :)
Great work Adam, really useful video. I'm still a little unclear as to the purpose of Consumer Groups
Thanks Phil! Think of consumer groups like separate applications that need to read your entire stream of data from end to end. Consumers in a consumer group are processes within the same application. This design allows for maximum parallelization.
Fantastic presentation, good job!
Thank you very much!
Adam, thanks for an awesome explanations!
Quick question...what are the options if we run into situation when our consumers can't keep the pace with producers and need to scale out to multiple instances (a lot of i/o on consumer side, e.g.)? what are the options to make sure there are no duplicate hadlings?
tnx!
Lots of thanks mr. Adam.. very very helpful tutorials ..👌
My pleasure!
Good presentation! Articulated very well.. Thank you!
Glad it was helpful!
Nicely explained, keep it up 👍
Thanks , very clear !
Glad it helped!
Great presentation. I am new to this, still understood. What is the difference between Kafka topic vs event hub. Both has got partitions and consumer group concepts.
Check this stackoverflow response on this stackoverflow.com/questions/63953085/azure-event-hub-vs-kafka-as-a-service-broker
Thanks for your work❤
This video is so great, thanks Adam
Glad you liked it!
Thanks, that helped me a lot!
You're welcome!
I have a question, I’m
Sending the events to the event hub using json file and I turned on the capture and I noticed that the file gets stored in ADLS in the format Of .Avro. My first question is the output of event hub always in the format of .avro ?
My second question I’m trying to read events from Databricks , should I write my pyspark code keeping in mind that the output is in .avro format ?
Well, it's up to you to decide what do you want to do with the data later on. Maybe use avro as input and output CSV/parquet or something else. It's entirely up to you. Each format has it's advantages and disadvantages. Best to read about them and then decide. You can also stream data directly to databricks instead of capturing as flat files docs.microsoft.com/en-us/azure/azure-databricks/databricks-stream-from-eventhubs
Hi prateek , were you able to change the format of the stored file ?
neelima wiyyapu hey Neelima, the events in the event hub will always be stored in ADLS in the .avro format when you turn the capture feature on and we can’t change the format of the file.
@@prateekkemkar8192 thanx for letting me know Prateek !
Great tutorial❤ is it possible you make vedio on connecting to azure event hub using azure service principal without using shared access key
At 29:51 - When you have set the retention period 7 HRS and CAPTURE:ON and TIME WINDOWS: 1 min. Will it still save the event in Azure blob storage and why as our retention period was 7 hours
Thank you brother.
Great video..
I have one doubt if I want to handle 20k messages/ second. Do I need to add 20 partitions with 20 Throughput units?
Is it one to one ratio?
It's a little bit more complex than this. Please check this doc as it nicely explains key things to note docs.microsoft.com/en-us/azure/event-hubs/event-hubs-scalability?WT.mc_id=AZ-MVP-5003556
great work adam, but I am still not clear were was .net code was written? It was written on local system visual studio?
One request ...don't know If I am asking more as you already have done a great job....May you please add some videos containing the business used cases and how these services interacting with each other in real time .
Thanks in Advance.
If I can I will create implementation videos on the future, thanks!
Hi Adam, Nice presentation!! I have a question for you. How we can pull data from Server sent Events (SSE) to Azure ? ( Azure Eventhub or Blob storage)
Hey Please elaborate on your scenario as the description is too generic. If you are talking about azure web applications then you should use Azure Application Insights service.
Adam thank you for video! Is packing events in batches a preferred way to send events or we can send them one by one?
If you send a lot of events very fast then I'd batch them together to reduce networking load.
super clear!
Thank you! :)
Thanks for great learning of EH, can you also add confluent kafka and difference between EH vs CK.
I am just enjoying your presentations and Demo. they are so clear and to the point. could you help us by covering topics related to AZ 204 certifications as well?
I'll check what will be my next goal after I will finish az-900 course :).
Very informative. Thank you. I am a little bit confused how to do checkpoint in Databrick (Notebook) when preforming stream processing from Event Hub?
That is a great question. I think it's simply using .option("checkpointLocation", "") when using readStream as per spark documentation.
spark.apache.org/docs/latest/structured-streaming-programming-guide.html#recovering-from-failures-with-checkpointing
Good Job, Adam!!!
my pleasure!
Hi Adam, I do not get one thing. While sending message you do not code anywhere for Storage & Blob. Why it is needed while Reading. Where is the connection between Sender app & StorageBlob
Hey, good question. At 18:20 I explain what is offset and checkpointing mechanism. When clients read data they need to save offset somewhere so that when they run again they won't start reading from the beginning of the stream, just from where they left off. SDK allows to save offset on blob storage. Remember that Event Hubs are not queues, they always have data for X amount of days, so if your program would fail then you would read everything from the last X days unless you saved checkpoints. When it comes to writing, it's just fire and forget so no need for blob.
@@AdamMarczakYT Thanks for the clarification and reply
thanks, Adam, for a concise and powerful tutorial! I would like to use Java to read the events from an event hub and to send them to an IoT Hub. Do you have something similar or could you recommend me some other source?
nice content. Congrats
Very helpful tutorial sir Adam. Just wondering if there's a way to test event fetching without implementing and using the SDK. To test it via API call using postman for example?
Way too Good. Keep it up.
Thanks a lot 😊
Hi Adam, thank you for the great video. I wanted to ask you if you have a video about how to configure a VM to send OS logs to the Event Hub. I am struggling to see how to set it up so I can consume the VM's OS logs and send it to Splunk. thank you for your suggestions.
@adam ... Thanks for a well put together tutorial.
My pleasure!
thanks Adam for clean explanation. Can you explain what is the role of epoch in event hubs ? I went through few articles but didn't quite got it. :/
Great tutorial Adam!
Do you have any tutorial for send recv in Python also?
Not yet! Thanks for watching! :)
Adam can you also create videos where all the services created on the azure webportal can be done through powershell and CLI also please
I might do that in the future with larger courses around azure services :) hard to fit in that in the 30 min overview video.
@@AdamMarczakYT you what I am trying to learn powershell scripting and after going thru many videos i finally remembered you and saw ur video and trust me you definitely have a place reserved in heaven.
Azure.Messaging.EventHubs.Producer is a invalid package and you also have an error while installing this
Oh! Well spotted my friend. I somehow did this both for Producer but also Consumer namespace. I think I just grabbed using statements without thinking and put dotnet add package as a list. Unfortunately VSO doesn't show red for errors in console so I didn't notice my mistake. Too bad I can't add overlay to the existing video to correct this. Thanks for that. I already updated github repo samples!
I added this as pinned comment on top so people hopefully take notice. As a multi-year .NET developer I'm ashamed of such silly mistake.
Dzięki za to video!
Thanks! And the next pass to read the avro?
Use Azure Data Factory to read Avro files.
bro, it's great!
If i need to order message as per the order, then i need to use partition key . But if i use the partition , all data will land in same partition which could be a problem again
Basically how do maintain order across the partitions
Awesome
Thanks!
Great video!! 👍
Thank you! 👍
Hi, how can you send events from other Azure PasS services ( like DataLake, Data Factory, MySQL, Databricks, etc.) to the Event Hub? Is it possible to customise the events as well, for logging purposes?
You can use REST API to send events from anywhere :)
@@AdamMarczakYT Ok, I will try in the near future, not sure how to do that yet (new to Azure). Btw thank you, appreciated ;)
Great video. Is it possible to set role base access for the consumer groups? I have basic tier so only have the default one and I can't set it for that.
I still don't understand consumer group. If I look at your code, the app is using blob storage to store a pointer to the location on event hub. If I have different apps, using different pointers, they can be at different locations in the stream. If I delete the files in the blob storage and rerun your code, it will read everything in the event hub from the start. So, what does consumer group give other than a label?
One app = one consumer group :)
Blob stores checkpoints (offsets) for each app, if you delete it then it doesn't know where it ended last read hence reading from the start.
docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features?WT.mc_id=AZ-MVP-5003556#consumer-groups
@@AdamMarczakYT What if my different apps used different checkpoints but the same consumer group? What if I shutdown the app for 8 days where the retention period of eventhub is 7 days, then what happens?
Hello Adam, nice video and very clear explanations. Can we send structured data in the event like JSON ? is there any limit of the size in byte for the event message ?
Hey, thanks! Yes you can send JSON. For limits and quotas for event hubs check out this doc docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quotas?WT.mc_id=AZ-MVP-5003556
Hi @Adam Marczak. I am new in Azure IoT Cloud. I am working with my app. When I test in Samples Examples I receive all data from simulated devices. When I add it in my application like a Plugin I receive the following message: 1836672 WARN [single-1] ConnectionHandler: onTransportError hostname[namespace.servicebus.windows.net:5671], connectionId[MF_561eee_1601915124635], error[Only authorized connections are allowed in development (myeventsnamespace.servicebus.windows.net denied)] . The same code works fine outside my plugin, but it does not work inside. Do you know what is happening?
Hi Adam, Great work, I just wanted to know that is it possible to send real time frames of a video to event hub? or does event hub deal with image data?
it's possible but not recommended, maximum event message size only only 256 KB, better upload this to a blob storage.
@@AdamMarczakYT Okay, Thanks Adam
So for this will iot edge work fine?
And also don't we need some sort of intermediate between device and blob storage for capturing real time frames. Will iot edge work there?
Good Job
Your efforts in preparing the content are commendable (y). Which Softwares do you use or recommend recording and editing the videos? I also wish to prepare some videos.
I use Camtasia to record screen and Davinci resolve to combine recording from camera with screen and do extra effects. But in most cases camtasia is good enough.
@@AdamMarczakYT Thanks for letting me know. I'm inspired by the way you present and thought of creating the tutorial videos. Though I'm new in creating videos and I would need expert to advise me to make the perfect videos. I got Filmora and created a small video on Git Introduction.
ua-cam.com/video/5Z9iD3jXa_Y/v-deo.html
I have subscribed to your channel because you have good content and nice presentation skills.
Thanks. Some day in near future I plan to release small tips and tricks video of what I learned over the last year doing my videos. This could be helpful to you. Thanks for stopping by.
Thank you
Hi Adam, are you creating any other azure exam playlist just like AZ900 ?
Thanks Adam for wonderful video on Event Hub. Just a request can you also make a video on Azure IoT Hub.
Great suggestion! It's on the list :) thanks!