Great video. What do you think about when to use batch processing vs when to use stream processing with respect to time window size? For example if you are generating a report say every day I think batch processing from raw log data makes more sense. However if you are trying to generate aggregated data near real time, every 2 mins say, then stream processing makes more sense. But what can be a time window size where switching from stream processing to batch processing is beneficial.
Hey, can we have a Video on Designing a Real-Time Data Streaming Service based System, something like Spotify (Music Streaming), ClubHouse (Audio Streaming), Hotstar Live (Video Streaming), Google Meet (Video Streaming) ?
Thank you for providing such valuable content, please add more such videos.
Thank you, I will
Also something on Knowledge Graphs and how Search Engines use them behind the scenes, maybe building one with something like Python and Neo4J.
I like that idea. I will try to think what's the best way to create a video on that topic.
Hey - just a quick call out, you pasted the edit URL for the document in description. People can delete/modify it unexpectedly.
Great video. What do you think about when to use batch processing vs when to use stream processing with respect to time window size? For example if you are generating a report say every day I think batch processing from raw log data makes more sense. However if you are trying to generate aggregated data near real time, every 2 mins say, then stream processing makes more sense. But what can be a time window size where switching from stream processing to batch processing is beneficial.
You are spot on about batch vs stream processing.
I don't think there's a definite time window size. It depends on your use case.
Hi can you showcase how windowing can be achieved on synthetically generated random time stamped data in python and consumed using Kafka
I actually have that in my backlog. Expect to see it in the coming months.
Hey, can we have a Video on Designing a Real-Time Data Streaming Service based System, something like Spotify (Music Streaming), ClubHouse (Audio Streaming), Hotstar Live (Video Streaming), Google Meet (Video Streaming) ?
I do plan on creating a video about Spotify system design soon. Thank you for the suggestion!