Fantastic man !! I lately found ur channel, content regarding system design is top notch and precise to the point !! Could you add these below things into ur video, Example what are the popular frameworks are being used for particular kind of jobs, Ex Starem processing, So what programming language will be the exact match , with this details it really fills the missing link in this valuable content. Your kafka demo video is super helpful ,because you have shown the glimpse of implementation, pls do these kind of tuts it really helps, Thanks a ton brother !!
Glad you found these helpful! Regarding popular frameworks, Flink and Beam are some good open source ones. Most cloud vendors like AWS and GCP have managed services built on top of this.
thanks sir for such simple yet comprehensive content , however if one is intrested adding a bit of streaming compononent to a simple webapp like those being built with solara /fast api in python , or shiny-based apps in R ? what is the best way going about that ? let say I want to let my function behind the app calculate / estimate certain relationship but based off real-time data (usually it comes with delay) like those being featured on nasa weather conditions or yahoo or on-line IoT data sources or simple news articles on FT? while not consuming minimum of memory possible ! I appreciate if you can provide some form of written / video-based content of how to at least go about that on conceptual / design level ? your input is highly appreciated
In that case, I would just spin up Kafka, and write a lightweight Consumer to process the data locally. As long as you are not storing data locally, your memory consumption should be low. You can also consider RabbitMQ, they tend to be more lightweight than Kafka.
absolutely fantastic man ! all of your videos are easy to understand and helpful. Thanks dude
Appreciate the feedback.
Fantastic man !! I lately found ur channel, content regarding system design is top notch and precise to the point !!
Could you add these below things into ur video, Example what are the popular frameworks are being used for particular kind of jobs, Ex Starem processing, So what programming language will be the exact match , with this details it really fills the missing link in this valuable content.
Your kafka demo video is super helpful ,because you have shown the glimpse of implementation, pls do these kind of tuts it really helps, Thanks a ton brother !!
Glad you found these helpful!
Regarding popular frameworks, Flink and Beam are some good open source ones. Most cloud vendors like AWS and GCP have managed services built on top of this.
Good one mate
Thanks for checking it out! I have some more stream processing videos coming up next week. In the middle of a move, so it's been difficult to upload.
thanks sir for such simple yet comprehensive content , however if one is intrested adding a bit of streaming compononent to a simple webapp like those being built with solara /fast api in python , or shiny-based apps in R ? what is the best way going about that ? let say I want to let my function behind the app calculate / estimate certain relationship but based off real-time data (usually it comes with delay) like those being featured on nasa weather conditions or yahoo or on-line IoT data sources or simple news articles on FT? while not consuming minimum of memory possible !
I appreciate if you can provide some form of written / video-based content of how to at least go about that on conceptual / design level ?
your input is highly appreciated
In that case, I would just spin up Kafka, and write a lightweight Consumer to process the data locally. As long as you are not storing data locally, your memory consumption should be low.
You can also consider RabbitMQ, they tend to be more lightweight than Kafka.
boom