This was great. After spending some time going through various tutorials, I found some that had typos in the code, others where the demo was filled with very complex js, and yours was simple and to the point.
good question. I think the answer is that a promise does not make a sync operation happen async, instead it gives you a way to deal with an already async operation. At the end of the day this is just a simple for loop and it will always happen synchronously
@@CodingWithChaim you can promisify this for loop Edit: I recreated the same environment you used in this video and spent 5 hours playing with worker threads. My original comment is wrong. While/for loop is still blocking even if it’s wrapped in a promise unless there is a promise within its conditional or within its block.
Now, can you tell me the limitations of using this worker thread ? Like how many workers can we spawn ? does it depend on the CPU ? can it be used as promise.all() ?
Are there any threading concerns (race conditions, deadlocking, etc.) that can arise with Workers? For instance, is the registering of the worker.on('message'...) code guaranteed to execute before the worker calls its postMessage()?
Hey , I wanted to run some queries on a worker thread and passed in sequelize instnace to the thread but it failed , is there any way how we can pass sequelize instance to the thread and execute the queries on the difrent thread itself on nodejs ,it would have been much easier if it was GO
Are worker threads able to do the same requests as the main thread? I'm aggregating 5.2M documents on Mongodb. It's no problem aggregating on the main thread but it's taking for ever on the worker thread..?
Hi chaim ! It was a nice explanation, but one thing i want to know about, if we make a worker thread then after it's process , will it not be an orphan task, so have we kill that child_process or not after completion?
great question! I actually dont know the answer off hand. My assumption would be that the worker will get cleaned up after its done the work, but I would have to verify to know for sure
Question on cleanup - when working with a singular worker instance to we want to be instantiating a new worker for every call to that url? Is it a better practice to reuse them?
I would say its better to create a new worker for each request because this way we can scale. If we only had one worker that we kept re using, we would run into the same bottleneck we do as with the main thread
@@CodingWithChaim can't believe you got back so quick! Last question - do we need to cleanup the old worker? This reminds me of RXJS observable patterns and I know that failure to cleanup observables can add up over time. Should we delete these workers after the message?
thats a great question! My hunch is that after they get used they will be garbage collected, but I would say you should verify that because I dont know that for sure
@@CodingWithChaim launching my "covid-project" in two weeks with some decently heavy computational work. I had a thought like 'wait, does this block my event loop?' two days later I find your excellent , succinct solution. Thanks again for the help. I'll circle back if I find anything down the road.
If im running the exact same app not locally but on GCP Cloud Run, would it still hang? I assumed each request spun up a new instance of your app thua nullifying any thread issues from the heavy endpoint.
wow a jewish youtuber, thats rare, as rare as seeing a rainbow. not trying to be mean or anything just abit surprised, thats all, im jewish too so the more the better. Hello bytheway Chaim, really great english too, without any hebrew accent as well. thanks for the video, i'v found about worker threads through google ofc, while searching for cool things about nodejs. didn't quite understand how to use them so i got here, very well explain and articulated. i'll save for future reference.
While the explanation on how to use worker's is good,the example is really really bad. If this is an Endpoint that is exposed to the client, there will be potentially hundreds of thousands of requests to that endpoint, so the node application will spawn that numbers of new threads with it's corresponding node instances, event loops etc, which is a nightmare in terms of performance. So keep that in mind..use workers for heavy duty, yes, but not if it will be requested an inmensurable number of times
So, if You have to do some chained awaits that update a database. How do you handle correctly the endpoint for anlot of users? I thought about doing nothing and let AWS spawn more tasks copies behind a las balancer. AM i right?
Can't believe how clean is this explanation. Thank you sir.
I've always known I needed to know Workers but am always so overwhelmed with how complicated it seemed thanks for making me understand this better
This was great. After spending some time going through various tutorials, I found some that had typos in the code, others where the demo was filled with very complex js, and yours was simple and to the point.
Perfect example, perfect explanation & presentation. Thanks for this video!
Chaim, thank you so much for the video. Your explanation is clear and awesome
Someone at my job gave me this exact explanation, excellent content
This is really good, seen it here first! Thank you man, keep up the great work.
One of the most interesting and very beautifully explained topic.
Thank you very much.
Amazing example. very sorted and easy. Thanks and love from India
סרטון מאוד מעניין ומעשיר ידע תודה לך חיים
Wonderful explanation!!!! Thank You!
Well thanks, i Will try this today. Just curious about what happens when you Open several Windows causing going out of threads in your system
Thanks for another great video! I'm not a node developer but wouldn't you be able to accomplish this by wrapping the loop in a promise?
good question. I think the answer is that a promise does not make a sync operation happen async, instead it gives you a way to deal with an already async operation. At the end of the day this is just a simple for loop and it will always happen synchronously
@@CodingWithChaim you can promisify this for loop
Edit: I recreated the same environment you used in this video and spent 5 hours playing with worker threads. My original comment is wrong. While/for loop is still blocking even if it’s wrapped in a promise unless there is a promise within its conditional or within its block.
great video - helped my solve a problem - thanks Chaim!
Thanks Chaim. This was very helpful.
can we define the pool size for this ?
Thank you very much! This was exacly what I was looking for. :)
thank you very helpful
Awesome video
Question, can I pass just a function inside the worker threads,
or does it really need to be a file ?
Thank you very much! More videos on the node! :)
Now, can you tell me the limitations of using this worker thread ? Like how many workers can we spawn ? does it depend on the CPU ? can it be used as promise.all() ?
Straight to the point. Great.
Are there any threading concerns (race conditions, deadlocking, etc.) that can arise with Workers? For instance, is the registering of the worker.on('message'...) code guaranteed to execute before the worker calls its postMessage()?
Simple and crisp, thanks
That's a great explanation! Thank you. 💯
Glad it was helpful!
Great video man thanks a lot 🙏🏻💯 I run some bots written in JS, would this make them faster?
It wouldn’t make them faster. But it would allow them to run in parallel if each of them has their own thread
Hey , I wanted to run some queries on a worker thread and passed in sequelize instnace to the thread but it failed , is there any way how we can pass sequelize instance to the thread and execute the queries on the difrent thread itself on nodejs ,it would have been much easier if it was GO
awesome video, well presented and explained
Should we always use worker thread instead of fork method of child process?
brilliant, very good example!
Thanks you, very clear explanation 👍
Thanks you very much!
Are worker threads able to do the same requests as the main thread?
I'm aggregating 5.2M documents on Mongodb. It's no problem aggregating on the main thread but it's taking for ever on the worker thread..?
Great content! Thank you!!! :D
Hi chaim ! It was a nice explanation, but one thing i want to know about, if we make a worker thread then after it's process , will it not be an orphan task, so have we kill that child_process or not after completion?
great question! I actually dont know the answer off hand. My assumption would be that the worker will get cleaned up after its done the work, but I would have to verify to know for sure
@@CodingWithChaim Thanks for answer
Question on cleanup - when working with a singular worker instance to we want to be instantiating a new worker for every call to that url? Is it a better practice to reuse them?
I would say its better to create a new worker for each request because this way we can scale. If we only had one worker that we kept re using, we would run into the same bottleneck we do as with the main thread
@@CodingWithChaim can't believe you got back so quick! Last question - do we need to cleanup the old worker? This reminds me of RXJS observable patterns and I know that failure to cleanup observables can add up over time.
Should we delete these workers after the message?
thats a great question! My hunch is that after they get used they will be garbage collected, but I would say you should verify that because I dont know that for sure
@@CodingWithChaim launching my "covid-project" in two weeks with some decently heavy computational work. I had a thought like 'wait, does this block my event loop?' two days later I find your excellent , succinct solution. Thanks again for the help. I'll circle back if I find anything down the road.
glad you found the video, hope it helps
okay, understood
is there a way for the worker to run local function instead of another script?
nodejs.org/api/worker_threads.html#new-workerfilename-options seems like it has to be a file
Watch at .75x speed
If im running the exact same app not locally but on GCP Cloud Run, would it still hang? I assumed each request spun up a new instance of your app thua nullifying any thread issues from the heavy endpoint.
Each request will not spin up a new instance of the app. I’m pretty sure that would only happen if you’re using a severless approach
🔥🔥🔥
wow a jewish youtuber, thats rare, as rare as seeing a rainbow.
not trying to be mean or anything just abit surprised, thats all, im jewish too so the more the better.
Hello bytheway Chaim, really great english too, without any hebrew accent as well.
thanks for the video, i'v found about worker threads through google ofc, while searching for cool things about nodejs.
didn't quite understand how to use them so i got here, very well explain and articulated. i'll save for future reference.
gg sir
While the explanation on how to use worker's is good,the example is really really bad.
If this is an Endpoint that is exposed to the client, there will be potentially hundreds of thousands of requests to that endpoint, so the node application will spawn that numbers of new threads with it's corresponding node instances, event loops etc, which is a nightmare in terms of performance.
So keep that in mind..use workers for heavy duty, yes, but not if it will be requested an inmensurable number of times
So, if You have to do some chained awaits that update a database. How do you handle correctly the endpoint for anlot of users? I thought about doing nothing and let AWS spawn more tasks copies behind a las balancer. AM i right?
Andrew Tate now teaches programming
it's ilegal bruh :"
Node Worker Threads, beautifully explained. Thanks.
{2024-01-22}
Wonderful explanation!!!! Thank You!
Beautiful explanation, thank you.