Important: The last part about using wait-notify is incorrect. My mistake, sorry about that. The object used to wait/notify should be the same object used by threads to synchronize. So correct code should be similar to: synchronize(sharedQ){ sharedQ.wait(); } synchronize(sharedQ){ sharedQ.notifyAll(); } That was a basic mistake which I should have caught. I feel bad about misleading the initial viewers. Sorry.
One more issue with the code is that Condition solution should use methods on Condition not Object. i.e. await signal. Not wait signal. Reference "Java Concurrency in Practice". Only one slide was wrong, no problem. Otherwise great video! Thanks Man!
I like your courses, however I don't understand this comment.. I keep getting exception about illegal monitor state.. what does sharedQ mean? Please explain or provide full code, which works :)
I generally don't put a comment for any video until I really like them. I must say that u explain each and everything very clearly. I don't have hands-on experience on multithreading and concurrency but with your videos, I find myself so comfortable with all these ..Keep uploading .after a long time I liked something good on youtube :)
This channel teaches more about the approach to solving the problem.They don't go to the solution straight away,first they come to common implemented pattern ,than they optimize the solution. Brilliant
Thank you for making this video, I watched it some few years back and I still came back to re-watch it in 2024. Thanks a lot for such a great teaching.
Hi @Defog Tech, This is the my first comment on youtube in last 5 years, Your videos are very helpful, I have really understood the Java concurrency on your channel. Please keep uploading such awesome content. Thanks
Awesome explanation and i always used to wonder while going through producer-consumer code, that why we use while statement to block the threads. Finally got the answer. Thankyou🙌
this video has cleared a lot of my doubts and the way you have explained will help me remembering it for a very very long time ! Thanks for your hard and crisp work !
I love your tutorials! I did this example myself and found a little but meaningful bug: -notEmpty and notFull should be called ".signal()" instead of ".signalAll()" because in case of releasing one spot in the queue by take() method, if there are multiple "putting" Threads, they are all notified at once about free space and they add concurently its own item exceeding the maxSize (and later condition `queue.size() == max` doesn't catch situation when this size is exceeded)
If we call signal() then the longest waiting thread will be notified and the rest of them will still be in a wait state. When they are notified they access the lock again and do a one on one insertion or removal of item.
8:50 instead of checking if queue.size==0 and then waiting on the condition, we must check that in a while loop because after resuming from the condition, another thread could have resumed before us, and the queue.size that we checked before would be outdated (the other thread that resumed before us took what was in the queue already)
why should we allow two threads to be in waiting in the first place? could you explain this? basically why java releases the lock after lock condition is false?
actually when it reaching the wait() at if statement, the the consumer is in blocked state and added to the waiting set. after being notified by the producer, it accquire the lock again and continue. If using while -> then continue checking the queue size since the next execution is under the wait() statement. If using if -> then jump out of if, then execute the remaining code. As mentioned, the next execution will be under the wait() statement.
Hi Defog_Tech. In case of multiple consumer threads at 9:00, how does thread-1 even get past the lock.lock() statement? Only thread-2 will go past that... I did not get how both threads were trying to execute near the try-block. Doesn't acquiring lock synchronize the method??
he said, java will release the lock once lock condition is false, this is both threads end being on the same statement. I also ddin't understand why java releases the lock.
synchronize(sharedQ){ sharedQ.wait(); } synchronize(sharedQ){ sharedQ.notifyAll(); } I am not getting these words In wait and notify section Could u please explain the code i am having trouble in this part ? java.lang.IllegalMonitorStateException at java.lang.Object.notifyAll(Native Method)
Great video! But I think there's a mistake at 9:37. As per official docs when the waiting threads are woken up they must re-acquire the lock to come out of await(). So only one thread(consumer) can poll from queue. So no NullPointerException will be thrown! No need to check queue size after await :)
Even I was thinking the same, otherwise it defeats the purpose of actual lock itself. "Signalall" means to just notify all waiting threads. The jvm will ensure only one thread is given the lock.
Super, This is the way to describe a tough subject like concurrency. Before watching this video i dont have any interest in concurrency. Now i think that its not so difficult. Thanks a lot
More videos from you will definitely help understand the technical concepts with clarity and ease. I know it will be little challenge for you to post more and more videos on core java,etc, while doing your regular work. But, please upload more videos which will be of great help. Keep doing the good work.
@@DefogTech Dude , I understand. Need not say sorry. We are eagerly waiting fo your content. I can consume any content from your channel. The explanations are so lucid that it gets stuck. I am more interested to see videos on distributed transactions and problems of joins in microservices and how to overcome them.
They do not enter the block at the same time, though after one thread exits the block the other thread will get the chance to enter. There is no way for that thread to know the queue is already empty without rechecking. Did I answer your question?
At 9:16, I am confused. When thread 1 acquires the lock after thread 2 has released it, the thread 1 will then check the statement "if queue.size() == 0" and in this case it is empty so thread 1 will wait at notEmpty.await() step. Am I missing something?
In case of 2 readers, 1 thread will acquire the lock, find that queue is empty, will wait and release the lock, after that, thread 2 will acquire the lock, find the queue empty, will wait and release the lock.. Thus we can have 2 threads both will continue after the if condition once the condition is satisfied. They dont restart from above the if statement, they only continue from where they left off.
Wait, just look back the code. It is ReentranceLock, so we can never have 2 threads enterring that block or reaching that if statement at the same time
@@hughgrant8749 They are not going in at the same time. But both of them continue (one after the other) after the if statement. So first thread will get the item, but second thread will get null. Both will acquire the lock properly. First thread takes lock, gets element releases it. Then Second thread takes lock, gets null..
@@DefogTech while the first thread is in the finally block releasing the lock, the second thread is waiting at lock.lock() line. after first thread releases it, the second one hits the if condition and waits because size is already zeroed by first. why do we need a while here?
Hi Deepak, Please start posting new videos. Your way of explaining the things in simple language helps to understand the concepts better. Great job! Looking forward to learn from your new videos
I like to add my humble opinion here. The while loop in the "take()" may allow N number of threads to enter the next line and which is removing the element. The first thread will remove the element successfully but the remaining threads will be ends up in Null. I would suggest using a slightly modified "Observer design pattern".
Complete Code for Producer Consumer Pattern using Locks: (which is explained in this tutorial) import java.util.LinkedList; import java.util.Queue; import java.util.concurrent.locks.Condition; import java.util.concurrent.locks.ReentrantLock; public class ProducerConsumerUsingLocks { public static void main(String[] args) { MyBlockingQueue queue = new MyBlockingQueue(10); Runnable producer = () -> { for (int i = 0; i < 20; i++) { System.out.println("Produced:" + i); queue.put(i); } };
Runnable consumer = () -> { for (int i = 0; i < 20; i++) { System.out.println("Consumed:" + queue.take()); } };
new Thread(producer).start(); new Thread(consumer).start(); } } class MyBlockingQueue { private Queue queue; private ReentrantLock lock = new ReentrantLock(true); private Condition notEmpty = lock.newCondition(); private Condition notFull = lock.newCondition(); private int max; public MyBlockingQueue(int size) { queue = new LinkedList(); this.max = size; } public void put(E e) { lock.lock(); try { while (queue.size() == max) { notFull.await(); } queue.add(e); notEmpty.signalAll(); } catch (InterruptedException e1) { // TODO Auto-generated catch block e1.printStackTrace(); } finally { lock.unlock(); } } public E take() { lock.lock(); E item = null; try { while (queue.size() == 0) { notEmpty.await(); } item = queue.remove(); notFull.signalAll(); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } finally { lock.unlock(); } return item; } } Thanks to the tutor.
Just started with going through Defog videos on Threads,Concurrency, etc.. Really a very clear explanation on these complex concepts. Also, could you please paste the final code you have implemented in the slides
Hi, You used while loop so as to restrict more than one consumer thread get into race condition when an item is put into queue. You are also using lock.lock() above... Do you think we really need while loop? and Do we really need lock.lock() when we are using conditions... Thanks.
Just a question sir , at 09:00, how come 2 threads be at await conditions .. since its inside lock . I presume that only one could go inside, take lock and await and others be waiting for lock only
Hi @Defog Why there is no while condition on put method It will have the same problem in adding the item in max index position Pls correct me if iam wrong
Hi Deepak, The complete code for implementing with Locks & Conditions @ 10:10 doesn't reflect the while loop which was discussed @ 9:50. Apologies if I missed something.
Unclear on how "while" condition is solving the issue of "if" condition. Time 10:00 Once second thread takes lock will it not execute the if condition ?
Yes, assume thread-1 took the lock, reached if (which evaluated to true) then went on to check the condition (await), at this point it is waiting for condition which is not true yet. So it cannot just hold the lock forever. If that happens other thread (producer) will never be able to acquire same lock and actually produce the item. This will become deadlock. Thus, if the Lock's Condition notEmpty() is not met, it releases the lock. And only the condition is true, it re-acquires the lock, but this time, it doesnt again start from the beginning. It resumes from where it left, which is Lock's Condition notEmpty() check.
@@DefogTech Thanks for your response.. I got it now.. Both threads would be at notEmpty.await() , and would release lock. Once they get the notify signal, one of the threads gets the lock, now it would start executing from line just after notEmpty.await(). In case we have "if" condition execution will directly go out of the "if" condition and proceed with removing elements from queue. Where as if we replace our "if" with "while" then once we get notify signal execution will evaluate queue.size condition before proceeding further ..
@9.04, how two thread could come at that point since we called lock above. I think only one thread will be in await and when the notEmpty.signalall is called that thread will remove the item and call the unlock so the other thread can come inside and do the if check and can go to the await state. What is the need to while there?
Great explanation on producer and consumer using different ways. It makes the idea really clear. I was wondering if there is any way to handle the fairness for consumers meaning if any consumer goes to wait-state first he will be notified first.
notEmpty and notFull not needed ..we can just use this.wait()... this implementation throw java.lang.IllegalMonitorStateException: current thread not owner... i tried it..
@Defog Tech i guess it should be while in put method also instead of if.Use case : when queue is full , two producers(A,B) try to put elements then they go into waiting, now one consumer wakes and take one element out. Then producer A will put one element and as soon as it leaves synchronized block producer B will try to insert and that will be successful which will exceeds queue size. Please confirm.Thanks for the explanation.
First of all thank you for making such a great playlist, by far the best on UA-cam in terms of concurrency. With respect to this video I have some doubts. May be I am missing some point but when you say at 9:05 that multiple threads will be waiting in if block condition, I don't think that's possible. Because we are using lock.lock at the very start of take method, there will be only one thread that will acquire that lock and will go inside and wait inside if condition, all the other threads will be blocked at lock.lock() statement only (because lock is a global lock in the blocking queue class, which is used for all the operations). Same is true for put method as well. Please correct me if I am wrong. (Am I missing something ? Only 1 thread can acquire the lock right ?) @Defog Tech
official documentation of await()- Causes the current thread to wait until it is signalled or interrupted. The lock associated with this Condition is atomically released and the current thread becomes disabled for thread scheduling purposes and lies dormant until one of four things happens:Some other thread invokes the signal method for this Condition and the current thread happens to be chosen as the thread to be awakened; or Some other thread invokes the signalAll method for this Condition; or Some other thread interrupts the current thread, and interruption of thread suspension is supported; or A "spurious wakeup" occurs.
I have similar problem where two threads need to read element by element.. i tried same concept but will not work. Two threads started but not moving further
This is a great explanation, but at 9:52 even though u add while loop there, thread 2 go to it,then the size of the queue is not empty ,then it go to remove the item and return,thread 2 will not go back to while loop again!so i hope u can explain it more,thx mate
I really want to thank you for this video, It taught me a lot of things Blocking queue, await-signal, wait-notify. But at the same time, the last mistake wasted a lot of time of mine. I should have seen this comment earlier. If possible, can you please mention this on the video itself. Maybe as some floating suggestion.
thank you for your videos they are helpful!! quick question, it might be better to use await instead of wait in the try block to avoid the monitorexception thing.
Another Gem of a video from you on java Concurrency. I've a question though, How does changing if condition to while would solve the problem ? Can you please elaborate ?
Could you elaborate on why we used while loop by removing the if condition? Thanks for the videos, they are very concise and up to the point. God Bless and Good Luck :)
But why the while loop is the solution for return null from collection. Previously it was if statement which is checking for size and both threads successfully pass this check. Why when we use while for check the size behavior cannot be the same and how it passed the if statement if only one thread can join the lock?
Why does java releases the lock when lock state is false. does this allow multiple threads to come in and wait? Because of this we need to solve the problem where multiple consumers trying to read the same item.
If we are taking same lock in both put and take method, then wouldn't it be that only one thread will do either put or take. If a thread is in put, no thread can go in take. And vice versa. Is that possible?
lots of questions around how come thread 2 acquires a lock once it releases. I think you didn't explain this properly. people has same question but you are replying them with whatever you know instead of clarifying. Our question is simple 9:50 where is logic to re acquire the lock after tread come out of wait state. does wait state acquires the lock for thread internally?
Correct, only 1 thread will acquire the lock. So let's say thread-1 got into lock block and queue is empty. It will do await (and release lock). Next thread-2 acquires the lock and queue is still empty and it will also do await and release lock. Now when producer thread does notifyAll, both threads will continue from where they left off. But only one thread will get the lock, will take item from queue and release lock. After that, other thread will also continue from same point and try to take item from queue but it is empty. Thus we need while condition instead of if condition to ensure even after getting the lock the queue still has items So even if only 1 thread can acquire the lock, Since threads continue from the statement where they left off, there can be issues.
@@DefogTech could you please explain this line further :- . "Now when producer thread does notifyAll, both threads will continue from where they left off." Exactly ,after notify ,how thread will acquire lock and execution line from where they will start execution
@@nitingoyal1000 When producer thread calls notifyAll, both consumer threads which were waiting for that condition will wake up from wait state, ready to continue from line of code when they stopped (and went to wait). But to continue from that line (which is within synchronized/locked block) both consumer threads will try to acquire the lock. Ofcourse only 1 will be able to acquire it first, that thread will continue and consume the item, then release the lock. Then second thread will get the lock, this thread will continue and try to consume the item (thus the need for while loop to ensure the item is still available to consume and some other thread has not already consumed it). Hope this helps clarify
@@DefogTech key to understand is that once inside the if body; you are waiting and when you get unblocked you come out of if block to the statement below the if block. With while statement, once you get out of the wait state you will go back to checking the condition in while to see if the statement is false. Since the statement is not false it will again go inside the while body and and go back into wait state and hence avoid reading null.
@@DefogTech "Ofcourse only 1 will be able to acquire it first, that thread will continue and consume the item, then release the lock" - at what line this happens? this is what everyone is confused about.
Great videos. it makes it easy to understand complex concepts. Though I have confused about using the while loop instead of the If condition. It could have been better to explain issues with if condition here and then introduce a while loop. anyway amazing explanation with exaples
@@DefogTechThanks so much for replying back. Some thing along the lines of Internal workings of a hashmap. When to use a list over a set. Performance in a multi threaded environment of these interfaces.
You used synchronized methods so lock was of this type, but wait and notify methods were called on object type, won't it throw illegalmotiorstate exception. Secondly why two objects/conditions for locks are required, can't we just take lock on whole queue?
Just Verified this. It indeed runs into IllegalMonitorState exception. A thread can only call wait() and notify() on an object of which it already holds the lock. Hope he corrects it soon.
I can see that you are first calling signalAll and then calling unlock. Shouldn't it be other way around? As you first unlock the resource and the let other threads know that the resource is available
There is a problem. When we have our queue full and it's in await state then the consumer consumes one and signals all. in that case, if we have multiple producers then our queue size goes beyond the max capacity. Try this with 3 producers and 1 consumer.
Great video! Thanks for the explanation. I checked your description regarding wait() and notifyAll() and understood. However, I have one question, whenever we call sharedQ.notifyAll(), it will wake up all producer and consumer threads waiting on sharedQ. Is it possible using wait/notify method, that producer thread wakes up only consumer threads and vice versa, consumer thread wakes up only producer threads?
The solution of locks I think is only suitable if we have multiple producers or multiple consumers. In case if there is single producer and single consumer we dont need to make put and remove method synchronized right? @Defog Tech
omg such a simple explanation for a concept we always are reluctant to read... i dint know it was too simple...one query though when replaceing the await/signal with wait/notify , in the code snippet i dint see the usage of while. is it not required to put the "while" instead of "if"
Your explanations are wonderful! Thanks a lot! I have just one comment.. You are calling "wait()" method on both "notFull" and "notEmpty" conditions. I guess you've meant to call "await()" instead, right?
One more comment.. :) Why do we need to notify each time we are consuming or producing? Isn't it better to query if the queue has a single element in case of consumer and if the queue has a single index available before notifying? Thanks again..
But since threads can work independently and time taken for producing an item and for consuming (processing) an item can vary.. it's bwtter threads notify each other instead of them constantly polling for available items
@@DefogTech Let me explain myself.. I think that the following consumer notification: notFull.signalAll(); should be wrapped with the following checking: if (queue.size() == maxSize - 1) { notFull.signalAll(); } or. on the other hand, the notification of the producer: notEmpty.signalAll(); should also be wrapped: if (queue.size() == 1) { notEmpty.signalAll(); } otherwise we are notifying the other threads many times where most of the notifications are not necessary (say for example if consumer took an item and now there are more than one index available, why does he need to notify the producer which wait just for one available index?) thanks
At 9th min, where you replaced "if" with a "while", I personally feel its unnecessary, because the second thread will go in await as the queue.size() will be 0. It will not get any null value or I am missing something.
Important: The last part about using wait-notify is incorrect. My mistake, sorry about that.
The object used to wait/notify should be the same object used by threads to synchronize. So correct code should be similar to:
synchronize(sharedQ){
sharedQ.wait();
}
synchronize(sharedQ){
sharedQ.notifyAll();
}
That was a basic mistake which I should have caught. I feel bad about misleading the initial viewers. Sorry.
One more issue with the code is that Condition solution should use methods on Condition not Object. i.e. await signal. Not wait signal. Reference "Java Concurrency in Practice".
Only one slide was wrong, no problem.
Otherwise great video! Thanks Man!
I like your courses, however I don't understand this comment.. I keep getting exception about illegal monitor state.. what does sharedQ mean? Please explain or provide full code, which works :)
can you provide source code with corrected wait/notify?
can u provide code of wait and notify section
can i call wait and notifyAll on this
I generally don't put a comment for any video until I really like them. I must say that u explain each and everything very clearly. I don't have hands-on experience on multithreading and concurrency but with your videos, I find myself so comfortable with all these ..Keep uploading .after a long time I liked something good on youtube :)
Thank you so much for your kind words! Such wonderful feedback keeps me going too
Even years later, thank you for this video, it 's very useful and helped me finally understand this topic... sincere thanks
fr
This channel teaches more about the approach to solving the problem.They don't go to the solution straight away,first they come to common implemented pattern ,than they optimize the solution.
Brilliant
Thank you for making this video, I watched it some few years back and I still came back to re-watch it in 2024. Thanks a lot for such a great teaching.
Greetings from Russia , having watched hours of content on concurrency/multithreading , this is the by far the best one i have seen , top notch.
Hi @Defog Tech, This is the my first comment on youtube in last 5 years, Your videos are very helpful, I have really understood the Java concurrency on your channel. Please keep uploading such awesome content. Thanks
this is the only channel that make me realize multithreading can be learnt thanks a lot man keep making more of such gems 😀
first time learning about producer-consumer. your explanation is very clear and easy to understand
GENIUS !! No other words. Such a smooth and clear explanation to the problem
Awesome explanation and i always used to wonder while going through producer-consumer code, that why we use while statement to block the threads. Finally got the answer.
Thankyou🙌
your videos are crisp and clear! that's the way to talk about complex topics.. thank you so much man!
Hi, I have one doubt.
At 8:52
How can two threads simulatenaosly access the code put after acquiring the lock?
Am I missing anything?
I had the same question in mind. Putting that while condition seemed unnecessary and impossible scenario.
It looks like await is taking the lock off. But how then it is going to protect integrity of the queue?
Correct ... it's not possible.
samething i got.
Same question..if lock if there an if would have been sifficient.
this video has cleared a lot of my doubts and the way you have explained will help me remembering it for a very very long time ! Thanks for your hard and crisp work !
I love your tutorials! I did this example myself and found a little but meaningful bug:
-notEmpty and notFull should be called ".signal()" instead of ".signalAll()" because in case of releasing one spot in the queue by take() method, if there are multiple "putting" Threads, they are all notified at once about free space and they add concurently its own item exceeding the maxSize (and later condition `queue.size() == max` doesn't catch situation when this size is exceeded)
Great spot. Even I thought of the same.
If we call signal() then the longest waiting thread will be notified and the rest of them will still be in a wait state.
When they are notified they access the lock again and do a one on one insertion or removal of item.
Easy explanation of any topic. I really like your way of explaining any topic.
Best clear explanation of a very complicated subject
8:50 instead of checking if queue.size==0 and then waiting on the condition, we must check that in a while loop because after resuming from the condition, another thread could have resumed before us, and the queue.size that we checked before would be outdated (the other thread that resumed before us took what was in the queue already)
why should we allow two threads to be in waiting in the first place? could you explain this? basically why java releases the lock after lock condition is false?
actually when it reaching the wait() at if statement, the the consumer is in blocked state and added to the waiting set.
after being notified by the producer, it accquire the lock again and continue.
If using while -> then continue checking the queue size since the next execution is under the wait() statement.
If using if -> then jump out of if, then execute the remaining code. As mentioned, the next execution will be under the wait() statement.
Hi Defog_Tech.
In case of multiple consumer threads at 9:00, how does thread-1 even get past the lock.lock() statement?
Only thread-2 will go past that... I did not get how both threads were trying to execute near the try-block. Doesn't acquiring lock synchronize the method??
he said, java will release the lock once lock condition is false, this is both threads end being on the same statement. I also ddin't understand why java releases the lock.
This channel always have unique topics and content.
BEAUTIFUL! Spoken way better than my professor
Bro you are best guru out there . Put more contents your channel will bloom
this is the best video I have seen ever for producer consumer problem ..thanxx man
Brilliantly explained!
It couldn't be better in any aspect.
No new tutorials for a long time on threads, May God bless you. Will try to share your channel with my mates, but please do not stop illuminating us.
Best videos of all time and even youtube understood that and suggesting on top of everyone
A perfect explanation I had ever seen. Thanks a lot!
Hi , I have been following all the videos from you which are really great in content with clarity . Would love to see more from you .
your hard work really appreciated expecting more Complex multithreading interview Q&A
synchronize(sharedQ){
sharedQ.wait();
}
synchronize(sharedQ){
sharedQ.notifyAll();
}
I am not getting these words
In wait and notify section
Could u please explain the code i am having trouble in this part ?
java.lang.IllegalMonitorStateException
at java.lang.Object.notifyAll(Native Method)
Great video! But I think there's a mistake at 9:37. As per official docs when the waiting threads are woken up they must re-acquire the lock to come out of await(). So only one thread(consumer) can poll from queue. So no NullPointerException will be thrown! No need to check queue size after await :)
Even I was thinking the same, otherwise it defeats the purpose of actual lock itself. "Signalall" means to just notify all waiting threads. The jvm will ensure only one thread is given the lock.
you're wrong
Super, This is the way to describe a tough subject like concurrency. Before watching this video i dont have any interest in concurrency. Now i think that its not so difficult. Thanks a lot
I saw your correction. The whole thing is now very clear. Thank you.
More videos from you will definitely help understand the technical concepts with clarity and ease. I know it will be little challenge for you to post more and more videos on core java,etc, while doing your regular work. But, please upload more videos which will be of great help. Keep doing the good work.
Thank you for the encouragement! I sure am planning to restart uploading first week of Jan 2020!
@@DefogTech Any update on this?
@@VijayKumar-vv6yw sorry sir, was stuck.. can you please let me know the topics you are interested in...
sorry about the delay!
@@DefogTech Dude , I understand. Need not say sorry. We are eagerly waiting fo your content. I can consume any content from your channel. The explanations are so lucid that it gets stuck. I am more interested to see videos on distributed transactions and problems of joins in microservices and how to overcome them.
I don't understand if there was a lock, how did two threads enter the method?
Exactly, I asked the same too.
They do not enter the block at the same time, though after one thread exits the block the other thread will get the chance to enter. There is no way for that thread to know the queue is already empty without rechecking. Did I answer your question?
You made it so easy to understand. Thank you very much!
At 9:16, I am confused. When thread 1 acquires the lock after thread 2 has released it, the thread 1 will then check the statement "if queue.size() == 0" and in this case it is empty so thread 1 will wait at notEmpty.await() step. Am I missing something?
In case of 2 readers, 1 thread will acquire the lock, find that queue is empty, will wait and release the lock, after that, thread 2 will acquire the lock, find the queue empty, will wait and release the lock..
Thus we can have 2 threads both will continue after the if condition once the condition is satisfied. They dont restart from above the if statement, they only continue from where they left off.
@@DefogTech thanks much, I get it now.
Wait, just look back the code. It is ReentranceLock, so we can never have 2 threads enterring that block or reaching that if statement at the same time
@@hughgrant8749 They are not going in at the same time. But both of them continue (one after the other) after the if statement. So first thread will get the item, but second thread will get null. Both will acquire the lock properly. First thread takes lock, gets element releases it. Then Second thread takes lock, gets null..
@@DefogTech while the first thread is in the finally block releasing the lock, the second thread is waiting at lock.lock() line. after first thread releases it, the second one hits the if condition and waits because size is already zeroed by first. why do we need a while here?
Hi Deepak, Please start posting new videos. Your way of explaining the things in simple language helps to understand the concepts better. Great job! Looking forward to learn from your new videos
you are the best.. please keep up doing the good work..
I like to add my humble opinion here. The while loop in the "take()" may allow N number of threads to enter the next line and which is removing the element. The first thread will remove the element successfully but the remaining threads will be ends up in Null. I would suggest using a slightly modified "Observer design pattern".
Thanks for this interesting Java Concurrency series.
Complete Code for Producer Consumer Pattern using Locks: (which is explained in this tutorial)
import java.util.LinkedList;
import java.util.Queue;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;
public class ProducerConsumerUsingLocks {
public static void main(String[] args) {
MyBlockingQueue queue = new MyBlockingQueue(10);
Runnable producer = () -> {
for (int i = 0; i < 20; i++) {
System.out.println("Produced:" + i);
queue.put(i);
}
};
Runnable consumer = () -> {
for (int i = 0; i < 20; i++) {
System.out.println("Consumed:" + queue.take());
}
};
new Thread(producer).start();
new Thread(consumer).start();
}
}
class MyBlockingQueue {
private Queue queue;
private ReentrantLock lock = new ReentrantLock(true);
private Condition notEmpty = lock.newCondition();
private Condition notFull = lock.newCondition();
private int max;
public MyBlockingQueue(int size) {
queue = new LinkedList();
this.max = size;
}
public void put(E e) {
lock.lock();
try {
while (queue.size() == max) {
notFull.await();
}
queue.add(e);
notEmpty.signalAll();
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} finally {
lock.unlock();
}
}
public E take() {
lock.lock();
E item = null;
try {
while (queue.size() == 0) {
notEmpty.await();
}
item = queue.remove();
notFull.signalAll();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
lock.unlock();
}
return item;
}
}
Thanks to the tutor.
how can 2 threads be in a lock.lock() block? time 9:05 on the video.
Just started with going through Defog videos on Threads,Concurrency, etc.. Really a very clear explanation on these complex concepts. Also, could you please paste the final code you have implemented in the slides
Hi,
You used while loop so as to restrict more than one consumer thread get into race condition when an item is put into queue. You are also using lock.lock() above... Do you think we really need while loop? and Do we really need lock.lock() when we are using conditions... Thanks.
Just a question sir , at 09:00, how come 2 threads be at await conditions .. since its inside lock . I presume that only one could go inside, take lock and await and others be waiting for lock only
really liked the way you explained. thumsup. God bless you
very very logical explaining..thnkx brother for ur wonderful efforts..
Hi @Defog
Why there is no while condition on put method
It will have the same problem in adding the item in max index position
Pls correct me if iam wrong
Neat and clear explanation...
Hi Deepak, The complete code for implementing with Locks & Conditions @ 10:10 doesn't reflect the while loop which was discussed @ 9:50. Apologies if I missed something.
You're right. I missed it in the final code. It should have while loop. Thanks for pointing it out!
Best java channel ever
Unclear on how "while" condition is solving the issue of "if" condition. Time 10:00
Once second thread takes lock will it not execute the if condition ?
Yeah right i am also confused here ...if lock is acquired by thread 1.how second thread will go into if statement.
Yes, assume thread-1 took the lock, reached if (which evaluated to true) then went on to check the condition (await), at this point it is waiting for condition which is not true yet. So it cannot just hold the lock forever. If that happens other thread (producer) will never be able to acquire same lock and actually produce the item.
This will become deadlock.
Thus, if the Lock's Condition notEmpty() is not met, it releases the lock. And only the condition is true, it re-acquires the lock, but this time, it doesnt again start from the beginning. It resumes from where it left, which is Lock's Condition notEmpty() check.
@@DefogTech Thanks for your response.. I got it now..
Both threads would be at notEmpty.await() , and would release lock.
Once they get the notify signal, one of the threads gets the lock, now it would start executing from line just after notEmpty.await().
In case we have "if" condition execution will directly go out of the "if" condition and proceed with removing elements from queue.
Where as if we replace our "if" with "while" then once we get notify signal execution will evaluate queue.size condition before proceeding further ..
@9.04, how two thread could come at that point since we called lock above. I think only one thread will be in await and when the notEmpty.signalall is called that thread will remove the item and call the unlock so the other thread can come inside and do the if check and can go to the await state. What is the need to while there?
I still don't understand how can two consumer threads wait on the condition because we have been using a lock over that method in the first place.
Great explanation on producer and consumer using different ways. It makes the idea really clear. I was wondering if there is any way to handle the fairness for consumers meaning if any consumer goes to wait-state first he will be notified first.
notEmpty and notFull not needed ..we can just use this.wait()... this implementation throw java.lang.IllegalMonitorStateException: current thread not owner... i tried it..
Thanks
Thank you for the support! Really appreciate it
@Defog Tech i guess it should be while in put method also instead of if.Use case : when queue is full , two producers(A,B) try to put elements then they go into waiting, now one consumer wakes and take one element out. Then producer A will put one element and as soon as it leaves synchronized block producer B will try to insert and that will be successful which will exceeds queue size. Please confirm.Thanks for the explanation.
First of all thank you for making such a great playlist, by far the best on UA-cam in terms of concurrency.
With respect to this video I have some doubts. May be I am missing some point but when you say at 9:05 that multiple threads will be waiting in if block condition, I don't think that's possible. Because we are using lock.lock at the very start of take method, there will be only one thread that will acquire that lock and will go inside and wait inside if condition, all the other threads will be blocked at lock.lock() statement only (because lock is a global lock in the blocking queue class, which is used for all the operations). Same is true for put method as well. Please correct me if I am wrong. (Am I missing something ? Only 1 thread can acquire the lock right ?) @Defog Tech
official documentation of await()- Causes the current thread to wait until it is signalled or interrupted.
The lock associated with this Condition is atomically released and the current thread becomes disabled for thread scheduling purposes and lies dormant until one of four things happens:Some other thread invokes the signal method for this Condition and the current thread happens to be chosen as the thread to be awakened; or
Some other thread invokes the signalAll method for this Condition; or
Some other thread interrupts the current thread, and interruption of thread suspension is supported; or
A "spurious wakeup" occurs.
I have similar problem where two threads need to read element by element.. i tried same concept but will not work. Two threads started but not moving further
Thanks!
Very well explained thank you !
This is a great explanation, but at 9:52 even though u add while loop there, thread 2 go to it,then the size of the queue is not empty ,then it go to remove the item and return,thread 2 will not go back to while loop again!so i hope u can explain it more,thx mate
agree, he did poor job at 9:00
I really want to thank you for this video, It taught me a lot of things Blocking queue, await-signal, wait-notify.
But at the same time, the last mistake wasted a lot of time of mine. I should have seen this comment earlier.
If possible, can you please mention this on the video itself. Maybe as some floating suggestion.
thank you for your videos they are helpful!!
quick question, it might be better to use await instead of wait in the try block to avoid the monitorexception thing.
Another Gem of a video from you on java Concurrency. I've a question though, How does changing if condition to while would solve the problem ? Can you please elaborate ?
why cant we use a semaphore instead of lock? we can restrict access to resources that is in this case our queue.
Could you elaborate on why we used while loop by removing the if condition? Thanks for the videos, they are very concise and up to the point. God Bless and Good Luck :)
Which is the better way of implementation of blocking queue? "Lock and Condition" or "Wait-Notify" and why?
But why the while loop is the solution for return null from collection. Previously it was if statement which is checking for size and both threads successfully pass this check.
Why when we use while for check the size behavior cannot be the same and how it passed the if statement if only one thread can join the lock?
The await condition suspends the lock.
Thanks for this video. Very beautifully explained.
Why does java releases the lock when lock state is false. does this allow multiple threads to come in and wait? Because of this we need to solve the problem where multiple consumers trying to read the same item.
Wonderful explaination!!! Thanks a ton.
I am getting illegalMonitorstateException. Any idea when it occurs? I followed the same approach.
If we are taking same lock in both put and take method, then wouldn't it be that only one thread will do either put or take.
If a thread is in put, no thread can go in take. And vice versa.
Is that possible?
lots of questions around how come thread 2 acquires a lock once it releases. I think you didn't explain this properly. people has same question but you are replying them with whatever you know instead of clarifying. Our question is simple 9:50 where is logic to re acquire the lock after tread come out of wait state. does wait state acquires the lock for thread internally?
At 10.06 even an if would have checked size of q for the new thread..why there is need of while?
@Defog Tech what is the differce using the "while" or "if"?Because both threads are wait state after the condition. Is "while" re-check the statement?
Nice explanation. But one question. How does it possible to have more than one threads at 8:54 if we use lock?
Correct, only 1 thread will acquire the lock. So let's say thread-1 got into lock block and queue is empty. It will do await (and release lock). Next thread-2 acquires the lock and queue is still empty and it will also do await and release lock. Now when producer thread does notifyAll, both threads will continue from where they left off. But only one thread will get the lock, will take item from queue and release lock. After that, other thread will also continue from same point and try to take item from queue but it is empty. Thus we need while condition instead of if condition to ensure even after getting the lock the queue still has items
So even if only 1 thread can acquire the lock, Since threads continue from the statement where they left off, there can be issues.
@@DefogTech could you please explain this line further :- . "Now when producer thread does notifyAll, both threads will continue from where they left off."
Exactly ,after notify ,how thread will acquire lock and execution line from where they will start execution
@@nitingoyal1000 When producer thread calls notifyAll, both consumer threads which were waiting for that condition will wake up from wait state, ready to continue from line of code when they stopped (and went to wait). But to continue from that line (which is within synchronized/locked block) both consumer threads will try to acquire the lock. Ofcourse only 1 will be able to acquire it first, that thread will continue and consume the item, then release the lock. Then second thread will get the lock, this thread will continue and try to consume the item (thus the need for while loop to ensure the item is still available to consume and some other thread has not already consumed it).
Hope this helps clarify
@@DefogTech key to understand is that once inside the if body; you are waiting and when you get unblocked you come out of if block to the statement below the if block. With while statement, once you get out of the wait state you will go back to checking the condition in while to see if the statement is false. Since the statement is not false it will again go inside the while body and and go back into wait state and hence avoid reading null.
@@DefogTech "Ofcourse only 1 will be able to acquire it first, that thread will continue and consume the item, then release the lock" - at what line this happens? this is what everyone is confused about.
Great videos. it makes it easy to understand complex concepts. Though I have confused about using the while loop instead of the If condition. It could have been better to explain issues with if condition here and then introduce a while loop. anyway amazing explanation with exaples
Very Good and clear explanation! Can you make a video on Advanced Collections and how to correctly incorporate them in your System design?
Sounds like a good idea, any specific collections you are thinking about?
@@DefogTechThanks so much for replying back. Some thing along the lines of Internal workings of a hashmap. When to use a list over a set. Performance in a multi threaded environment of these interfaces.
You used synchronized methods so lock was of this type, but wait and notify methods were called on object type, won't it throw illegalmotiorstate exception. Secondly why two objects/conditions for locks are required, can't we just take lock on whole queue?
Just Verified this. It indeed runs into IllegalMonitorState exception. A thread can only call wait() and notify() on an object of which it already holds the lock. Hope he corrects it soon.
@@NilanshuSharma1 Good catch, thats my bad! Will highlight this as an error
My bad. Thanks for pointing this out @Anish Agarwal. I should have caught this much earlier.
When Using ReentrantLock why not same object we are using for lock and unlock.
I can see that you are first calling signalAll and then calling unlock. Shouldn't it be other way around? As you first unlock the resource and the let other threads know that the resource is available
Awesome video, it would have been great if you could add the link to the source code too.
There is a problem. When we have our queue full and it's in await state then the consumer consumes one and signals all. in that case, if we have multiple producers then our queue size goes beyond the max capacity. Try this with 3 producers and 1 consumer.
Great video! Thanks for the explanation.
I checked your description regarding wait() and notifyAll() and understood.
However, I have one question, whenever we call sharedQ.notifyAll(), it will wake up all producer and consumer threads waiting on sharedQ.
Is it possible using wait/notify method, that producer thread wakes up only consumer threads and vice versa, consumer thread wakes up only producer threads?
The solution of locks I think is only suitable if we have multiple producers or multiple consumers. In case if there is single producer and single consumer we dont need to make put and remove method synchronized right? @Defog Tech
Great Video, keep up the good work!
omg such a simple explanation for a concept we always are reluctant to read... i dint know it was too simple...one query though when replaceing the await/signal with wait/notify , in the code snippet i dint see the usage of while. is it not required to put the "while" instead of "if"
Option 3 with semaphore would have been great. Thank you.
Hi Mate; error correction: you should use notEmpty.signal() and not notEmpty.signalAll()
In Lock and Condition mechanism, You are showing with "conditionObject.wait()" which is wrong ,Please correct it.
How can we add a new item if the lock is held by take method and we can't enter the put method?
Thanx u bro
very nice explanation.
Your explanations are wonderful!
Thanks a lot!
I have just one comment.. You are calling "wait()" method on both "notFull" and "notEmpty" conditions.
I guess you've meant to call "await()" instead, right?
yes thats right..
I was also wondering why await() suddenly becomes wait()
Nicely explained. Thank you
Clear explanation!!
One more comment.. :)
Why do we need to notify each time we are consuming or producing?
Isn't it better to query if the queue has a single element in case of consumer and if the queue has a single index available before notifying?
Thanks again..
But since threads can work independently and time taken for producing an item and for consuming (processing) an item can vary.. it's bwtter threads notify each other instead of them constantly polling for available items
@@DefogTech Let me explain myself.. I think that the following consumer notification:
notFull.signalAll();
should be wrapped with the following checking:
if (queue.size() == maxSize - 1) {
notFull.signalAll();
}
or. on the other hand, the notification of the producer:
notEmpty.signalAll();
should also be wrapped:
if (queue.size() == 1) {
notEmpty.signalAll();
}
otherwise we are notifying the other threads many times where most of the notifications are not necessary (say for example if consumer took an item and now there are more than one index available, why does he need to notify the producer which wait just for one available index?)
thanks
At 9th min, where you replaced "if" with a "while", I personally feel its unnecessary, because the second thread will go in await as the queue.size() will be 0. It will not get any null value or I am missing something.