Respected Sir, I am working as Assistant Professor in Engineering College for the department of Computer Science and Engineering, As I was unable to understand this concept by reading number of books,since from one year, but your video of 9.25 minutes has made very comfortable understanding of this concept ID3 decision tree. So once again I am very grateful for you, for sharing your knowledge and videos at the free of cost sir. I am hats off to you and your University sir.
Thank you very much.. I read few papers and more I read, more confusing it went. You made it crystal clear with 'real life example explained it very effective way'. Thanks again.
wow..your explanations are so helpful. I have just gone a few minutes into your lecture and i'm excited! I know i will be here for a while.Will learn everything I can. Thanks again.
Studying at TUM. I admire german students who are following the lecture contents from the uni. Taking ML course atm, but here, the lecture is just like dumping only the whole concepts, regardless of whether students can understand them or not... So nice explanations in every video in ML related playlist.. I fcking regret that I did not choose the UK to study my master's.
What if one of the nodes never forms a pure subset? How do we make decisions then? Also, what if there are not a single pure subset in the decision tree?
We decide my measuring the purity of split. Purity can be measured by calculating Entropy. Low entropy means low uncertainty and it leads to high purity. Entropy value 0 means high purity and Entropy value 1 means smallest purity. So we make our decision which attribute to split on by calculating Entropy.
entropy can't be higher than 1, so I think your math is off. An entropy of 1 would mean that the data is half true and half false. If it was 0 then that would mean that the entire set or subset would be the same, all true or all false, it would be pure.
In the provided example variable Outlook has three possible values: {sunny, overcast, rain}. In simple case when each of those values has Pr = 1/3, the entropy of this distribution will be >1. So, yes, it can, even though I didn't check his math.
I love this explanation!! I have just one comment: At min 8:00 ".... and in all other cases John will not play.." This statement ist wrong, because if you have attributes that are not recorded yet, you can make no decision about how John will play or not :-) But this statement is true, if we assume that the dataset is correct and complete and is only used for all decision. I just want to make clear, that if you look at attributes that are not available, for example, if John plays when he is sick or not, you can make no founded decision.
Hello, thank you for simple explanation, however, i have a doubt from(6.20), when we consider the candidates for sub nodes of "rain" should we consider both "Humidity" and "wind"? or just "wind" ?. pls explain
I recently did a video on decision trees and I think I am referring to your question in this video ua-cam.com/video/WlGuizdVaiY/v-deo.html at 11:35. Also, I created a video series on how to code a decision tree from scratch in python. Maybe, that’s also of interest to you. Anyway, here is the link: ua-cam.com/play/PLPOTBrypY74xS3WD0G_uzqPjCQfU6IRK-.html
A pure subset will always be possible, if you have enough branches in your tree. Like someone already commented, a subset of one item is pure. But, a decision tree that has too many leaves is not a good model. If you limited the example tree in the video to a depth of 0 (in other words, nothing deeper than the root node), then you would split a new data entry based on the weather, and then stop and decide if John will play, yes or no. If it's overcast, the tree will decide he played. If it's sunny, the tree will decide he didn't play, and if it rained then the tree will decide that he did play. But remember how the guy in the video said that each node keeps track of the count of each class at each node? The amount of confidence the tree has in each of those decisions won't be the same. Sunny days have 2 Yeses and 3 Nos, right? So the most likely class, if you don't split the data again, is NO, he won't play. But the confidence will be low because it's a very impure subset. And vice versa for rain (3 Yeses, 2 Nos). Kinda funny that the tree works out this way since you would think one would be more likely to play on a sunny day, and less likely on a rainy day lol
Great video Any suggestions on how to analytically make career desicion that does not possess statistical probability My thought was a payoff table but I don't know how to do it without percentages Suggestions?
I agree with others that this is by far one of the easiest, yet pretty informational video lecture on Decision Tree. I know there are numerous resources to refer, but what would be your recommendation if you were to read a little more in-depth about the topic and do some hands on practice?
All of your videos are great, i could have never understood these concepts from books.. Do you have any complete playlist covering all topics of Machine Learning?
I don't think he would play in Rain w/ High Humidity. The sequence in which you chose to evaluate the attribute of "Wind" was random...had you chosen Rain-->Humidity--> I don't think he would have played :-)
I guess there are many tools, among them rapidminer and openoffice softwares, you can look at this document, it is very helpful and practical docs.rapidminer.com/downloads/DataMiningForTheMasses.pdf
Vikram Jadhav We use the following textbook at Edinburgh: www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0123748569 But there are better textbooks by Chris Bishop, David Barber, and Hastie et al.
Thank u for explaining it in so simple way . but here you didn't considered the humidity factor while taking the decision.. :/ . It would have been better if u have included of selection of the attribute factor in this slide itself.anyway thank u
The best example of decision tree generation algorithm I've seen so far, thank you
You are good. 6 years after production and I am utmost grateful for this gift
Great didactics. Your explained it in really comprehensible way, without missing the point, something that very few lecturers can actually do.
Respected Sir,
I am working as Assistant Professor in Engineering College for the department of Computer Science and Engineering, As I was unable to understand this concept by reading number of books,since from one year, but your video of 9.25 minutes has made very comfortable understanding of this concept ID3 decision tree.
So once again I am very grateful for you, for sharing your knowledge and videos at the free of cost sir. I am hats off to you and your University sir.
By far the best and easiest to understand description of decision tree based machine learning on the internet. Thank you Victor for sharing!
Thanks! Happy you found it helpful.
This playlist is by far the best (and very thorough) explanation of decision trees I've come across. Thank you so much for posting this!
This was not only the best example but also had the best explanation, thank u sir!
This lecture simply follows Occam's razor.
Lecture is simple , and that is why it is too good.
Simply tells the concepts.
You helped me.
This was so so SO helpful. SImple and clear to understand. THANK YOU!
Professor Lavrenko
I deeply appreciate your great explanation and simple example.
thank you
So much better explained then in my books for the university, thank a lot mate!!!
2019 and from all the videos I have watched that discuss this topic, this clip still gets it done the best way. Thank you.
SO BLOODY SIMPLE WHEN YOU EXPLAIN IT!
cheers
Thank you very much.. I read few papers and more I read, more confusing it went. You made it crystal clear with 'real life example explained it very effective way'. Thanks again.
You just explained what my assistant couldn't in just 10 min! Thanks a lot!
You made Decision Trees look so easy to solve. Thank you so much, sir.
wow..your explanations are so helpful. I have just gone a few minutes into your lecture and i'm excited! I know i will be here for a while.Will learn everything I can. Thanks again.
the best and the most helpful professor i've ever seen. Thanks a lot Pr @Victor Lavrenko
That such an easy way to understand .. Even easier than our Dr. who I can't almost understand anything from her! many thanks
It is so easy to understand just with one great example. Great job!
I am learning from your classes very easily! Thank you very much dude!
Thank you for making this concept so simple. Truly amazing :)
Excellent explanation Victor, and the example very appropriated...
Thank you!
Thanks, Victor. You made a perfect sense for Decision Tree concept
Thanks alot Sir. I wanted to learn decision trees. And this is just what i wanted. your way of teaching is superb..!!
Thank you Victor for such a great example and explanation
Clear and beautiful explanation.....
Great lecture, however your volume is very, very low. Thanks!
Studying at TUM. I admire german students who are following the lecture contents from the uni.
Taking ML course atm, but here, the lecture is just like dumping only the whole concepts, regardless of whether students can understand them or not...
So nice explanations in every video in ML related playlist..
I fcking regret that I did not choose the UK to study my master's.
wait, the new data is Rain, High and Weak, isnt the class is YES ? cause in D4 said same too
What if one of the nodes never forms a pure subset? How do we make decisions then?
Also, what if there are not a single pure subset in the decision tree?
maybe you should check another algorithm..
Why did you decide to split it on "outlook"??
Why not start splitting them on "Humidity" or "Wind"??
We decide my measuring the purity of split. Purity can be measured by calculating Entropy. Low entropy means low uncertainty and it leads to high purity. Entropy value 0 means high purity and Entropy value 1 means smallest purity. So we make our decision which attribute to split on by calculating Entropy.
But the entropy for outlook is 1.578, while it is 1 for humidity, and 0.985 for wind. Wouln't a split on outlook be the worst one to pick?
entropy can't be higher than 1, so I think your math is off. An entropy of 1 would mean that the data is half true and half false. If it was 0 then that would mean that the entire set or subset would be the same, all true or all false, it would be pure.
In the provided example variable Outlook has three possible values: {sunny, overcast, rain}. In simple case when each of those values has Pr = 1/3, the entropy of this distribution will be >1. So, yes, it can, even though I didn't check his math.
@WAI HO YAU www.saedsayad.com/decision_tree.htm find information here :)
What if the humidity had been High on a Sunny day in which John played?
or
What if the wind was weak on a Rainy day in which john has not played?
THANKS VICTOR , REALLY APPLAUDED
I love this explanation!!
I have just one comment:
At min 8:00
".... and in all other cases John will not play.."
This statement ist wrong, because if you have attributes that are not recorded yet, you can make no decision about how John will play or not :-)
But this statement is true, if we assume that the dataset is correct and complete and is only used for all decision.
I just want to make clear, that if you look at attributes that are not available, for example, if John plays when he is sick or not, you can make no founded decision.
thank you, that was a Great explanation .
D4 is similar to D15, which makes it easy to predict that Jone will play in D15
Hello, thank you for simple explanation, however, i have a doubt from(6.20), when we consider the candidates for sub nodes of "rain" should we consider both "Humidity" and "wind"? or just "wind" ?. pls explain
Thanks for a great lecture. Very clear, entertaining and informative!
Excellent Victor. You are great.
Amazing introduction. Thanks a lot for publishing!
What happens if a pure subset is not possible from the given data?
i was hoping to hear the answer to this for the whole video...
I guess if you get to subset of one item, it will be pure.
any update?
I recently did a video on decision trees and I think I am referring to your question in this video ua-cam.com/video/WlGuizdVaiY/v-deo.html at 11:35.
Also, I created a video series on how to code a decision tree from scratch in python. Maybe, that’s also of interest to you. Anyway, here is the link: ua-cam.com/play/PLPOTBrypY74xS3WD0G_uzqPjCQfU6IRK-.html
A pure subset will always be possible, if you have enough branches in your tree. Like someone already commented, a subset of one item is pure. But, a decision tree that has too many leaves is not a good model. If you limited the example tree in the video to a depth of 0 (in other words, nothing deeper than the root node), then you would split a new data entry based on the weather, and then stop and decide if John will play, yes or no. If it's overcast, the tree will decide he played. If it's sunny, the tree will decide he didn't play, and if it rained then the tree will decide that he did play. But remember how the guy in the video said that each node keeps track of the count of each class at each node? The amount of confidence the tree has in each of those decisions won't be the same. Sunny days have 2 Yeses and 3 Nos, right? So the most likely class, if you don't split the data again, is NO, he won't play. But the confidence will be low because it's a very impure subset. And vice versa for rain (3 Yeses, 2 Nos). Kinda funny that the tree works out this way since you would think one would be more likely to play on a sunny day, and less likely on a rainy day lol
I think you did a grt job, making it very simple.
Hi Victor, It was a wonderful explanation
Very good example and explanation!
Thanks!
Very well explained. The example you have chosen is awesome. Merci
Sudheer Rao Thank you! Happy to know you found it useful.
Very informative and easy to follow. Thank you
Why is John so predictable? :)
Good example and well explained. Thanks!
Great video
Any suggestions on how to analytically make career desicion that does not possess statistical probability
My thought was a payoff table but I don't know how to do it without percentages
Suggestions?
Thank you. it helps me a lot. I am using WEKA, but the tree size that I get during analysis is 1. how I can solve it? please help me.
I agree with others that this is by far one of the easiest, yet pretty informational video lecture on Decision Tree. I know there are numerous resources to refer, but what would be your recommendation if you were to read a little more in-depth about the topic and do some hands on practice?
[towards data science] ``article
Thanks, It was useful for me,
I could understand the basic general concept .
All of your videos are great, i could have never understood these concepts from books..
Do you have any complete playlist covering all topics of Machine Learning?
So simply explained !
Good thing he is, else this would have made for a very confusing example :)
Thank you!
I don't think he would play in Rain w/ High Humidity. The sequence in which you chose to evaluate the attribute of "Wind" was random...had you chosen Rain-->Humidity--> I don't think he would have played :-)
What book are you using for the class?
Victor Lavrenko gugtytrfttffgj
Professor Lavrenko,
I would be deeply appreciated if you could post a link to download your slide?
Best,
Top
web.arch.usyd.edu.au/~wpeng/DecisionTree2.pdf
The textbook being referenced to here in these videos is a publicly available one?
Really helpful. Understood it pretty easily. Thank you a lot, Victor. :)
You're welcome. Very happy it helped you.
How did you calculate the Entropy to select the root node?
why for Rain wind is taken and humidity is not taken ?
What if I pair 'sunny' with 'wind'? What would happen if there's no pure subset in the tree?
That's a really nice explanation. Thank youuuu lots!!!
Thank you sir for your great lecture.
To which book do you refer?
Very clear and helpful
Is there a way I can contact you directly as I have a question on Decision trees not covered in this video?
Top notch explanation
great seasion . clear and great example!
please make sure you have good audio quality
why do you choose the "outlook" as the first attribute, why not others??
what if i have dataset containing more than 500 data? is there a tool that could help me or do it manually just like u did?
I guess there are many tools, among them rapidminer and openoffice softwares, you can look at this document, it is very helpful and practical docs.rapidminer.com/downloads/DataMiningForTheMasses.pdf
Thanks Victor Lavrenko
Is there any book for reference for Data mining?
Vikram Jadhav We use the following textbook at Edinburgh:
www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0123748569
But there are better textbooks by Chris Bishop, David Barber, and Hastie et al.
Thanks for great explaining
Wow, you made it so easy to understand the decision tree. But I don't understand how someone would play tennis in strong wind:)
very clear to me , thank you
Thank you, great simple example!
Indian guy asking a doubt which is going to be explained in the coming slides is the most Indian thing ever
Thanks, explaination is very clear.
Why not run a logistic regression here? What is the use of decision tree?
Very clear explanation ! Thank you sir ^_^
I think there should be an extra layer after rain about the humidity.....
Thank you, you just saved my neck. I was getting nowhere with my class material until I saw this. Where's the "next slide"?
very good lecture
thank u sir
Very very clear, thank you!
Nice lecture, but the voice is very very low .
Is this section so-called "ID3"?
thanks alot Victor ,, this example helps me so much , continue :)
thank you very much, nice explanation.
good video, it's really helpful
Why dont we use Temperature?
these are very helpful, thanks...
Thanks Victor!
PERFECET!! thanks victor.
What university is this from?
That was really helpful, thank you!
Good explanation
Great video.
Beautiful Explanation :)
Great tutorial!
Thank u for explaining it in so simple way . but here you didn't considered the humidity factor while taking the decision.. :/ . It would have been better if u have included of selection of the attribute factor in this slide itself.anyway thank u
Humidity is also used in splitting
What book are you using?
web.arch.usyd.edu.au/~wpeng/DecisionTree2.pdf