Like I said at the end of the video, I have some ideas for this channel. Here are some types of videos: 1. Concepts with Comedy - Short, but pithy explanations of certain big idea concepts in the Data Science Space (Eg. the AB Testing Video I made) with some bad comedy to keep it engaging. 2. Algorithm Concepts - Explaining the details of Machine Learning Algorithms (like this one). 3. The Math You Should know - Subset of (2) with a ton of Math (Linear, Logistic Regression and Kernel Vids I made) 4. Papers and Discussions - Reading and explaining the concept math behind recent research papers in deep Learning, machine learning (Eg. Attention, Masked RCNN, CycleGANs vids) I made explicit playlists of these videos, so check 'em out in the "playlist" section. add the entire playlist for some continuous quality content ;) I'll have a video on High dimensional data and PCA coming up - either the next or the one after that. So subscribe to keep an eye out for it! I have some goals for this channel. Lets make it big! Thanks for all your support!
Thank you for saving my summer project essay! It's really helpful to have someone show me the details rather than one sentence saying "doing this with Lagrange multipliers" on Boswell's paper 🥳
It does contain numbers, but it would take super long to go through each calculation, and if you’re watching this video, you should already know how to plug in those “numbers”
I would be thankful if you tell me what maths are required for SVM and any reference to learn that. I have searched on internet for a looooooong time and every single one says just the basic math. I'm too frustrated over the internet rn.
Maybe a tad to rushed, but balancing time to make video and release is propably a tricky buisness :) That being said, I like your videos 1000x more than Siraj, wish I could move some of his views to you ;) Keep it up!
Thanks for the constructive criticism, and the compliments ;). I admit the math was rushed. Things that took me 30 minutes to type were shown on screen for 5 seconds. But I am worried that long and drawn out videos with a ton of math may be overwhelming. And I didn't just want this to be some 1 hour lecture on the topic. That said, I'm trying out different things -- hopefully I can find a sweet spot.
Yoooooo!! With super crisp explanations of the maths behind it, visualization super-awesome and picture-perfect presentation this video is a nice contribution to the community of ML. Super awesome work!! Keep it up.
thank you for the deep dive into rbf kernels, my hope is to fill in my gaps w my maths so I can watch these videos and get better intuition of these topics.. for now not quite there but we still getting there !
Why is phi used to denote the transformation function at 1:22? I asked Chat GPT and it doesn't know why, I looked online and did not see phi used in this context at all. At the risk of sounding ignorant, I don't believe this symbol should have been used. It seems any other alternative would have been suitable without the additional confusion of the symbols meaning in context of the equation. Like why use the symbol phi when you'd assume it adds some additional context but instead it adds nothing to the understanding of the function?
Good video man. However, for those of us who are not excellent in math, I feel like it would be a lot more helpful if you could derive these equations by hand.
hi bro,thank you for the short and clear material.can you help me understand in detail with mathematics how you use the summand c in c(with a(x)+b)=0 to compare the equation by 1 (at 04:30)?I'm just watching your translation video.
Great video! I was wondering how exactly the final optimization is implemented to find the alphas and the lambdas, which is the whole point. At first glance, it seemed to me that the Lagrangian has far too many variables(about the size of the dataset), and too few constraints to work with. So I poked around and found this: en.wikipedia.org/wiki/Sequential_minimal_optimization
In 4:07 on the top right corner, if you multiply the hyperplane equation by y_i, wouldn't that expression be always greater or equal to 0? If your hyper plane equation is
@@keren718 It's because y_i is either worth 1 or -1 it's the "label" of each points it's -1 when the points are above the line and 1 when they are underneath it
Usually it's a mapping from the variables Xs to one function of them. Just like when you see a 3d Graph of a function of two variables. The graph makes some points to have a "height". Same idea for more dimensions. So if you add a height to each point. based on a function. Maybe there's a function. Where the heights given. Make the linearly unseparable points, actually separable. For example (x^2+y^2). Makes points around 0,0 be at a "low height". Further points get up. So you can separate points in a circle. around 0,0 By giving height to the far points. That mapping (x1,x2,...,xn, f(x) ) is phi. At the end we don't need to reformulate the problem. Since the solution of the optimization problem. Makes you only need phi(x1).phi(x2). We define this as Kernel(x1,x2).
Hi really good video thanks for posting it, I had a question, did you do a video about the non perfect separation case, optimization problem solution, I see that here the dual problem is posed, but one which is solved? subgradient/gradient decend, any book you recommend other than the standard 'statistical learning' ? Thanks!
you make the convex optimisation problem- with the variables w,b and k(whatever that symbol was). Now you need to solve the convex optimisation problem to find these optimum variables- just put your set of equations with constraints in a quadratic solver.
You might have found the answer to this question by now, but I think it's worth explaining since I could not find similar information elsewhere. After you get the alphas from the quadratic solver, you can substitute them in the w*phi equation. After you get this vector, you can then substitute it on the constraint y(w*phi+b)=1 and solve for b. To predict new values, a new kernel matrix with phi_k and phi_test has to be computed.
Either the derivations you show to simplify in the larangian step are wrong or you have messed up your optimization assumption. I could point out the mistakes if you wish to correct your video. :P
I think that you are right, because the sum (a_n*y_n*(wt.phi(x_n)) isn't equal to 0 he made a mistake by considering wt.phi(x_n) as a constant, which gaved him wt.phi(x_n)*sum(a_n*y_n) = 0 because sum(a_n*y_n) = 0, indeed wt.phi(x_j) != wt.phi(x_k) if you choose j and k properly
at @4:57 how did you converted the maximization problem into minimization problem as far as i know you should multiply it with -1 if you want to convert a maximization problem to minimization.
How should one go about otpimizing a SVM (decisionProcedure; SVM type,Kernel type, shrinking, etc...) and hey! you don't need to be "funny" at all cost, your content is great and we, your audience, are mostly nerds..... so please keep up the good work because we need to understand the math behing the algos
I think that you are right, because the sum (a_n*y_n*(wt.phi(x_n)) isn't equal to 0 he made a mistake by considering wt.phi(x_n) as a constant, which gaved him wt.phi(x_n)*sum(a_n*y_n) = 0 because sum(a_n*y_n) = 0, indeed wt.phi(x_j) != wt.phi(x_k) if you choose j and k properly
I think each column of w represents the weights of X1 to Xn in y=w1x1+... So when we transpose we get w1 to wn in rows and multiplying it with vector matrix of x will yield us the y=w1x1+... Equation
When I use "x", it is a variable. x_0 on the other hand is a _specific_ point. I use d_H with x_0 to show this as a distance of a _specific_ point from the hyperplane. If you replace both x_0s with x. That is also correct, and it gives the generalized formula. I just applied it to a specific instance where x = x_0
@@CodeEmporium Yes, that makes sense, but x_0 (as a specific point) should also contain y component inside, right? I mean, the non-vector formula on the left side uses x_0 and y_0 intuitively for 2D, but d_H contains x_0 and no y_0 which leads into my confusion that y coordinate seems to be omitted from the calculation as only x_0 is being used and it has been previously used for 1 component of 2D explicitly...? So x_0 in d_H is a vector of x, y coordinates of first specific point in the dataset, ok?
great video of clear explain ML topics! liked & subed! is this video from a series? if so can you post the links for previous and next video, or playlist. Thanks a lot!
If you can't do better than putting a bunch of symbols on a PPT and explain them in a rushed manner, you might as well not make an explainer video. This is the most jargon-y but superficial video I have ever come across in a while, please take a intro to linear algebra class.
Yea. Made this about 4 years ago. At the time, the goal of making this video was more to teach myself to learn for an exam. This is less of an “explainer” and more “my note taking and prep work for the exam”. So the explanation isn’t great per se. ill say tho that my more recent videos are better at this since now I am trying to explain concepts instead. Thanks for watching!
Maybe. Made this while learning about the topic at the time. This wasn't made to teach, more work out some math for a test. I have a very different teaching style these days. Maybe I'll remake it
Like I said at the end of the video, I have some ideas for this channel. Here are some types of videos:
1. Concepts with Comedy - Short, but pithy explanations of certain big idea concepts in the Data Science Space (Eg. the AB Testing Video I made) with some bad comedy to keep it engaging.
2. Algorithm Concepts - Explaining the details of Machine Learning Algorithms (like this one).
3. The Math You Should know - Subset of (2) with a ton of Math (Linear, Logistic Regression and Kernel Vids I made)
4. Papers and Discussions - Reading and explaining the concept math behind recent research papers in deep Learning, machine learning (Eg. Attention, Masked RCNN, CycleGANs vids)
I made explicit playlists of these videos, so check 'em out in the "playlist" section. add the entire playlist for some continuous quality content ;)
I'll have a video on High dimensional data and PCA coming up - either the next or the one after that. So subscribe to keep an eye out for it!
I have some goals for this channel. Lets make it big! Thanks for all your support!
you explain well, Please try to do a kaggle competition or a computer vision project with code
You Will very soon have 1M subs...
Keep Growing 👑👑👑
I would say go for 3 and 4, excellent content brother.
Hi please let me know if I can contact you in some way
InstaBlaster.
Thank you for saving my summer project essay! It's really helpful to have someone show me the details rather than one sentence saying "doing this with Lagrange multipliers" on Boswell's paper 🥳
Great stuff, hats off !
Dude kindly keep making videos of "Maths you should know"
Can you please go for Hidden Markov Models and Kalman filters ?
Why doesn't anyone go through an example with real numbers so we can actually see these formulas in action?
because most of these variables are vectors or matrices
@@henrychen1544 ... Which contains numbers, so it is a reasonable question
It does contain numbers, but it would take super long to go through each calculation, and if you’re watching this video, you should already know how to plug in those “numbers”
There are lot of videos with examples and numbers. This video is when you want to know hows not whats.
@@henrychen1544 an illustration with actual number is definitely possible. Real life data science problems involve actual numbers.
The best video of SVM! Thank you for your excellent work!
You are very welcome :)
This is exactly what I was looking for. End to end explanation clearly showing the steps involved. Thanks a ton man!❤
you were just wonderful!. you explained the concept amazingly, exactly what I needed to hear and at correct speed.
@2.20 the equation of distance for point vector from hyperplane, the denominator should be ||w|| than ||w|| squared.
I would be thankful if you tell me what maths are required for SVM and any reference to learn that. I have searched on internet for a looooooong time and every single one says just the basic math. I'm too frustrated over the internet rn.
Maybe a tad to rushed, but balancing time to make video and release is propably a tricky buisness :)
That being said, I like your videos 1000x more than Siraj, wish I could move some of his views to you ;)
Keep it up!
Thanks for the constructive criticism, and the compliments ;). I admit the math was rushed. Things that took me 30 minutes to type were shown on screen for 5 seconds. But I am worried that long and drawn out videos with a ton of math may be overwhelming. And I didn't just want this to be some 1 hour lecture on the topic. That said, I'm trying out different things -- hopefully I can find a sweet spot.
Plz for God sake keep doing this
God has spoken. You're prayers have been answered. Your messiah is here.
But yeah. I don't plan on stopping anytime soon. Thanks for the support!
Yoooooo!! With super crisp explanations of the maths behind it, visualization super-awesome and picture-perfect presentation this video is a nice contribution to the community of ML. Super awesome work!! Keep it up.
Thanks for the compliments :D I shall be pumping more content (video in 2 days!)
Very well done on the explanation part and i am obsessed with your Math explanation especially the term and usage of it.
thank you for the deep dive into rbf kernels, my hope is to fill in my gaps w my maths so I can watch these videos and get better intuition of these topics.. for now not quite there but we still getting there !
The most informative and easily understandable video....Thanks a lot ❤❤❤
Why did you change the minimization in the primal form to maximization in the dual form ?
Most crisp and to the point explanation.
Dude.. You are a saviour! Great work. Keep it coming.. 👍🏻
Thanks homie!
Math can be done using some example.. Writing equations may not help
Why is phi used to denote the transformation function at 1:22? I asked Chat GPT and it doesn't know why, I looked online and did not see phi used in this context at all. At the risk of sounding ignorant, I don't believe this symbol should have been used. It seems any other alternative would have been suitable without the additional confusion of the symbols meaning in context of the equation. Like why use the symbol phi when you'd assume it adds some additional context but instead it adds nothing to the understanding of the function?
Why we have multiplied yn with (Wt * phi(n) + b)?
Could you please offer some theoremes and proofs?
This is wonderful. Can you please make a video on Pegasos: Primal Estimated sub-GrAdient SOlver for SVM
0:42 that “kernalization” had me laughing 😂😂😂😂
OMG This is so helpful!!!. So simple but so general. Thank you very much sir!
So glad this is helpful!! Thanks for commenting and watching !!
I'll suggest you keep your good work going .. you have a great potential.
Thanks for the support Gaurav! You can look forward to more videos.
Excellent explanation, keep up the good work.
Thanks a ton! Look forward to some more videos homie!
I think you have a typo in the equation appearing in 9:06, last term. It should be "sum_n(xi_n(C-alpha_n-lambda_n))"
No. its right.
brooooooooooooo.....you saved my life...!!!
thanks a ton.....
Saving lives is what I do..you are welcome
One edit. In the dual formulation you dropped the double summation which might create some confusion
wow, this is what i was looking for!
Nice! Glad it helps!
Good video man. However, for those of us who are not excellent in math, I feel like it would be a lot more helpful if you could derive these equations by hand.
Thanks for the feedback homie. I'll try to make a video that's more accessible soon :)
hi bro,thank you for the short and clear material.can you help me understand in detail with mathematics how you use the summand c in c(with a(x)+b)=0 to compare the equation by 1 (at 04:30)?I'm just watching your translation video.
Great video! I was wondering how exactly the final optimization is implemented to find the alphas and the lambdas, which is the whole point. At first glance, it seemed to me that the Lagrangian has far too many variables(about the size of the dataset), and too few constraints to work with. So I poked around and found this: en.wikipedia.org/wiki/Sequential_minimal_optimization
In 4:07 on the top right corner, if you multiply the hyperplane equation by y_i, wouldn't that expression be always greater or equal to 0?
If your hyper plane equation is
I guess its because y_i can have a value of either less than zero or more than zero but I am not sure and I am also wondering about this..
It's because y_i is either worth 1 or -1 it's the "label" of each points it's -1 when the points are above the line and 1 when they are underneath it
@@keren718 It's because y_i is either worth 1 or -1 it's the "label" of each points it's -1 when the points are above the line and 1 when they are underneath it
Can you make a video on support vector regression ? can you also make a video on the steps for making SVR Matlab code ????
How did the constraint change for linearly inseperable case at 5:49?Why should Xi satisfy yn(wtx+b)>=1-Xi?
We get a closed form for w in terms of the Lagrange multipliers. But for prediction, we also need the intercept 'b'. How do we obtain it?
1:15 What is a feature space? What is Phi of x?
Usually it's a mapping from the variables Xs to one function of them.
Just like when you see a 3d Graph of a function of two variables. The graph makes some points to have a "height". Same idea for more dimensions.
So if you add a height to each point. based on a function. Maybe there's a function. Where the heights given. Make the linearly unseparable points, actually separable.
For example (x^2+y^2). Makes points around 0,0 be at a "low height". Further points get up. So you can separate points in a circle. around 0,0 By giving height to the far points.
That mapping (x1,x2,...,xn, f(x) ) is phi.
At the end we don't need to reformulate the problem. Since the solution of the optimization problem. Makes you only need phi(x1).phi(x2). We define this as Kernel(x1,x2).
Hi really good video thanks for posting it, I had a question, did you do a video about the non perfect separation case, optimization problem solution, I see that here the dual problem is posed, but one which is solved? subgradient/gradient decend, any book you recommend other than the standard 'statistical learning' ? Thanks!
How can you substitute optimal weight vector to be the argmax(min....) ?? I am confused buddy.
Also Why are we taking argmax of the minimum distance ??
And I also noticed that the w_optimum vector given by argmax(1/l2_norm(w_vector)) got changed suddenly to min1/2l2_norm(w_vector) in the Primal Form
VERY nice video contain easy and well explanation ,,,,god keep giving you lot of intelligent
Please make some.videos about ML CNN related maths calculation videos in details
May someone please write the steps on how to find w and b parameter of an hyperplane given labeled training data set ?
you make the convex optimisation problem- with the variables w,b and k(whatever that symbol was). Now you need to solve the convex optimisation problem to find these optimum variables- just put your set of equations with constraints in a quadratic solver.
You might have found the answer to this question by now, but I think it's worth explaining since I could not find similar information elsewhere.
After you get the alphas from the quadratic solver, you can substitute them in the w*phi equation. After you get this vector, you can then substitute it on the constraint y(w*phi+b)=1 and solve for b.
To predict new values, a new kernel matrix with phi_k and phi_test has to be computed.
Either the derivations you show to simplify in the larangian step are wrong or you have messed up your optimization assumption. I could point out the mistakes if you wish to correct your video. :P
I just thought I should let you know you're awesome. Thanks for sharing. You have a subscriber!
Thanks you. Really appreciate it :)
wow this is exactly what I needed. Thanks!
did i miss it ? where do we introduce the support vectors ?
At 6:09, shouldn't that be \xi>2, so that one minus that be less than -1?
Dude, I like your representation and husky voice coming on middle
Sir Please help me if possible What is the difference between PSO and SVM ? How to prove SVM is better than PSO?
I love you, bro. Thanks!!
Thank you
It’s so help, thanks for sharing! But there’s a mistake in 9:13, g() is wrong.
I think that you are right, because the sum (a_n*y_n*(wt.phi(x_n)) isn't equal to 0 he made a mistake by considering wt.phi(x_n) as a constant, which gaved him wt.phi(x_n)*sum(a_n*y_n) = 0 because sum(a_n*y_n) = 0, indeed wt.phi(x_j) != wt.phi(x_k) if you choose j and k properly
Awesome! Keep it up. Waiting to see more amazing videos like this from you.
Great video. Thanks for making it.
Thanks for watching bad commenting!!
Such a good explanation, thanks!
What is phi of X?
You are simply awesome, bro!!!!!!!
Obsessed with your videos ! Many Thanks :)
at @4:57 how did you converted the maximization problem into minimization problem as far as i know you should multiply it with -1 if you want to convert a maximization problem to minimization.
1/W2 to W2
what does w* represent? thank you
Great video but Its hard to follow. Ive done andrews course so i was able to make some sense of this.
How should one go about otpimizing a SVM (decisionProcedure; SVM type,Kernel type, shrinking, etc...) and hey! you don't need to be "funny" at all cost, your content is great and we, your audience, are mostly nerds..... so please keep up the good work because we need to understand the math behing the algos
How can we prove SVM is better than PSO
@@kamakshim8001 I dunno, master!!! I subscribed to your channel, not the otehr way around!
post a detailed video on this topic
9:15 left 3rd equation looks incorrect
I think that you are right, because the sum (a_n*y_n*(wt.phi(x_n)) isn't equal to 0 he made a mistake by considering wt.phi(x_n) as a constant, which gaved him wt.phi(x_n)*sum(a_n*y_n) = 0 because sum(a_n*y_n) = 0, indeed wt.phi(x_j) != wt.phi(x_k) if you choose j and k properly
Take my sub, this is a great video.
Thanks Piqcyl! Glad you got something out of it ;)
Explained very well!!!!
hey sorry if I missed it, but what is w (and why do we transpose it)?
I think each column of w represents the weights of X1 to Xn in y=w1x1+...
So when we transpose we get w1 to wn in rows and multiplying it with vector matrix of x will yield us the y=w1x1+... Equation
@@jagannathan1014 thanks Jagan!
At 2:19, the hyper-plane point-vector distance d_H, should the x_0, in both cases, be just "x"?
When I use "x", it is a variable. x_0 on the other hand is a _specific_ point. I use d_H with x_0 to show this as a distance of a _specific_ point from the hyperplane. If you replace both x_0s with x. That is also correct, and it gives the generalized formula. I just applied it to a specific instance where x = x_0
@@CodeEmporium Yes, that makes sense, but x_0 (as a specific point) should also contain y component inside, right? I mean, the non-vector formula on the left side uses x_0 and y_0 intuitively for 2D, but d_H contains x_0 and no y_0 which leads into my confusion that y coordinate seems to be omitted from the calculation as only x_0 is being used and it has been previously used for 1 component of 2D explicitly...? So x_0 in d_H is a vector of x, y coordinates of first specific point in the dataset, ok?
Shades of Milhouse: "My mom says Im cool!" No, great explanation. Thanks.
Nice video!!
Great tutorial, please keep it up
Definitely. Thanks for the support!
I love the math you should know
ahhhhh I need to review linear algebra and calc
You're brilliant!
I keep hearing you say "five x" instead of phi axe.
you are the best man!!
Thanks! I'm thinking of starting my own religion soon.
why do you put a reverb on your voice? :/
My god hard to understand
After the Lagrangian appliance I can't believe that you can be funny ;), no possibility :-]
I is funny I swear ;) it’s just all this math talk makes me sound like a robot hahah
@@CodeEmporium Sure I believe, I've just wanted to pull your leg a little :):). Very good math videos, please continue at this level !
ahhhh why couldnt my prof link me to this video
Poor prof didn't know :)
good video
Amazing
Is the indicator function described in this video at all? I watched the entire thing but maybe I missed it or am not connecting it to it in the video
great video of clear explain ML topics! liked & subed! is this video from a series? if so can you post the links for previous and next video, or playlist. Thanks a lot!
hard to grasp. Excess of math notation without explanation
I failed to explain how does SVM works in an interview. Only if I had watched this video before.
Thanks for the compliment! feel like this video went way too Technical. I honestly wish I made this explanation more easy to grasp.
@@CodeEmporium Yes it did, but you don't need to remember every step, just intuition is enough. And I got my intuition here.
So harrrddd
I’m lost haha
felt a bit rushed in the math part.
math video doesn't make sense if the math is rushed
an ACTUAL math example would've helped
its too math heavy throwing so many equations without providing the intutive context makes it very gibberish.
Me he mareado
I still don't undserstand lol
If you can't do better than putting a bunch of symbols on a PPT and explain them in a rushed manner, you might as well not make an explainer video. This is the most jargon-y but superficial video I have ever come across in a while, please take a intro to linear algebra class.
Yea. Made this about 4 years ago. At the time, the goal of making this video was more to teach myself to learn for an exam. This is less of an “explainer” and more “my note taking and prep work for the exam”. So the explanation isn’t great per se. ill say tho that my more recent videos are better at this since now I am trying to explain concepts instead. Thanks for watching!
one of the worst explanations of SVMs I've seen.
Maybe. Made this while learning about the topic at the time. This wasn't made to teach, more work out some math for a test. I have a very different teaching style these days. Maybe I'll remake it
Additional SVM blog : medium.com/sfu-cspmp/a-practical-guide-to-support-vector-machines-svm-ccd6a4d4dd04