![Adian Liusie](/img/default-banner.jpg)
- 6
- 255 089
Adian Liusie
Приєднався 10 лип 2020
Intuitively Understanding the Shannon Entropy
Intuitively Understanding the Shannon Entropy
Переглядів: 92 783
Відео
Intuitively Understanding the Cross Entropy Loss
Переглядів 78 тис.3 роки тому
This video discusses the Cross Entropy Loss and provides an intuitive interpretation of the loss function through a simple classification set up. The video will draw the connections between the KL divergence and the cross entropy loss, and touch on some practical considerations. Twitter: AdianLiusie
Beginners Overview of Machine Learning and Artificial Intelligence
Переглядів 8303 роки тому
This is a recorded talk which I created for my old school, Dubai College. This video is an introduction for complete beginners into artificial intelligence and machine learning, gives the general idea of the field and an overview of the machine learning approach.
Understanding Neural Networks Training
Переглядів 7723 роки тому
This video gives an overview of general optimisation of neural networks and explains how training is done through loss minimisation. This video will provide a foundation to then take a deeper look at more complex practical algorithms like stochastic gradient descent and backwards propagation which will be covered in future videos.
Intuitively Understanding the KL Divergence
Переглядів 82 тис.3 роки тому
This video discusses the Kullback Leibler divergence and explains how it's a natural measure of distance between distributions. The video goes through a simple proof, which shows how with some basic maths, we can get under the KL divergence and intuitively understand what it's about.
Understanding deep neural networks
Переглядів 7843 роки тому
This video gives a basic introduction to neural networks and discusses what they are, how they work and ways to see the system in a matrix framework. This is the first video in a series which will ultimately build up to programming neural networks and running the back propagation algorithm from scratch in python.
Very good!
This is so well explained. thank you so much!!! Now I know how to understand KL divergence, cross entropy, logits, normalization, and softmax.
I wish more professors can hit all the insights that you mentioned in the video.
Well done, Adian. I just found out-though I'm not surprised at all, in the Shannon sense 🤓 -that you're doing a PhD at Cambridge. Congratulations! Best wishes for everything 🙂
How does the use of soft label distributions, instead of one-hot encoding hard labels, impact the choice of loss function in training models? Specifically, can cross-entropy loss still be effectively utilized, or should Kullback-Leibler (KL) divergence be preferred?
Useful video.
great video, thank you!
Nice video
What a great and simple explanation of the topic! Great work 👏
Excellent. Short and sweet.
distirbution
thank you codexchan
Thank you for the best explanation
this is a great explanation thank you !
It does not explain the most important part - how the formula for non-uniform distribution came about
Great explanation. Thank you so much!
Adian. I know you're probably super busy doing PhD things, but come back & make some more videos! You're a gifted orator.
Best explanation on the interwebs!
what is true class distribution?
the frequency of occurrence of a particular class depends on the characteristics of the objects
So a Kale Divergence of zero means identical distributions? What do the || lines mean?
<3 this. 👌
wowowowo
Perfectly explained in 5 minutes. Wow.
So clear explanation! Thanks!
Three bits to tell the guy on the other side of the wall what happened, and it suddenly made sense. Thanks.
This is an incomplete hypothesis because on p(x) and ln(p(x)) come without proof is grossly insulting and it ignored the basic principle of conservation of energy meaning if pressure goes up then volume also goes up to release presured to go back to normal and that is the frustrating truth of life. The beast never dies it only change its phases or faces
Excellent video. Can someone help me understand why is it called Divergence in the first place? Why are we taking 1/N power to normalise it to sample space, I did not understand the logic behind this.
not really a great explaination, so many terms were thrown in. that's not a good way to explain something.
Thankkkk youuuuu.
Thank you so much for this video and clear explanation!
make more videos please , you are awesome
Excellent expositions on KL divergence and Cross Entropy loss within 15 mins! Really intuitive. Thanks for sharing.
Great video! Can you make a video about soft actor critic?
Thank you!
Bro really explained it in less than 10 mins when my professors don't bother even if it could be done in 5 secs, true master piece thus video keep it up man 🔥🔥🔥
I'm just rewatching this video to freshen up my deep learning fundamentals. Super clear video, thank you so much!
I appreciate your effort, but the video is quite confusing. For example, in the example about 8 football teams, you explain why 3 bits are required by flat out stating as a starting premise that 3 bits are required! It's a circular argument.
Dude, took information theory from a rigorously academic and formal professor. I'm a little slow and under the pressure of getting assignments done, couldn't always see the forest for the trees. Just the sentence "how much information, on average, would we need to encode an outcome from a distribution" just summed up the whole motivation and intuition. Thanks!
god this is an incredible video thank you so much
can someone explain the triplets part
Thanks for the explanation!! One thing is, formulas were confusing with how you denoted *q1* & *q2* for probabilities for coin 2, instead of *p2* & *q2=1-p2*
Thank you a lot
Only that is not a distance ('cause is not symmetric), but a pseudo distance. Great video!
I expected something else, but it's also ok.
Great video. Thanks for sharing. Really intuitive.
for everyone trying to understand this concept even more thoroughly, towardsdatascience's article "The intuition behind Shannon’s Entropy" is amazing. it gives added insight on why information is the reciprocal of probability
Thank you!!
And it's paywalled...
Please make more videos this is literally the only time I've ever seen entropy be explained in a way that makes sense
Great video!
Great content! Thank you.
This one I would say is a very nice explanation of Cross Entropy Loss.