Відео

Very good!

This is so well explained. thank you so much!!! Now I know how to understand KL divergence, cross entropy, logits, normalization, and softmax.

I wish more professors can hit all the insights that you mentioned in the video.

Well done, Adian. I just found out-though I'm not surprised at all, in the Shannon sense 🤓 -that you're doing a PhD at Cambridge. Congratulations! Best wishes for everything 🙂

How does the use of soft label distributions, instead of one-hot encoding hard labels, impact the choice of loss function in training models? Specifically, can cross-entropy loss still be effectively utilized, or should Kullback-Leibler (KL) divergence be preferred?

Useful video.

great video, thank you!

Nice video

What a great and simple explanation of the topic! Great work 👏

Excellent. Short and sweet.

distirbution

thank you codexchan

Thank you for the best explanation

this is a great explanation thank you !

It does not explain the most important part - how the formula for non-uniform distribution came about

Great explanation. Thank you so much!

Adian. I know you're probably super busy doing PhD things, but come back & make some more videos! You're a gifted orator.

Best explanation on the interwebs!

what is true class distribution?

So a Kale Divergence of zero means identical distributions? What do the || lines mean?

<3 this. 👌

wowowowo

Perfectly explained in 5 minutes. Wow.

So clear explanation! Thanks!

Three bits to tell the guy on the other side of the wall what happened, and it suddenly made sense. Thanks.

This is an incomplete hypothesis because on p(x) and ln(p(x)) come without proof is grossly insulting and it ignored the basic principle of conservation of energy meaning if pressure goes up then volume also goes up to release presured to go back to normal and that is the frustrating truth of life. The beast never dies it only change its phases or faces

Excellent video. Can someone help me understand why is it called Divergence in the first place? Why are we taking 1/N power to normalise it to sample space, I did not understand the logic behind this.

not really a great explaination, so many terms were thrown in. that's not a good way to explain something.

Thankkkk youuuuu.

Thank you so much for this video and clear explanation!

make more videos please , you are awesome

Excellent expositions on KL divergence and Cross Entropy loss within 15 mins! Really intuitive. Thanks for sharing.

Great video! Can you make a video about soft actor critic?

Thank you!

Bro really explained it in less than 10 mins when my professors don't bother even if it could be done in 5 secs, true master piece thus video keep it up man 🔥🔥🔥

I'm just rewatching this video to freshen up my deep learning fundamentals. Super clear video, thank you so much!

I appreciate your effort, but the video is quite confusing. For example, in the example about 8 football teams, you explain why 3 bits are required by flat out stating as a starting premise that 3 bits are required! It's a circular argument.

Dude, took information theory from a rigorously academic and formal professor. I'm a little slow and under the pressure of getting assignments done, couldn't always see the forest for the trees. Just the sentence "how much information, on average, would we need to encode an outcome from a distribution" just summed up the whole motivation and intuition. Thanks!

god this is an incredible video thank you so much

can someone explain the triplets part

Thanks for the explanation!! One thing is, formulas were confusing with how you denoted *q1* & *q2* for probabilities for coin 2, instead of *p2* & *q2=1-p2*

Thank you a lot

Only that is not a distance ('cause is not symmetric), but a pseudo distance. Great video!

I expected something else, but it's also ok.

Great video. Thanks for sharing. Really intuitive.

for everyone trying to understand this concept even more thoroughly, towardsdatascience's article "The intuition behind Shannon’s Entropy" is amazing. it gives added insight on why information is the reciprocal of probability

Please make more videos this is literally the only time I've ever seen entropy be explained in a way that makes sense

Great video!

Great content! Thank you.

This one I would say is a very nice explanation of Cross Entropy Loss.