Transformers - Part 1 - Self-attention: an introduction

[@wangkuanlee3548]

Superb explanation. This is the clearest explanation of the concept of weight in self-attention I have ever heard. Thank you so much.

[@mar-a-lagofbibug8833]

Thank you for sharing.

[@kencheligeer3448]

It is a very brilliant explanation about self-attention!!! Thank you.

[@kacemichakdi3048]

Thank you for your explanation. I just didnt understand how we chose W_k and W_q???

[@lennartsvensson7636]

These matrices contain learnable parameters that can be trained using standard techniques from deep learning.

[@kacemichakdi3048]

Thank you

[@mustafakocakulak5895]

Best explanation ever :) Thank you

[@murkyPurple123]

Thank you

[@piyushkumar-wg8cv]

Intuition buildup was amazing, you clearly explained why we need learnable parameters in the first place and how that can help relate similar words. Thanks for the explanation.

[@euisasriani_01]

Thank you for great explanation. I still don't understand about how to gain Wq ad Wk.

[@po-yupaulchen166]

Great and clear explanation. One question about W_Q and W_K. Since z1 = k1^T *q3 = x1^T * (W_k^T * W_Q) * x2, and W_k and W_Q are trainable matrices, could we just combine it as a matrix like
W_KQ = W_k^T * W_Q to reduce the number of paramters?

[@lennartsvensson7636]

What you are suggesting should be possible as long as the matrices are quadratic.

[@andrem82]

Best explanation of self-attention I've seen so far. This is gold.

[@exxzxxe]

A first class explanation of self attention- the best on UA-cam.

[@jhnflory]

Thanks for putting these videos together!

[@prasadkendre149]

greatful forever

[@ahmedb2559]

Thank you !

[@prateekpatel6082]

pretty bad example . Even if we have trainiable Wq and Wk , what if there was a new sentence where we had Tom and and he , the WQ will still make word 9 point to wmma and she