Visual Guide to Transformer Neural Networks - (Episode 3) Decoder’s Masked Attention
Вставка
- Опубліковано 8 чер 2024
- Visual Guide to Transformer Neural Networks (Series) - Step by Step Intuitive Explanation
Episode 0 - [OPTIONAL] The Neuroscience of "Attention"
• The Neuroscience of “A...
Episode 1 - Position Embeddings
• Visual Guide to Transf...
Episode 2 - Multi-Head & Self-Attention
• Visual Guide to Transf...
Episode 3 - Decoder’s Masked Attention
• Visual Guide to Transf...
This video series explains the math, as well as the intuition behind the Transformer Neural Networks that were first introduced by the “Attention is All You Need” paper.
--------------------------------------------------------------
References and Other Great Resources
--------------------------------------------------------------
Attention is All You Need
arxiv.org/abs/1706.03762
Jay Alammar - The Illustrated Transformer
jalammar.github.io/illustrated...
The A.I Hacker - Illustrated Guide to Transformers Neural Networks: A step by step explanation
jalammar.github.io/illustrated...
Amirhoussein Kazemnejad Blog Post - Transformer Architecture: The Positional Encoding
kazemnejad.com/blog/transform...
Yannic Kilcher UA-cam Video - Attention is All You Need
www.youtube.com/watch?v=iDulh...
Correction in the Video: A huge shout-out to @jackzead "What comes out of the encoder should be the key and value matrix and what comes out of the bottom part of the decoder is the query matrix."
*TIMESTAMPS*
Here are the timestamps associated with the concepts covered in this video:
0:00 - Recaps of Parts 1, 2
0:52 - Residual Connections
4:25 - Transformer Decoder
9:52 - Decoder's Masked Attention
I was wondering if you would correct this. Thanks.
@@muhammadsaadmansoor7777 I tried but was not able to find a way to add notes to a video which has been published. Apparently, UA-cam has a few tools that get enabled to the channel owners with a certain number of subscribers only. Let me know if you know how to add notes to the video.
@@HeduAI You can pin this comment at the top. Excellent video, congratulations.
@@ivanmatas3884 Thanks :) This comment is already pinned to the top.
@@HeduAI I think it's not anymore pinned to the top - don't know how this works though.
Maybe you can also add to the video description that there's this error or the correction.
Thanks a lot for this extremely awesome content! Saved me loads of time and was fun to watch at the same time!
These explanations are so good that they should be included into Tensorflow's and PyTorch's documentations, seriously !
This made my day :) Thanks!
True
This series is a gem. Really clear explanation, much easier to get back to the paper and the math with such overview. Huge kudos!
You are the mother of StatQuest and 3Blue1Brown. Both of these guys are awesome in explaining complex ideas in simple words. But you are the best.
I don't know about StatQuest (haven't seen his ones) and 3Blue1Brown is good because of the visualization he brings with his advanced animations. But honestly, here she explained all these concepts using simple animations and had a good structure throughout the videos, each connecting well to the other. Very commendable if you ask me.
This series deserves so much more "attention"
I have tried to learn transformer from various sources with several attempts in the past a couple of months, but I couldn't get the gist of it. I wish I have spotted your 3 episodes earlier, they were extremely helpful and easy to follow.
I really appreciate the time, effort, love and joke you put into your videos, and sincerely wish you could keep making these types of videos on DL topics. I love the fact that each concept/technique is well explained with a concrete example as well as why it was done this way not the others.
You are truely gifted at explaining complicated concept for people with limited DL background. Please keep up the good work
Please make series of NLP , Deep Learning. Your teaching style is amazing.
Thank you! I am glad you enjoyed it!
You have an amazing gift of explaining complex things in an intuitive way!
Thank you! Glad you found it useful :)
it hurts that ur skills are so under-rated, the ai community needs more of you!
You are truly truly gifted in explaining very technical topics in an intuitive way! I often get embarrassed to go to UA-cam to learn about DL concepts considering that I have studied the topic for a couple years now, but in all honesty, the scientific community desperately needs people like you. Thank you for investing your time in making the world a better place!
Roy, you truly made my day. Thank you so much!
Her way of explaining is so crystal clear. Its very evident she is a genius
When Azerio from the Assassin's Creeds calls you a genius, you've gotta agree :D Thanks!
You are my savior. Went through a bunch of videos yesterday, didn't understand shit.. finally found these awesome short 3 videos today.
this is my 3rd comments for my entire life using UA-cam! The best video for explaining transformer! Hope you will make more videos in future!
Batool you are really a great teacher. Thanks for the content. Please provide more vid on LLMS and AI
Your three episodes are the clearest and best explanation to Transformer that I've ever come across. They help me a lot and I thank you for that. I can see that you got talent to explain things complicated in a very simplistic way. Looking forward to watching more of them.
Super clear explanations. Really appreciate you putting this series together! (and love the pop culture references)
I've watched many videos about transformers, this is by far the best.
Literally THE best 3 video playlist 👌👌
havn't seen such a good series of vids in a realy long time
I passed a long time searchig a good explanation untill i find these series . awesome !! keep going 😍😍😍
These explaination videos are so good. It would be a crime not to subscribe. You earned yourself a sub today!
Your explanation is too good..I am able to understand these transformers because of the way explained
I don't know who you are but I do know that this series was the best explanation of Transformers I have seen or read and I have seen and read a lot. Thank you.
I was trying to understand it for quite a long time, and for the first time I don’t feel like a child lost in the woods 😂 you really have a gift!
This has got to be the sweetest comment :)
You have ny utmost respect, ma'am!
There might be a mistake in the video. What comes out of the encoder should be the key and value matrix and what comes out of the bottom part of the decoder is the query matrix. No? The query is the matrix that is looking for the right key and value pairs. The rest is very good. I always learn something new in your videos.
Hmm... you maybe right. Let me confirm this and get back in a few days. If it is indeed a mistake, I will try my best to inform future viewers about it by either editing the video or pinning the edit comment to the top. Thanks for the critical feedback!
The encoder output is used as the Query and Key input to the 2nd Multi-Headed Self Attention (also known as Encoder-Decoder Attention) of the Decoder Block. And the Value comes from the 1st Self-Attention Block of the Decoder. The explanation in the video is correct.
@@nivu Well, I have seen Jay Alammar's explanation and I have also seen 2 code implementations of Transformer and all 3 references explain and show that the encoder outputs the key and value matrices and the 1st self-attention block of the decoder outputs the query. Again this makes sense since Key and Value matrices always come in pairs (a key is a kind of an index into the value, they are supposed to be together). The query is the question that asks which key is the most relevant and so it makes sense it comes from the decoder since the decoder knows the ground truth.
In case of language translation from english to german, the encoder would study the english langage, the 1st self-attention block of the decoder would study the german language, and the 2nd attention block of the decoder would study the interaction between the 2.
This is what Jay Alammar says about this point: "The “Encoder-Decoder Attention” layer works just like multiheaded self-attention, except it creates its Queries matrix from the layer below it, and takes the Keys and Values matrix from the output of the encoder stack."
@@jackziad Thanks! Have added the corrective comment and pinned it to the top, accrediting you. Thanks again for pointing this out.
Best video series ever! Thanks is not enough.
i have never heard things being explained so clearly and articulated with all key details as much as possible ! thank you so much .... > :)
Really glad to hear this :)
The most difficult topic "Transformer Network" is explained very clearly with visual guide. Hats off to the speaker. I appreciate the efforts taken to prepare this guided tour.
Thanks a ton Usha! :)
Please make more videos explaining stuff intuitively! These are the best videos I've found on the topic and helped tremendously
Fantastic. Just finished the series without stopping.
"Layer normalization simply means standardizing neuron activations along axis of the features"
I've seen several descriptions of layer normalization, and they are usually paragraphs long. Yours is one sentence long and somehow 100x clearer.
Animations are perfect as well.
Yer a wizard 'edu
this series is AWESOME and this channel should have more suscribers!!!
This was an amazing mini series. Well done!
Amazing videos! The best out of all the transformer videos ever watched. Thank you!
These explanations are amazing!
It is amazing how you explain these complicated models in a way that is so simple to understand. It reminds me that Einstein said "If you can’t explain it simply, you don’t understand it well enough."
This is a great playlist and one of the best I have come across for transformers.
Thanks!
And looking forward to your next set of videos.
Thank you for all your videos about transformers. They are incredible. It's obvious you've worked hard on them, and they've been really instructive.
Fantastic explanation!! Thank you
Very clear and helpful! Thank you!
Your video is worth more than 1000 page book. Your 3 videos allow me to understand and implement from scratch Transformer architecture. Any gap in explanation forces me to think for myself, deepening the understanding. Thank you.
wow... this video give me perfect understand about transformer. Your teaching style is amazing, perfect, no question left.
plz make more video. I'm very thanks for your effort
This video series is the best one I have found about transformers! Seriously, a job well done! Thank you for your effort!
Great Series!
Thank you very much. Great explanation. 👌
What a phenomenal video.
This is well done, subscribed.
This is an awesome video, thank you very much!
Amazing explanation🔥🔥
Thank you
Thank you!
This is the best guide I have seen on here. :)
This was such an awesome explanation! Thank you very much!
Some video includes concept only and other includes maths. But your way to teach is just incredible. You haven't just included maths and concepts but also examples to show each detail without skipping in very less time.
Your comment made my day :)
Wonderfully explained. Looking forward for more
You've done a great job
This is an extraordinary good explanation! Thank you very much! Instant subscribe!
The best transformer explanation 👏👏
Stupendous! Thank you for this intuitive explanation, I was really struggling to get my head around Transformers.
You probably saved me for tomorrow exam, thank you so much!
You explain the difference between training and inference so well. This video deserves so many more views.
finally wait is over , thankyou maam
Thank you! It's a really incredible tutorial. Everything is explained in detail and intuitively!! Love your teaching style and it's really worth a subscription. :)
Thank you so much. May god bless you.
tysm ~~ this series really is the masterpiece
This is epic content on youtube. Please keep uploading!
This is brilliant!! Thank you very much :)
Excellent explanation.
extremely well explained
Brilliant explanation
These videos were sooo good. You should continue making more
This is JUST EPIC!! This is the best explanation out there!!
I looked everywhere but always seems something is missing.
Thank you so much for making this great Video.
Very well explained
Best teacher in the world
The best detailed explanation of the transformer. Hope to see more of this type videos.
You got the magic, of explaining very complicated material and making it so obvious, within you. Thank you for the series on the attention model, you did a Great job. 👍
You have created so great learning asset. Hats off to you.
Truly it is the clearest explanation of the transformer architecture. Please keep posting that kind of great explanations!
Best explanation on transformers I've found
You are a ROCKSTAR 💚💚💚. So lucidly explained... Shows your eloquence. Glory to you.
This is the easiest to understand explanation of Transformers with amazing graphics. I would pay for this content :)
Finally & finally, I understood Transformers!!! Best playlist that I found on Transformers!!! Thank you @Hedu AI. Hope to see you make more videos.
One of the best explantion process I have ever seen
Amazed by the way of explaination!!.Most intuitive and brilliant articulation of one of the hardest concepts!!A huge THANK YOU!!!
Please, go on making videos. This is the best content I've found on Transformers
Beautifully explained! Thanks a lot!!
You are very welcome :)
mind blowing explanation i have seen ever
Best explanation ever, Please continue to upload videos about new research papers in the area. Keep up the good work!
Thanks...so easy to understand transformers
Wow, you absolutely killed it! Your explanation is mind-blowingly easy to follow for even one who does not have any background in AI, like me. I have watched a lot of videos on this topic ever since ChatGPT hit the headline, but none of them managed to make me feel like I understood anything. No video has come close to the way you broke down the process step by step. Your use of visual aids was simply fantastic and made the whole concept so much clearer. Thank you soooooo much.
Very intuitive
Just amazing!
You are simply awesome!!!
Great Explanations!
Very very wonderful content, voice, explanation, animations...so good...thank you so much.
Really u have amazing skill to explain complex things in an easy way
its mind blowing explanation i have seen ever
Nunca había visto una explicacion tan excelente como la suya. Muchas felicitaciones. Por favor publique mas videos.
This serie is so good! PLEASE make more AI tutorials
Nicely done. Very clear and succinct. Loved the narration, and the acting parts from Game of Thrones. Huge thank you for making me understand Transformers!!🙏
Nice work!
More videos please, if this channel continues at the same level, it will become one of the most important references in the world
Wow, this brightened my day :)
Really I wonder how could you do that, simple , clear , straightforward, beautiful ..... thank you very much,,,, Keep going