Відео

very few people know these concepts well enough to give detailed explanation with formulae. thanks a ton. I was having a lot of queries and this video helped resolve those

A very clear and amazingly detailed explanation of such a complex topic. It would be nice to have more videos related to ML from you!

This is great

does 5:45 - 8:15 refer to the old RNN training method ? and hence next video is the real transformer decoder

thank you for your work, these are incredible videos. but there is one thing I didn't understand. during the training phase, the entire sentence already correctly translated is given as input to the decoder and to prevent the transformer from "cheating" masked self attention is used. How many times does this step happen? because if it only happened once then the hidden words would not be usable during training. During the training phase, after each step does backpropagation occur and then does the mask move, hiding fewer words?

pretty bad example . Even if we have trainiable Wq and Wk , what if there was a new sentence where we had Tom and and he , the WQ will still make word 9 point to wmma and she

Hi, great video! I have just a question about this. When we compute Z = KT * Q, where KT is the transpose of K, we are doing Z = (W_K * X)T * (W_Q * X) = XT * W_KT * W_Q * X. Now, calling M = W_KT * W_Q, we have Z = XT * M * X. So why we are decomposing M into W_KT * W_Q? At the end we use only the product of W_K and W_Q, so why do we learn both separately and not just learn M directly? Thank you

Need an example with BATCH being fed into that. What would be the rows in batches? What would Y looks like? Only then it is possible to really see how masks works.

The first sentence of the video solved my problem 😅 "what enables us to parallelize calculation during training"

These videos are wonderful, thank you for putting in the work. Everything was communicated so clearly and thoroughly. My interpretation of the attention mechanism is that the result of the similarity (weight) matrix multiplied by the value matrix gives us an offset vector, which we then add to the value and normalize to get a contextualized vector. It's interesting in the decoder, we derive this offset from a value vector in the source language, add it to the target words and it is still somehow meaningful. I presume that it is the final linear layer which ensures that this resulting normalized output vector maps coherently to a discrete word in the target language. If we can do this across languages, I wonder if this can be done across modalities.

Great job Dr Lennart . Everyone should learn from you

The linear production equations have operands in the wrong order.

Excellent, thank you!

Best video on masking!

Thank you for the nice explanation. I think you missed to mention that in order to get zeros masking with the softmax you need to set the values (upper triangle of the matrix) to negative infinity.

Excellent video! Thank you!

Hi Lennart, where from have you retrieved all the details that you are presenting here? I mean, have you maybe studied/analyzed a source code of an existing implementation of transformer-based models? I haven’t found anywhere else this detailed explanation. Bravo! And thank you.

What is the difference between attention and self-attention?

Intuition buildup was amazing, you clearly explained why we need learnable parameters in the first place and how that can help relate similar words. Thanks for the explanation.

Just binged the entire playlist, helped me understand the intuitions behind the math. I hope you make more videos :)

Excellent video, as all of yours are. Thank you, I have learned a lot. One thing I'm not clear on is why we need the free parameters W. And how might those be trained? Thank you again.

Would you be interested in working together, in say, creating a "better" model than tranformer models? I believe we can integrate reasoning , that would be similar to how humans reason in the higher layers of the tranformer

Hi Professor, are there lectures of courses, or weblinks to your what you teach? Love your clear, precise and well paced coverage of the concepts here! many thanks.

Beautifully explained, thank you. Transformers are so simplistic yet powerful.

this is so clear explanations, thanks so much

Thank you professor for this amazing series on the transformer!

Best UA-cam video explaining Transformer ever!

Thank you !

Thank you !

Dear Lennart, that was awesome, could you please make a tutorial in python as well? :)

best explanation! Thank you Mr. Svensson.

The best transformer video!

Best explanation of self-attention I've seen so far. This is gold.

best explanation

greatful forever

At 7:14 ,I was the notations would be sm(Z_11, Z_12) and sm(Z_21, Z_22) for the second column...... This that correct?

Very helpful. Thank you!

After the training of the model, when we are giving an unknown source sentence to the model how does it predict or decode the words?

"Revolution in visual synthesis" is an excellent label of this epoch. Good video and helps people understand how inverting diffusion helps us arrive towards the concepts of "stable" diffusion.

Thanks a lot Lennart. What a crisp and clear explanation of BERT.

Lennart, you are the UA-cam Wizard of Transformers!

Well done!

A first class explanation of self attention- the best on UA-cam.

I really love this series, Many thanks to you sir. But may I ask if I want additional sources to study the transformer from, what do you recommend?

Thanks a lot, the only complete course about transformers that I found. One question, Why K = [q1 q2 ... q_(nE)] and not K=[ k1 .... ] (or its typo?)

Very easy to follow

「動画の音が良くない」、

This is the best explanation on Transformer I have found in the web. Can you doing another set of video for T5 ?

Undoubtedly, these 8 videos best explain transformers. I tried other videos and tutorials, but you are the best.

Awesome!