Transfer learning and Transformer models (ML Tech Talks)
Вставка
- Опубліковано 16 тра 2024
- In this session of Machine Learning Tech Talks, Software Engineer from Google Research, Iulia Turc, will walk us through the recent history of natural language processing, including the current state of the art architecture, the Transformer.
0:00 - Intro
1:07 - Encoding text
8:21 - Language modeling & transformers
29:46 - Transfer learning & BERT
43:55 - Conclusion
Catch more ML Tech Talks → goo.gle/ml-tech-talks
Subscribe to TensorFlow → goo.gle/TensorFlow
product: TensorFlow - General; re_ty: Publish; - Наука та технологія
![[1hr Talk] Intro to Large Language Models](/img/n.gif)
This was one of the best videos I’ve seen explaining transformers and NL models….well done and look forward to the other videos in the series!🍻
Thank you for the awesome talk on all the main NLP models, in particular, the great explanation of the Transformer model!
Please @Tensorflow Team continue with this lecture series ML Tech series
best youtube NLP walk through without cutting corners. best delivery as well.
The comparison between attentions heads and CNN filters made so much sense!
This is the best video on youtube that introduces transformer models
After a year returning to that video finally I fully (or at least saw the entire video in a row) understand what is going on!! Maybe one more time to fix and go to the next part!! Thanxx
One of the best explanation i have come across on transformers. Thanks
Great overview and explanation of the Transformer network. I am just starting my exploration into NLP and this talk has saved me lots of time. I now know that this where I need to be focussing my attention. Thank you 👍🙏😍
One of the best explanations of transformers that I've seen!
I believe this video provides the most comprehensive explanation of transformers.
So clear. This is one of the best videos explaining transformer architecture.
After reading about language models, word embeddings, transformers, etc. for a month, this video put everything in order for me. Thanks!
I didn't understand a single word 😕
Great Video. The first Transformer explanation that (correctly) does not use the Encoder/Decoder diagram from the Transformer paper, well done!
Additionally talking about the exact outputs (using only one output for predictions) was very helpful.
Very straightforward.
Thank you so much
Best explanations so far of the attention or QKV concept... I was searching for a good way to visualize it.. Thanks a ton!!
The explanation is so clear, thank you.
Very well explained! Thank you very much. Especially loved the comparison between CV kernels and multiple QKV parameters.
A cristal clear explanation of Transformers. Papers in many cases are very difficult to follow. Pointing out the important omited details which are critical for the model, even if not explained, is very useful. Many out there try to explain transformers without having a clue of what it is. Clearly, this is not the case. Thanks in its deepest tokenized meaning for sharing your knowledge. BTW, the last programming tip is really helpful. A small hands on demo of using BERT(or any flavor of BERT) with a classifier for a particular application would be amazing for another video.
My favorite explanation so far. Great job.
Great presentation. Really well structured.
Best Video on Transfer Learning. So much clarity
awesome; the very BEST explanation on self-attention and trasformers
super professional explanation of the topic! Excellent work!
Wow that explanation actually dissipated many of my questions. Thanks a lot Julia!
Great presentation! Really easy to understand exaplanations of some hard topics, thank you.
Very clear explanation. Thank youuu
Mapping to geometry is pro. I have thought since my education about 40 years ago that current mathematics is taught incorrectly. Here is a pro example of how math should be taught!
Excellent presentation of complex NLP topic.
She's so good! I've watched a few videos attempting to explain these self-attention version of transformers and this one is by far the best in so many aspects with actual deep understanding of the architecture at the top followed closely by coherently communicating concepts, good script, presentation and graphics! I hope she narrates more videos like this... I'm about to search and find out lol! 🧐🤞 🤓
Hands down the best explanation, this after watching so many videos, terrific, Looking forward to some videos on understanding on BARD and its fine tuning
Thank you very much ! Great video and very well explained. Yes a video about sentiment analysisfine tuning would be Amazing !
Great video, it would be nice to have a video of reinforcement learning in future ml tech talks.
This is very beautifully explained!
This video initialised my attention weights for transformers,
Awesome.. great explanation. Thanks.
She has done an incredible job.
It's very important library in nlp great work
That was super interesting. Very clear explanation
Amazing talk! very informative. Thank you :)
Crystal clear .. Tnx
Excellent teaching !
Very nice topic discussion! Thank you 🙂
Great explanations, thank you so much for this video!
Glad it was helpful!
Great Presentation
Simply great! 👏👏👏
Simply awesome
Great Explanation
Agreed with all. This person should take the lead for other Google educational videos.
Thank you
amazing video !
Nice, thank you ❤
Thanks a ton for the explantion! Just wanted to ask how do we arrive at the values for matrices K, V and Q?
Thank you for shraing !
at minute 35 the video describes transfer learning, and it is said that during the fine tuning phase ALL the parameters are adjusted, not only the classifier parameters. Is that right? In contrast, when using a pre-trained deep network for a specific image calssification, I froze all parameters belonging to the CNN and just allowed the classifier parameters to vary
real help, thanks.
very good presentation!
well done video!
I expected some wishy-washy feel-good "explanation", but I'm pleasantly surprised. So far the best explanation. Goes after the relevant distinguishing key features of the transformers without getting bogged down in unnecessary details.
Great slides.
44:40: Thanks for your attention 😁
Caught that too
Great talk, really clear, thanks!
Also I see what you did "Thanks for your attention" 🤣
very nice
Interesting
nice video
Julia,
Your presentation has triggered a Eureka moment in me . What makes a great training video? Can AI help answer that. Here is a suggestion. Get a collection of videos and rank them by review comments. Using a large language model, find patterns and features and see whether there are correlations between the features and the views and review rankings. The model should be unsupervised. Some of the features can be extracted from comments
i love it
تحياتي الخالصة شكرا جزيلا
Great presentation! Are the slides available for download? This would be fantastic. Thank you.
I came here from tutorials sections of tensorflow official webpage, but i get caught by her beauty
Immaculate!
Where can I get the slides for this talk? Great talk
This is a positive comment. UA-cam should let it past it’s sentiment filter.
Can someone explain the inputs dict shown in the code at 42:15.
This is a fairly good presentation. There are some areas where it summarizes to the point where it becomes almost misleading, and at least very questionable: 1. Several other sources that I read claim that the Bert layers will have to be frozen during fine tuning, so I think it is still open for debate what the right thing to do is there? 2. This presentation glosses over the outputs of the pretraining phase. I think the output corresponding to the CLS token is pretrained with the “next sentence prediction task”. So, is this output layer dropped entirely in the fine tuning task? Otherwise I don’t see how the CLS token output would be a good input for sentiment classification. 3. The presentation suggest that the initial non contextual token step is also trainable and fine tunable. Isn’t it just fixed byte pair encodings? I know that these depend on frequencies of letters in the language but can these be trained in process with Bert? 4. This presentation equals transformers very silently to transformer encoders, and thus drops the fact that transformers can also be decoders. I think all initial transformers were trained on sequence to sequence transformation, and then the decoders were trained on next token prediction giving rise to things like GPT, whereas the encoders were trained on a combination of masked token prediction and next sentence prediction giving rise to the BERT like models.
👍🏻👍🏻👌
Thanks for your ATTENTION 🤗🤗.. Pun intended!44:39
Where can I get the GitHub file
Please provide the slides
Sorry to say, but this was not very good. Key information is missing mostly the WHYs ? why is there a need for Query and Key Matrices? what is the main function of these matrices? How does the Attention function alter the Feedforward NNs?
Thanks for not calling Sentiment Classification as Sentiment Analysis!
okay what you are saying is completely vague . like for the query matrice you mentioned ( some other representation [why do we need another representation at all ?])
I always find the face of presenter distracting when it is on the slides … can you just talk over slides instead of covering them with presenter’s face??