Transformers, explained: Understand the model behind GPT, BERT, and T5
Вставка
- Опубліковано 1 тра 2024
- Dale’s Blog → goo.gle/3xOeWoK
Classify text with BERT → goo.gle/3AUB431
Over the past five years, Transformers, a neural network architecture, have completely transformed state-of-the-art natural language processing. Want to translate text with machine learning? Curious how an ML model could write a poem or an op ed? Transformers can do it all. In this episode of Making with ML, Dale Markowitz explains what transformers are, how they work, and why they’re so impactful. Watch to learn how you can start using transformers in your app!
Chapters:
0:00 - Intro
0:51 - What are transformers?
3:18 - How do transformers work?
7:41 - How are transformers used?
8:35 - Getting started with transformers
Watch more episodes of Making with Machine Learning → goo.gle/2YysJRY
Subscribe to Google Cloud Tech → goo.gle/GoogleCloudTech
#MakingwithMachineLearning #MakingwithML
product: Cloud - General; fullname: Dale Markowitz; re_ty: Publish; - Наука та технологія
Ability to break down complex topic is such an underrated super power. Amazing job.
Transformers! More than meets the eye.
😂
Transformers! Robots in disguise!
Autobots wage their battle to fight the evil forces of the Decepticons!!!!!
Transformers! No money to buy…
Oczywiście
How did you condense so many pieces of information in such a short time? This video is on a next level, I loved it!
This is awesome. This has been one of the best overall breakdowns I've found. Thank you!!
This is a really awesome video! Thank you so much for simplyifying the concepts.
Great explanation of the key concept of position encoding and self attention. Amazing you get the gist covered in less than 10 minutes.
@Dino Sauro tell me more...
@Dino Sauro thanks for the heads up
She has one of the wealthiest company on earth providing her resources. First hand access to engineers, researchers, top notch communicators and marketing employees.
@@an-dr6eu True, but this young lady talks a mile-a-minute from memory. She's knows it cold regardless of the resources at Google.
This is such an informative video about transformers in machine learning! It's amazing how a type of neural network architecture can do so much, from translating text to generating computer code. I appreciate the clear explanations of the challenges with using recurrent neural networks for language analysis, and how transformers have overcome these limitations through innovations like positional encodings and self-attention. It's also fascinating to hear about BERT, a popular transformer-based model that has become a versatile tool for natural language processing in many different applications. The tips on where to find pertrained transformer models and the popular transformers Python library are super helpful for anyone looking to start using transformers in their own app. Thanks for sharing this video!
Dale you are so good at explaining this tech, thank you!
You have the gift of making things simple to understand. Keep up the good work 🙏
Amazing video! Nice explanation and examples 😄👍
I would like to see more videos like this and practices ones
I loved it and very simple ,clear explanation.
i really enjoyed the concepts you explained. simple to understand
So easy and clear to understand. Thanks
Thanks you did a great job. I spent some time already looking at different videos to capture the high level idea of what transformers are about and yours is the clearest explanation. I actually do have an educational background in neutral networks but don't go around remembering every details or the state of the art today so somebody removing all the unessesary technical details like you did here is very useful.
Amazing video! 🎉 You explained that difficult concepts of Transformers so clearly and made it easy to understand. Thanks for all your hard work!🙌👍
Are you serious? The concepts were not really explained. Just a summary of what they do but not how they work behind the scenes.
No.
This is a GREAT explanation! please lower the background music next time it could really help. thanks again! awesome video
Wow, this is so well explained.
Love how you simplified it. Thank you
It s so simplified that you can t understand anything
I love how to simplify something so complex, thank you so much Dale, the explanation was perfect
how did you do that
@@decepticon-barricade934 This one? Just type ":" (colon) followed by "thanksdoc" and end it with another colon. I can add other emojis like 🤟too!
@@nahiyanalamgir7056 it needs desktop UA-cam i think
@@decepticon-barricade934 Apparently, it does. When will these apps be consistent across devices and platforms?
@@nahiyanalamgir7056 thanks though
Love the content and thanks for the great video! (one thing that might help is lower the background music a bit, I found myself stopping the video because I thought another app was playing music)
This was a really, really awesome breakdown 👏🏾
Charm, intelligence and clarity! Thanks!
Fantastic!. Thanks for simplifying the concept
This is a very well produced video. Credits to the presenter and those involved in production with the graphics
That's a really good high-level explanation!
This is one of the best vids I've watched on this topic!
I have more respect for Google after watching this Video. Not only did they provided their engineers with the funding to research, but they also let other companies like OpenAI to use said research. And they are opening up the knowledge for the general public with these video series.
Very well explained.. This really is a high level view of what Transformers are, but it's probably enough to just get your toes wet in the field!
Excellent presentation and explanation of concepts
Thank you so much. I really needed this video, other videos were just confusing
Simply loved it!
Such a simple yet revolutionary 💡idea
Nice amount of info parted in this video. Very clear info on what Transformers are and what made them so great.
Amazing explanation!
Positional Encoding, Attention and Self Attention. That's it! Really well summarized.
Hi Google! First of all, thank you for this wonderful video. I'm working on a multiclass (single label) supervised learning that uses Bert for transfer learning. I've got about 10 classes and a couple hundred thousand examples. Any tips on best practices (which Bert variants to use, what order of magnitude of dropout to use if any)? I know I could do hyperparameter search but that'd probably cost more time and money than I'm comfortable with (for a prototype), so I'm looking to make the most out of my local Nvidia 3080.
Informative! Thank you
Thanks for your hard work.This video is very helpful!!!
This is an excellent video introduction for transformers.
super well done. Thanks for this!
so super helpful for my thesis, thank u
thank you! I'm just starting to learn about gpt and this was quite helpful, though I will have to watch it again :)
Very interesting, informative, this added perspective to a hyped-up landscape. I'll admit, I'm new to this, but when I hear "pretrained transformer" I didn't even think about BERT. I appreciate getting the view from 10,000 feet.
wow, what a great summary! thanks!!!
Takeaways:
A transformer is a type of neural network architecture that is used in natural language processing. Unlike recurrent neural networks (RNNs), which analyze language by processing words one at a time in sequential order, transformers use a combination of positional encodings, attention, and self-attention to efficiently process and analyze large sequences of text.
Neural networks, Convolutional neural networks (for image analysis), Recurrent neural networks (RNNs), Positional encodings, Attention, Self-attention
Neural networks: A type of model used for analyzing complicated data, such as images, videos, audio, and text.
Convolutional neural networks: A type of neural network designed for image analysis.
Recurrent neural networks (RNNs): A type of neural network used for text analysis that processes words one at a time in sequential order.
Positional encodings: A method of storing information about word order in the data itself, rather than in the structure of the network.
Attention: A mechanism used in neural networks to selectively focus on parts of the input.
Self-attention: A type of attention mechanism that allows the network to focus on different parts of the input simultaneously.
Neural networks are like a computerized version of a human brain, that uses algorithms to analyze complex data.
Convolutional neural networks are used for tasks like identifying objects in photos, similar to how a human brain processes vision.
Recurrent neural networks are used for text analysis, and are like a machine trying to understand the meaning of a sentence in the same order as a human would.
Positional encodings are like adding a number to each word in a sentence to remember its order, like indexing a book.
Attention is like a spotlight that focuses on specific parts of the input, like a person paying attention to certain details in a conversation.
Self-attention is like being able to pay attention to multiple parts of the input at the same time, like listening to multiple conversations at once.
Great, you learned how to copy paste
@@an-dr6eu first step on becoming a programmer
@@an-dr6eu your comment comes over somewhat 'catty' 😢
You have no idea how much time I potentially have saved just by reading your blog and watching this video to get me up to speed quickly on this. "Liked" this video. Thanks
very well explained.👍
Soo cool! Great work
Thanks! This is a great intro video!
I knew little on transformers before this video. I know little on transformers after this video. But I guess in order to know some, we'll need a 2-3 hours video.
Easiest to understand explaination ive heard so far
Super Explanation!!
Simplest Explanation ever
Excellent explanation i ever seen, recommending everyone's this link
this is brilliant
The visuals are very helpful. Thanks.
You're very welcome!
phenomenal video
do transformers learn the internal representation one language at a time or all of them at the same time? I remember that Chomsky said that there's no underlying structure to language and that for every rule you try to make you'll always find an edge case that contradicts the rule.
Great video. Thank you!
Very well explained. Thank you.
Very good lecture, thanks!
crazy how things have changed so much
Nicely done. Very helpful. Thanks!
10/10. Very helpful
Thank you for sharing
Very well explained. This video is must watch for anyone who wants to demystify the latest LLM technology. Wondering if this could be made into a more generic video with a quick high-level intro on neural networks for those who aren't in the field. I bet there are millions out there who want to get a basic understanding of how ChatGPT/Bard/Claude work without an in-depth technical deep dive.
NICE SUPERB PRESENTATION
Great video.
Amazing video, thank you so much!
Thanks! Great video.
Great video for people who are curious but don’t really want to (or can’t) understand how transformers actually work.
Very informative video. Thank you!
OMG the BEST transformers video EVER!
Thank you
great video, thanks!
From 5:28, shouldn't it be the following:
"when the model outputs the word “économique,” it’s attending heavily to both the input words “European” and “Economic.” "?
For européenne, I see that it is attending only to European. Please let me know if I am missing something here. Thanks for the great video.
Thanks, that was very interesting
Great content 👍
Very impressive video. Thanks for the way you shared information via this video.
Reference your video timeline 05:05, how you created such a video, please.
Amazing!
Well done
Great video. Thx.
Thank you!
Fantastic video
a very nice video. thanks
You have actually given the BEST explanation on Neural Machine Translation that I read so far but you are missing a few elements
But your explanations, your analyses and your delivery are excellent. You're definitely a great communicator and teacher.
Actually Google and others have an algo they're not interested in sharing and I pretty much know what it is. I am working with my programmer on the coding of my new app, the revolutionary Universal Sentence builder and the Universal Dictionary and I keep adding and changing stuff to simplify the concept and I push at a later date the programming of my Sentence Analyser app. It is like most of my apps a simple (and brilliant concept) coded with very few lines of code.
You know Alfred Hitchcock was always adapting into the screen his scenario never changing anything not even a comma while Francis Ford Copolla (The Godfather) was doing the opposite: They say that his script was like a newspaper that had new contents every day. Well I am more like Copolla with my apps. I change stuff all the time and I usually make my programmers go crazy. It's a good sign. :-) Mind you I don't know if one can do like Hitchcock with an app. Come up with a definite version once and for all. This would be quite an achievement!
In the case of my Universal Sentence builder, the main task was to process the data entered by the user and we've been at it since July 2022. :-) It's either I am dumb or it is a complex task. Actually it is the latter for I have started with French, this langage being the most complex in the world. The good news is I am sure I will be imitated but you can rest assured that my imitators will also have a jolly hard time with French :-)
When I saw this title, I was hoping to better understand the mathematical workings of transformers such as matrices and the like. Maybe you could do a follow-up video explaining mathematically how transformers work.
thank you for your time
Where is optimus prime?
He's on the thumbnail...
He will be in theaters in June 9... Transformers : Rise of breasts..
😂😂😂😂
Where are robotaxis?
We got lied to
woww, she's good at explaining things
You are amazing!
I'll jump on where others are doing the same - would love advice for someone who understands half the concepts that are alluded to as complex naturally and the innovation feels obvious I'm unsure how to break into the space without some guidance or connection between having exactly that great natural grasp but wildly anxious that language and logic are strengths and math is a mental turn off. For someone needing that type of translation/guide where my approach is language usage & finer cues what is the key terms to get to that understanding? Hate being fascinated and all the tools to play in this space and being unable to start because how I approach topics so welcome any advice.
Just go to school.
Positional encoding = time, attention = context, self attention = thumbprint (knowledge)... looks like a good start for AGI 😀
Good(Pro) Explanation.
Thanks a lot.
How did you sync your talking cadence to the background music?
be interested to see a video on transformers on time series data.
Please remove background music, it's really disturbing when you only listen to this otherwise great video
When I was a kid, I knew the trouble of translation were due to literally translation words, without contextual/ sequential awareness. I knew it's important to distinguish between synonyms. I've imagined there's a button that generate the translation output then you can highlights the you words that doesn't make sense or want improvement on it . then regenerate text translation. this type of nlp probably exist before I program my first hello world (+15y ago)!
Thanks!
great video