What are Transformers (Machine Learning Model)?
Вставка
- Опубліковано 10 бер 2022
- Learn more about Transformers → ibm.biz/ML-Transformers
Learn more about AI → ibm.biz/more-about-ai
Check out IBM Watson → ibm.biz/more-about-watson
Transformers? In this case, we're talking about a machine learning model, and in this video Martin Keen explains what transformers are, what they're good for, and maybe ... what they're not so good at for.
Download a free AI ebook → ibm.biz/ai-ebook-free
Read about the Journey to AI → ibm.biz/ai-journey-blog
Get started for free on IBM Cloud → ibm.biz/Bdf7QA
Subscribe to see more videos like this in the future → ibm.biz/subscribe-now
#AI #Software #ITModernization
In our graduation years we used to learn about something called codec, as in coder-decoder (something like modem for modulation-demodulation or balun for balanced-unbalanced in the domain of communication technology. So as I can understand from the video is that the transformers are just a fancy and advanced name for a codec, which functions at much bigger capitalistic scale.
Transformers are a type of machine learning model used primarily for natural language processing (NLP) tasks. They have revolutionized the field of NLP due to their ability to handle long-range dependencies and capture complex linguistic patterns. Here are key points about transformers:
1. **Attention Mechanism**: Transformers use an attention mechanism that allows them to weigh the importance of different words or tokens in a sequence when processing input data. This mechanism enables the model to focus on relevant information while ignoring irrelevant or redundant parts.
2. **Self-Attention**: In a transformer model, self-attention refers to the process of computing attention scores between all pairs of words or tokens in an input sequence. This mechanism allows the model to capture dependencies between words regardless of their positions in the sequence.
3. **Multi-Head Attention**: Transformers often employ multi-head attention, where multiple attention heads operate in parallel. Each attention head learns different aspects of the input data, enhancing the model's ability to extract meaningful information.
4. **Encoder-Decoder Architecture**: Transformers typically consist of an encoder-decoder architecture. The encoder processes the input sequence, while the decoder generates the output sequence. This architecture is commonly used in tasks like machine translation and text generation.
5. **Positional Encoding**: Since transformers do not inherently understand the order of tokens in a sequence like recurrent neural networks (RNNs), they use positional encoding to provide information about token positions. This allows the model to consider sequence order during processing.
6. **Transformer Blocks**: A transformer model is composed of multiple transformer blocks stacked together. Each block contains layers such as self-attention layers, feedforward layers, and normalization layers. The repetition of these blocks enables the model to learn hierarchical representations of the input data.
7. **BERT and GPT**: Two popular transformer-based models are BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pretrained Transformer). BERT is designed for tasks like sentiment analysis and question answering, while GPT focuses on generating human-like text.
Transformers have significantly advanced the capabilities of NLP models, leading to breakthroughs in areas such as language translation, text summarization, sentiment analysis, and dialogue systems.
you put great effort on writting this
Chatgpt generated
Thanks you for your enthusiasm and explanation , by far the best
Hi Martin from the Homebrew Challenge! ML and beer clearly go hand in hand!
I really love your videos as they are really friendly to understand. Really graceful for the high quality of the synthesis of key messages on AI/ML/DL. I am a medical doctor and biomedical researcher. I can see the great potential of using the different technics to further develop a bunch of areas, for example: economic evaluations based on modeling (using a combination of approaches in the sensitivity analysis to find out the internal consistence of the predictions…to gain internal validity as a cornerstone to have external validity). So, looking forward to learn more through your channel.
Thank you, again for sharing good quality knowledge.
L.
Congratulations to all the team work!, I will keep learning more. Thank you all, Leslie.
What a great way to introduce the topic. First 4 seconds made me laugh out loud. Well done (and the rest of the video as well)
Dr. Ashish Vaswani is a pioneer and nobody is talking about him. He is a scientist from Google Brain and the first author of the paper that introduced TANSFORMERS, and that is the backbone of all other recent models.
Agreed
He should be documenting his work like our guy, and make interesting vids.
Hope it happens.
like to see the energy which you put on to it, Thanks for this.
I have been more then blown away by the unfathomable exponential growth from just increasing transformers an a few weights lol
Banana joke GPT-4:
Sure, here's a banana joke for you:
Why did the banana go to the doctor?
Because it wasn't peeling very well!
Good transformer 🤣
is the fact that he is able to write text mirrored incredible or is there a simple trick here?
There is a trick. Hint: he's not left handed.
its flipped and rotated, done through editing
@@IBMTechnology yeah, I though he can't be left handed.
@@vaibhavthalanki6317 it's not a glass, it's a mirror I think.
@@IBMTechnology thought he was left handed
Thank your video,your video really easy understand
Thanks for this video - a simple and concise introduction to transformers.
Do large language models really possess reasoning capabilities? Or, the way they operate makes it seem so.
Thank you!
How does the transformer take something of variable length (like a sentence) and shove it into a neural network (which requires a fixed number of inputs)?
Generic NNs take only fixed inputs but this is one of the specialities of these types of models! RNNs (the older model used for NLP) were created back in the 80s addressing mainly this issue, along with memory being important for sequences. LSTMs n now transformers came in to solve the issues with RNNs
Actually, I'm interested in the hand writing presentation style. How is it made ?
Great video! I'm stumped on how you made this. Did you really write backwards? Can you reveal your magic trick?
You write it naturally and then flip the video when editing.
Exactly what's gng on in my mind lmao
This was greate!!!
Are encoders and decoders both RNN? Plz clear my doubt.
This is the pinnacle performance of training.
The Transformer technology is the reason why you see AI everywhere.
how do i use transformers on a new pair of language?
How do you get loads of loss on on a neural network in given ways for analytics
Need Explanation for GRU , BERT , LSTM
I think he write on the glass normally and the camera got it backword so they montage it to be flipped so the written words can be shown notmally.
You are mirror writing, Great skill🤩
Hi, what does an autoregressive language model mean?
can we use transformers over spacy for NER?
how can text algorithm (transformer) work in image domain like vision transformer over CNN
Transformers are being used in many ways. For example you could take a bunch of vectors (representing image features extracted from Convolutions) and feed them into Transformers to decode as text. This gives you a lot of power combining the NLP and Computer Vision Domain
@@ChocolateMilkCultLeader
Generic features or specific?
@@strongsyedaa7378 what do you mean?
Sir Will u give me a research topic in transformer
“Before too long, they might even be able to come up with jokes that are actually funny.”
2 years later, here’s the banana joke ChatGPT 4 (already 1y old) came up with for me.
> Why did the banana go to the doctor?> Because it wasn't peeling well!
I think we can call that a win.
So is it like ... a layered, parallelized autoencoder?
Does this mean they can fix my adhd?
I don't quite know why but all this transformer tech helps me understand my own glitched mind better
How are you able to write a mirror image of the words so effortlessly? :O
Hey its the guy from the beer channel...
Things are judged by their appearance. And this video looks way way better than it actually is. That explains the views.
Can we use this method to detect outliers in time series data
While you can use Transformers for Time Series, I'm not sure why you'd want some network architecture to look for outliers instead of regularizing it and let the network learn to ignore those during optimization.
Transformers are a bit overkill for anomaly detection. A lot of time more traditional methods might perform better faster (especially if the resources for training the models are constrained like not having dedicated chips or an insufficient amount of training data)
Perhaps the AI made the banana joke as a subtle way to tell us humans that we are a cruel species that mash anything we come across. The AI finds it funny because the banana would rather cross the road and take on the high likelihood of being mashed violently by a vehicle to avoid the certain mashing by humans. Perhaps the AI identified with the banana 🤔
Next level empathy: thinking about a banana's perception of reality 🧠
Q
You guys are overthinking it. 😁
that's how I interpreted it too - like yeah, the AI knows the banana doesn't want to be mashed by a car, neither do I
No, you’re definitely overthinking it. The AI doesn’t think anything because it is incapable of such context like “we are a cruel species that mash anything we come across”. Unless you specifically input that in the prompt, it has no mechanism to even conceive of the phrase.
Lol, i find the banane joke funny :)
This was prophetic. I wonder whether at that time you realized that Transformer would revolutionize the world.
Aged like a wine!
Why did the attention mechanism NOT cross the road? Because it was paralyzed!😜😁
BTW did I hear that part correctly near the end of the video?
Transformers: More than meets the eye...
In reference to the summary of an article example, How does that work? How does the program know to summarize the article and not continue it?
Also, how do you go from language processing to playing chess or other games or functions?
I'm not a machine learning expert so I can't verify the validity of this answer, but from my POV I think these questions about "how the program... instead of..." is generally dependent on
1. The actual architecture of the model (in this case, a transformer)
2. The input data it's based upon (text vs maybe piece type and board position labels for a chessboard)
3. The output data it's trying to predict (predict a summary text vs predict the next words in an article)
Because such supervised/semi-supervised learning models learn off labelled data, (to a certain extent, for semi-supervised learning), all the model is really doing is mapping an input to an output. Think of it like a maths graph (which is actually exactly what it is); given a dataset with many points, you'd want to find a "best fit" line that models the rough trend accurately without over or underfitting. Machine models do this but on many axes (due to the use of vectors, some with just an insane number of dimensions).
Of course there are many other things like hyperparameters, activation functions, loss functions, and nuanced variables to each model architectures, but hopefully this gives you a good understanding of ML in general.
A summary is a continuation of the text in that case. Consider a webpage on the internet which has an article and then at the bottom of the page it says, "here is a summary of the key points we learned above" and it goes on to summarise. This is an example of the kind of content the AI is trained on. So as long as you do some Prompt Engineering then you can ask your question in such a way that the answer comes from completing the text! It's like magic! 🙂
@@xerxel69 Yeah, articles do often contain a summary section at the end. Or parts of an essay say, "To summarise so far". Not sure if it can learn this totally unsupervised. Mu guess is summaries are a popular feature - so they will train it specifically to look for them and learn from them i na focused way. Not sure though.
Do I get this right, that a transformer is a special case of a state machine, which is designed to learn on, or update it's weights on demand, and is still general enough to cover most data?. Wouldn't an FPGA be optimal to implement such a state machine in flip flop, so that you can generate with 100mhz.
It really all boils down to performing matrix multiplications. GPUs are best at that. An FPGA can be a GPU if it wants to (:
Where can I summon autobot?
Correct me if I'm wrong but it seems that translating a document would require a human doing Quality Control right before publishing. Transformers are impressive in how close they come to mimicking humans but they seem to be The Great Pretenders. Now, how does that QC step get implemented in real time?
IBM: Next-Level Tech explained.
Chat: How does he write backwards on that invisible board?
See ibm.biz/write-backwards
I just have to say it
TRANSFORMERS MORE THEN MEETS THE EYES!
this dude can write reversely. so awsome
ha. it looks the right way around to him. The final image is inverted in the video we see. Fun trick.
the joke was just too deep for your puny mind to get
I do searches for Transferormer in Machine learning.and in my mind same those transformer there and video starts with the same.
Is he writing on reverse so we can see it correctly?
Latest update on banana humor of AI
Why did the banana go to the doctor?
Because it wasn't peeling well! - GPT 3.5 11th January 2024, 23:06 IST
the banana … skidded …
it wanted to split
I don't like people ripping Me off, whether IBM or Google.
Are you guys open to Guest Speakers
You do realize the joke about the chicken crossing the road is a suicide joke right? He wanted to get to the other side?
Now it can indeed write funny banana jokes!!
Indian SME's might be able to create this and be a unicorn. Easily.
But how does the human write backwards?
See ibm.biz/write-backwards
Optimus Prime
I came here to understand how on earth he writes backwards or what camera trickery I am obviously missing, LOL.
See ibm.biz/write-backwards
@@IBMTechnology LOL thanks!!! I suppose it shouldn't surprise me there is a video about that. Very cool and elegant technique.
The banana joke is an instance of an “anti-joke”… just like the chicken joke.
And with this simple idea the civilization ends. No, kidding, the AI will be so smart, it will leave us alone as we will be like bugs to it.
I dont think the joke was that bad. Picture meatwad from AquaTeen Hunger Force, but very pale beige.
Instead of the content I started thinking how this guy writing in opposite direction 😭😂😂 Is this some AI trick or fr?!
See ibm.biz/write-backwards
They've got better jokes now. 😂
it doesn't "come up" with a thing, it regurgitates what it's learned. It's nothing but a copy machine and being made out to be much more than it really is by all the AI hype machine artists.
not very descriptive.. it is for those who already are studying deeply about sequencing, encoder decoder etc.
may be i am not smart enough to understand..
I didnt get it
Why do they always translate English sentence to French in every video that explains Transformers :D
🤣😂
Well, jokes are hard.
Kids take several years to learn how to be funny.
KIDS ARE OBSOLETE, AI IS BETTER
I can't concentrate I can't understand how he manages to write backwards
ok but how about a more detailed explanation ?
wait, does this guy write backwards?
See ibm.biz/write-backwards
are you writing backwards in real time? because if so..... 🤯
See ibm.biz/write-backwards
@@IBMTechnology one of the few times in my life I wish to be lied to 😂
ask it why did the potato cross the road
The joke would’ve worked if it was a potato. Pretty close though.
Did I miss something? This didn't seem to give any clue as to how transformers actually work.
Your skills in writing backwards were really distracting.
See ibm.biz/write-backwards for how it's done
You didn't really explain anything.
didn't find it helpful to conceptually understand transformers
To reach the other bunch. chat gpt3.5