The Emergent Abilities of LLMs - why LLMs are so useful
Вставка
- Опубліковано 21 тра 2024
- LLMs have been shown to have abilities which they were not trained for. For example, LLMs can translate between languages without being directly trained to do so. These abilities have been shown to rapidly appear once the LLMs gets to a certain "critical size".
These special abilities are called the Emergent Abilities of LLMs - appearing to emerge at a particular scale. In this video, we will learn what Emergent Abilities are, how they were discovered, why they are important, and some potential explanations for why they appear.
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬
🖥️ Website: www.assemblyai.com/?...
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: ua-cam.com/users/AssemblyAI?...
🔥 We're hiring! Check our open roles: www.assemblyai.com/careers
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
What are emergent abilities of LLMs?
- Emergent abilities are abilities that LLMs can complete without being explicitly trained to do so. These abilities appear rapidly once LLMs are scaled to a large enough size.
Why are emergent abilities important?
- LLMs have been rapidly adopted in the last year because of their incredible versatility. While they are not perfect, they demonstrate competency on a wide range of tasks which makes them useful for many types of applications.
What accounts for emergent abilities?
- There is not a single explanation for emergent abilities, so additional studies are needed to form a more conclusive answer to this question. There are some potential explanations for emergence, like multi-step reasoning and misaligned evaluation metrics.
0:00 Discontinuous learning
0:46 Background
1:06 Scaling language models
1:47 Discovering emergence
2:25 Emergence as a general concept
4:00 Emergence in LLMs
5:47 Emergent abilities: fact or illusion?
7:52 What does this all mean?
9:08 Final words
9:35 Outro
#MachineLearning #DeepLearning - Наука та технологія
What a great video for such niche concept. Im so glad you posted this!
Parameter count log scale exaggerating the suddenness but ok.
I love the idea that the model doesn't see the difference between learning spelling, grammar, semantics or high level science, it just needs to get better at predicting the next word.
Good video!
This makes me think the corpus contains all the abilities already, and LLMs can access them at certain scales. Text is like a condensed report of human experience. All the experience we have collected in the corpus is feeding this process. Model architecture, as long as it can do sequence modeling, doesn't matter.
So interesting to see the ai eye contact/gaze in action
Wow - very good video! Topic that requires deep understanding very well & clearly explained. Thank you!
Glad you enjoyed it!
great video, lots of new good info for me
Glad to hear it!
great!!
Related: Scientists are yet to explain the evolutionary emergence of human consciousness from the building blocks of life, amino acids, RNA etc. I think research into emergent abilities of AI/LLMs could give rise to some interesting theories for the life sciences.
I recently read an article saying this emerging capabilities is kinda illusion, it is more linear than non linear
If your x axis is logarithmic like that, even a linear trend will appear exponential.
If I may ask, for my own understanding, would this mean that let's say you have a ransom note, and you want to find out who the author is. You know it's one of 300 people. The more writing samples you insert into the training data, the closer that you will get to a possible emergent phenomona of being highly accurate at who was the author?