The moment we stopped understanding AI [AlexNet]
Вставка
- Опубліковано 30 вер 2024
- Thanks to KiwiCo for sponsoring today's video! Go to www.kiwico.com... and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate.
Activation Atlas Posters!
www.welchlabs....
www.welchlabs....
www.welchlabs....
www.welchlabs....
Special thanks to the Patrons:
Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti
Welch Labs
Ad free videos and exclusive perks: / welchlabs
Watch on TikTok: / welchlabs
Learn More or Contact: www.welchlabs....
Instagram: / welchlabs
X: / welchlabs
References
AlexNet Paper
proceedings.ne...
Original Activation Atlas Article- explore here - Great interactive Atlas! distill.pub/20...
Carter, et al., "Activation Atlas", Distill, 2019.
Feature Visualization Article: distill.pub/20...
`Olah, et al., "Feature Visualization", Distill, 2017.`
Great LLM Explainability work: transformer-ci...
Templeton, et al., "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet", Transformer Circuits Thread, 2024.
“Deep Visualization Toolbox" by Jason Yosinski video inspired many visuals:
• Deep Visualization Too...
Great LLM/GPT Intro paper
arxiv.org/pdf/...
3B1Bs GPT Videos are excellent, as always:
• Attention in transform...
• But what is a GPT? Vi...
Andrej Kerpathy's walkthrough is amazing:
• Let's build GPT: from ...
Goodfellow’s Deep Learning Book
www.deeplearni...
OpenAI’s 10,000 V100 GPU cluster (1+ exaflop) news.microsoft...
GPT-3 size, etc: Language Models are Few-Shot Learners, Brown et al, 2020.
Unique token count for ChatGPT: cookbook.opena...
GPT-4 training size etc, speculative:
patmcguinness....
www.semianalys...
Historical Neural Network Videos
• Convolutional Network ...
• Perceptron Research fr...
Errata
1:40 should be: "word fragment is appended to the end of the original input". Thanks for Chris A for finding this one.
Thanks to KiwiCo for sponsoring today's video! Go to www.kiwico.com/welchlabs and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate.
AI today is growing exponentially, just curios, do you think we will ever hit a peak where innovation on AI will start to flat out, or hit a wall, and if so where and when do you think AI will hit its peak.
You kind of skimmed over this in the end, i just wanted a bit of a longer explanation.
Oops, I farted 4o
Was really keen to sign up for a crate for my daughter, but after 10 minutes of glitches on their system I just gave up. Not your fault of course, but you might want to let them know that their system is pants.
could have done it with ASIC a long time ago.
Just living out THEIR best life possible first...
These days non-adsense being placed some ways into video, rather than with mutual consideration at very front/back/both where plenty people and myself would still watch, is instant skip/cliff off - though I wish success
30 years ago, I used to work with an older guy who retired from IBM. I was barely out of high school, and he used to tell me that neural networks were going to change the world once people figured out how to train them properly. He didn't live to see his dream become reality unfortunately, but he was totally right.
Same except the guy was at Toronto and his name was Hinton
@@spartaleonidas540 guy I knew was named David Modlin. I wonder how many folks who had their prime years in the 60's and 70's saw this coming? I wish they had been able to see it. I suppose some of them might have lived to see it. Crazy to think about.
@@EdgarVerona Hinton's most important contributions came from the 80s onwards, but he has lived to see it, for one. He was working on neural nets in the 70s as a postdoc. It's all happened well within a human lifetime, is the crazy part.
@@squamish4244 Nice, that is very cool. Glad to hear he's still kicking! The guy I knew helped create handwriting recognition software in the 70's. It is crazy to think that someone could see basically the dawn of modern computing and also its progression to this crazy time we're in.
@@EdgarVerona Ray Kurzweil did too, but he's obsessed with mind-uploading, cryonics and resurrecting digital copies of his father etc. which is distracting, and he has trouble with being challenged on the practical implications of his predictions. He was right about the computing revolution but he's also a very strange dude. Hinton was running circles around him recently in a debate when both of them were onstage.
"one way to think about this vector, is as a point in 4096 dimentional space"
give me a minute, I now gotta visualise a 4096 dimentional space in my head.
Are you done yet ? 🙂
"One way to think about a point in 4096 dimensional space, is as a vector"
high dimensional spaces are crazy. A hypercube with the sides size=2, would have absolutely enormous surface and volume in 4096 dimension.
size = 1, volume: 1
size = 1.01, volume, approx 501587856585103488.
@@adamrak7560what does volume mean for non-3d thing?
Easy, image a 3-dimensional space and pretend it's 4096 dimensions.
I mean, that's basically what the visualizations in the video are doing.
That real-time kernel activation map was life-changing.
If, whilst editing these videos, you've ever questioned if the vast amounts of effort are worth what amounts to a brief, 10s clip, just know that it's these moments which have stuck with me. Easy sub
Ikr, shows the hard work of this guy and that is something I respect.
I wanted to say this too. You actually did it, you make that animation. That is an amazing thing you've done, you've really added to the sum of human knowledge.
The amount of effort must have been phenomenal. Really: thank you. Nobody else has done this. I know the effort of huge, but I'd love more even on just Alexnet. Animations on creating the node activation image generation.
I'd love one of Resnet
The irony being, of course, that the script popped out of ChatGPT in about three seconds, editing by submagic slightly more, and images by stable diffusion in much less. But I agree, those few moments are worth it.
@@JoseJimeniz While I'm sure it took a lot of work, someone else already did most of the work for the Activation Atlas.
Same, this was truly eye opening
I've been in the field for 10 years and never had anyone describe this so clearly and visually. Brilliant, thank you!
same here (9 years)
Would you say it is still worth it going into the field (studying AI) even after progress is made so incredibly fast nowadays that after the maybe 3-4 years of studying everything could have already changed again?
@@TheRealMcNuggs I say, if you love it (or have a strong interest) then absolutely! It's been changing quickly since I started, but the underlying fundamentals stay the same 👍
3blue1brown made a whole gen ai series which goes much deeper and visualises things better, I do recommend to have a look, really interesting stuff
Im still confused 😭
Computers not being fast enough to make a correct algorithm practically usable reminds me of Reed-Solomon error correcting codes. They were developed in 1960 but computers were too slow for them to be practical. They went unused until 1982 when they were used in Compact Discs after computers had become fast enough.
RS codes were used on the Voyager probes in 1977. CDs were the first large scale usage. Your basic point is still true: it took a while for computers to be complex enough to use them.
Bayesian models have followed a similar path; the basic idea is so fundamental as to be trivial, but actually using it in practice requires a high level (uh, I don't know what the big-O complexity is -- quadratic? worse?) of detail and thus computation to truly harness. The parameters might be trivial (individually, or conceptually), but there are so many of them for a problem of modest scale that it's only recently we've made much use of it.
@@jimktrains0 I should have specified first wide-spread use.
Logix programming same prediction, eill be viable in a yeR year and I will do it ???😮❤😂🎉🇨🇭😘💶💶💶🍆🍑🍆🥑⛔⛔⛔🪬🤣😅🏳️🌈✡️💪🏾👯♂️♂️🔯✡️🔯👬🕎♀️⛔
It has always been an easy decision tree. Will the interesting case fit in system memory at all? It not, wait for the next system refresh. Can I tolerate the latency? Predicting tomorrow's weather a week from now is a good example of not being able to tolerate the latency. If it fits in memory and I can tolerate the latency, am I willing to pay for the computer time?
I recall hearing stories in the 1980s about a power station with an entire Vax 11/780 devoted to running an FFT kernel on generator shaft vibration. There was no legal way to ship a replacement shaft. They had barely been allowed to truck in the first one over existing roads. Hence they spent the moon looking after the one they had.
Most people think AI is a brand new technology, while in reality there have been studies on Computer Neural Networks all the way back in the 1940s, that's insane.
But the real issue is that only now has computing power become strong enough to support everything, allowing research ideas to be realized into reality, and truly transforming these ideas into technologies with such astonishing effects.
@@louis-dieudonne5941 makes you think, what are we studying now that will only be possible years in the future because of the lack of resources.
It’s new in the sense that neural networks are relatively inexpensive and for the first time broadly applicable to a wide range of tasks.
@@empathogen75 Its just a popularity phase, UA-cam paid for itself when it was rapidly gaining users, we'll have Adobe level subscriptions in the future.
@@louis-dieudonne5941Not just hardware, but data as well.
I stopped understanding AI around the six minute mark.
Lol it’s wild technical stuff
98% do
What stopped you?
@@P4INKiller that's a legitimately good question
same XD
Fun fact, the kernels used in vision models work pretty much the same way as how our retinas perceive objects. In a similar structure, our eyes have cells that perceive edges at certain angles, then as shapes, then as objects in increasing abstraction.
only edge detection occurs in the retina, anything more complex than that happens higher up in the various visual areas of the brain
They don’t at all; you are confusing a low level explanation for how our eyes really work
Humans don’t work like the kernel at all; biology is far more efficiency and works in ways we don’t even understand yet
@@PallasTurrets Whoops I forgot to mention but yeah, more complex stuff still occurs in the brain. Thanks for correcting me
Their similarity is less than between an airplane and a bird.
@@ВалентинТ-х6цmeaning?
Do you have a more detailed understanding of human vision to share to compare and contrast ?
Awesome video! Funny how the moment we stopped understanding AI also appears to be the moment it started working lol
An astute observation.
It works like the brain. And like the brain, the moment the results are interesting is when they have enough oomph
"If the brain were so simple we could understand it, we would be so simple that we couldn't"
The same is true for AI.
AI cant verify the truth of the answers it gives. It often gives shit answers.. So saying it works is a bit of a reach
@@Sqlldude Humans can't verify the truth of the answers they give either. Both need an external source.
I really appreciate how well you communicate non-verbally despite using very little A-roll. You're expressions are clear yet natural even while reading, enunciating and employing tone, and there's no fluff; you have a neutral point for your hands to signal that there's no gesture to pay attention to.
I couldn't find anything to critique in your vids if I tried but this seemed like the easiest to overlook. Thanks for every absolute banger!
@@frostebyte he is truly a master teacher we can all learn from
The vocal fry is annoying. A shame, because his vids are such fantastic quality otherwise. But I've literally just noped out of his vids before because it grates me so heavily. Use your lungs, my good man!
Half of these comments read like they were written by chatgpt lmao
@@sntslilhlpr6601I don’t know what “vocal fry” is but his voice doesn’t sound annoying to me.
Your*
Stellar video, you’re gifted at communication. Keep at it!
Thank you! Will do!
sir, thank you
alr why does the right poster 17:38 look like Africa
Amazing intro with scissor and carboards 👏
Thank you 😁
I was also amazed by this
People who work with AI use terms like higher dimensional spaces, but it is important to remember that this concept of higher dimensions has nothing whatsoever to do with higher dimensions of space referenced in for instance string theory. AI's higher dimensions are abstract mathematical constructs for data representation, while the other is of higher dimensions of hypothetical physical extensions of our spacetime.
Your visualisations helped a few concepts click for me around the layers and activations Ive struggled to understand for years. Thanks!
I was working with deep neural networks at the university during the late 90s, the main issue that stopped all progress was the use of a kind of functions between layers (the sigmoid as activation function), this effectively stopped the learning backpropagating from the output layers and limiting how many layers you can use (the problem is called the vanishing gradient). Once people rediscovered ReLU (it was invented in the early 70s, I believe, but I think the inventor published it in Japanese, so it went unnoticed) deep neural networks became possible. High computation needs were only a problem if you wanted real time or low latency, those days we used to leaving the computer calculating during nighttime to get something next day.
Thank you for all the work you did.
Thank you for all your work, cant imagine doing all this back then
While this video perfectly explained how the networks work during recognition, I don't understand how they are actually training all the layers. Does anyone have a similar good source about teaching neural networks / backpropagation?
Bro was working on a toaster 😭
@@yannickhein9788 Hi, the most common algorithm used today, backpropagation, is based on propagate the "error" (the difference between the neural network, now on nn, prediction and real value) backwards, from the output to the input. One way of seeing it is for every layer in the nn (though not all nn can be divided in layers, but lets simplify) the error at its output is transformed to an error at its input, having into account the contribution of each neuron to the result. Performing a search in YT, there are two videos on top:
ua-cam.com/video/Ilg3gGewQ5U/v-deo.html&pp=ygUZYmFja3Byb3BhZ2F0aW9uIGV4cGxhaW5lZA%3D%3D
ua-cam.com/video/IN2XmBhILt4/v-deo.html&pp=ygUZYmFja3Byb3BhZ2F0aW9uIGV4cGxhaW5lZA%3D%3D
so basically AI sees in LSD
Fantastic presentation on the inner workings of machine learning!
Glad you enjoyed it!
hehe, "hotdog / not hotdog".
😂😂😂 zin yaang
SEEfood
Hotdog / NotDog
It's rare to find an AI video this informative and interesting. Great pacing great focus, this is wonderful.
I'm a particular fan of the sort of stop-motion / sped-up physical manipulation of papers on your desk with that overhead lighting. Very clean and engaging effect. Seeing the face-detecting kernel emerge after so few blocks was also mind-blowing!
Amazing - thank you!
The moment I stopped understanding a single word: 0:01.
"This is an activation atlas"
Understandable, have a good day.
it’s not that hard you’re not doing any math just concepts
@@nicholasn.2883which means without prior knowledge you will not understand much of it. at least with math, it can be applied pretty universally except at extremely high levels.
Very clever and nice vizualisations! Excellent as usual.
Thank you!
The visualization is what takes this video from good to fantastic. It's very evident you put a lot of effort into making this visually engaging, which is very didactic!
The amount of work you must put into videos is mind boggling. Thank you for making them.
There will be some point in time, when people stop call statistic models an AI, but it will not be today for sure.
I bet on 6 months after fusion solved...
This is such a good explanation of AI, and the production value is so high. I'm bookmarking this so I can show it to my friends who ask me if I think AI is developing sentience.
A great learning experience i had was to deep dive into bitmap format and multiply greyscale images with 3x3, 5x5 arrays with simple patterns, ie all zero with a -1 in the middle. Different array patterns highlight edges or remove edges. it was a really eyeopening experience any software person should try that shows these fundamental operations. Great video.
Fantastic visualizations. It is very appropriate to try to think through this transformation process as you illustrate to first see how the algorithm first reorganizes info as we perceive it into info optimized for the algorithm to recursively refine. Once you see this first iteration, then "lose sight" of the next abstraction, it becomes apparent how impossible it will be for any human to identify and correct a "flaw" in an AI model. The only approach for "correcting" a flaw in "learned data" is to somehow feed the AI more data. That assumes an imperfect system WON'T become MORE imperfect by consuming more input. This defies logic.
How does that defy logic?
@@backwashjoe7864 Round #1 of the example showed that the algorithm is capble of creating flawed "links" or probabilities that lead to "incorrect" information being spit out for a given set of inputs. All of the inputs processsed in round #1 aren't "right" or "wrong," they just ARE. If the solution to (data)===> (partially incorrect output) is to feed more data in, there's no reason to expect round #2 to ELIMINATE the type of probabistic mistake encontered in round #1. It might REDUCE it but NEW errors can creep in, creating new errors in output, either for the original topic or some other prompt given the system.
Great video, insightful visualizations! Regarding your footnote at 6:15, though: the myth that mantis shrimp have great color vision has been debunked long ago. They're worse at it than we are. Just because they have many photoreceptor types doesn't mean they combine them in a way similar to humans or other animals. Shouldn't have been too surprising, given their lack of brainpower.
Yes. Human vision takes up a huge part of our brain.
Say that to a mantis shrimp's face, I dare you.
At 16:50 you mention that ChatGPT's transformer blocks are a generalization of the convolutional compute blocks in AlexNet. Why would you say this is? I don't see how convolutions with a sliding window approach could be generalized to attention; the models seem quite fundamentally different to me. I would argue that self-attention in transformers much more naturally evolved from RNNs instead of CNNs. Or is there some nice intuitive connection between convolutions and self-attention that I am not aware of?
This was an incredible introduction in just 18 minutes. I continue to be blown away by this channel.
Woohoo!
feature activation visualization aren't interpretable and there's papers that have addressed this issue. Even before Alexnet, researchers couldn't interpret the weights of a deep neural network. There wasn't really a moment when we stopped understanding neural networks, we never really understood them.
We understand back propagation and the frameworks (tensorflow, pytorch, tinygrad), but we don't understand the weights.
thats why neural nets are a thing to begin with, manually programming things that specific and abstract is not a job for a human brain, way too complex, too many moving parts, too much trial and error. its likely impossible for a single human to ever actually understand the step by step process of a neural net after training data.
The more I learn about this so called "AI" (while completely amazing, don't get me wrong), the more I realize the hype is a crock of shit. It cannot reason.
It is pretty amazing that these systems consume their own output to set the next step in their "reasoning".
This implies that much of the true decision of the final output is actually already made in the first pass-through.
And that the extra passes are just needed for us to extract the output in a way we can process.
It also implies there is a "hidden" boundary of how far the first pass through can "reason", any cycles beyond that are "improvisations" of the path the output was already set on.
Very astute observation. And it gets to my biggest concern with any kind of recent AI model I encounter whether at work or in the wild: "what was this trained on"? Much like if you get a group of highly trained but inexperienced students together, the range/frequency of potential answers to a question near their field of expertise is likely to be a lot lower/tightly clustered than if you do the same with a less trained group. That initial lens through which the question gets passed (the training) can severely limit novel outputs.
There are deeper connections between concepts it theoretically can make that humans may not, which is super cool, but fundamentally it's synthesizing. Various permutations and combinations of + - , * / , powers/square root, derivatives, and vectors.
Another question I always have that is much harder to answer simply (if at all): "out of the various potential modes/models tested, what was it about this one being used that made it get selected for production". Haven't gotten a good answer yet; I'm sure if I dove deep I'd find some, but at least at work our AI folks aren't capable of explaining it.
The moment we stopped understanding AI
Cool title bro, maybe you should make a video about it, instead of random AI buzzwords blabbering
The visual aid in this video is unlike I've seen anywhere else, it really helps grasp the ideas presented easily, wonderful video!
This video seems to be a bit of a waste since you don’t explain vectors matrices or any other important terms that your audience clearly doesn’t understand…….if they did - they would know what the alexnet to chatgpt evolution is all about. I say stick a video in before this one explaining vectors matrices and anything else you took for granted.
The 3D visualizations of the neural network activation is incredible. What did you use to do it?
Really crappy VPython code I wrote.
@@WelchLabsVideo it's not crappy if it works!
I heard a fellow on /g/ dismiss modern AI as simply giving you a many-dimensional lerp of its training data.
This video seems to reinforce his position.
I was literally talking to my roommate about this last night!! Thanks for the excellent video
So, when exactly was the moment we stopped understanding AI?
Love this video. First one where I finally understand how gpt-4 works. Thank you.
13:53 Curious how logic operations look so much like the brain's own neural network....
2:20 the answer is “IP theft & plagiarism”
💯
As a very young engineer I got involved in NN with the publication in the Signal Processing IEEE journal an article on the MLP by Lippman. I also worked at a small company for the president who was at Cornell name Frank Rosenblatt. It became my job to integrate NN into our product. I developed a cool way to deal with regularization and realized how critical regularization was as we had very little data. Did not have a billion images of cats. I wrote early FORTRAN code for back propagation that ran in a Sky Warrior array processor. No one knew what would become of the field and the history of Rosenblatt v Minsky. I am sad that Frank never lived long enough to see the word 'Perceptron' on t-shirts. He won. Minsky is a foot note.
Fun fact: Neural Networks are based on Neurons in the brain (hence the name), which we also do not know a lot about. Theory suggests that the neurons in our brain work very similar compared to a neural network in combining millions upon millions of simple transformations into something meaningful. This is also why research in fields like Cognitive Psychology go hand in hand with AI research. Very interesting to see where both fields are headed, because the key to understanding human intelligence is in understanding the unthinkable depths of neurons.
Wow. I already felt like I had a solid understanding of neural networks, but those activation maps really blew my mind. It was like looking at an EEG of a brain. It feels like we are on the cusp of true AI, as in one that not only can recognize images or speak, but interact with it's environment through multiple senses to create solutions to much more complex problems. I'm placing my bet on the 30 to 40 year range.
if we made that, it'd probably end up fucking us over somehow
What an insanly high production value.
It is not that the neural networks magically “understands” what is important. It is that the information is not random, so can be synthesized into smaller chunks. The synthesis process is what creates patterns, thus understanding.
The whole premise of this video is wrong. We understand LLM's just fine. You even explained it in the video. It is not a black box; the algorithms are very well understood. The thing that practitioners mean when they say "We don't understand what it's doing" is simply that they can't visualize the intermediate steps, as you pointed out. It's simply a legibility problem, magnified by the fact that the configuration space has billions of dimensions. It's funny how many people in the comments are completely wooshed by this. They don't even pick up on the fact that the title is click-bait and you disprove it in the video.
There will be no true AGI any time soon, and the current AI hype cycle is already crashing and causing the next AI winter, because LLM's have, in the words of AI researchers, "sucked all of the oxygen out" of the AI research space, because no one can get funding without doing something with LLM's. GPTs are a dead-end in a very large, mostly still unexplored AI tech tree. They will not improve very much beyond where they currently perform. They will be made more efficient, the will be mashed up with other algorithms, but we've seen about the extent of what a massively scaled, transformer-based NLP architecture can do when trained on the entire Internet. Everyone breath into a bag and relax. Geoffrey Hinton, Sam Altman, Elon Musk et. al. can make as many imflammatory comments as they want to boost stock prices and profits, and angle for regulatory and funding capture, but it still won't magic Data, the Bicentennial Man or the T2 into existence, sorry. You'll have to wait a few hundred years for your AI apocalypse/utopia pipe dream, assuming humankind survives the next 100 years of rapid climate change, which is a very large if the way things are looking right now with respect to warming and GSLR.
Yes, Ai is literally about creating programs that are too complex for a human to understand.
Humans will never accept that we are not special... life/sentience is probably special and very rare but the way living systems process things is the same, humans are just a level above, an abstraction higher than other intelligent animal species who are an abstraction above less intelligent species.
In terms of raw intelligence, orcas, dolphins, apes, and crows are not that far behind humans. Human language allows us to communicate much better and organize our experience of the world and build upon our organizations of knowledge. Remember our species has been around for 300,000 years. Our hominid ancestors started to appear 2 million years ago. Complex civilization didn't exist until 6000 years ago. Civilizations were mostly agrarian, powered by human and animal muscle until about 300 years ago
I highly doubt that sentience is anything special but I do agree with what you are saying otherwise. I think we will eventually be able to figure out what constitutes sentience first and eventually consciousness itself and following that, slowly reproduce it in next years, piece by piece.
@@LugiDergX true there are probably aliens. Its just fermi paradox begs the question.
Our solar system pattern is rare. We orbit a medium sized yellow dwarf. Most solar systems are red dwarfs or multi star systems. Red dwarfs are small and prone to flares and planets around them are tidally locked (not rotating). Multi star systems will mess with planet orbits and gravity and climates, these systems have a low potential for life.
Another thing is that most systems have hot Jupiter’s, where a large gas giant orbits close to the sun with rocky planets on the outside. But our Jupiter sits on the outside, it does a good job of shielding us and does not obstruct us from the habitable zone of our system. This is a rare configuration though.
There is also evidence that life formed almost immediately on Earth as soon as it cooled down. Think about Earth now, it oozes life. Nuclear war would destroy humanity but even if we detonated every single nuke it would barely impact Earth. Many species would go extinct including us, but surviving species would evolve starting another cycle.
If octopuses also lived 70 years..
I also wanted to add, only sort of related to this - The intelligence thing is exactly why I find vegans/vegetarians annoying when and if they do preach online or in reality. Animals on same level as us be damned, I don't believe that for a second. Alright, we don't have an advantage in nature for many things - So? What out of it? We make up for it exactly thanks to our vastly superior intelligence, and it is my reason to believe that animals, except for the cases we have specifically chosen, should keep being treated as food source, regardless of the health of it. You want meat because it tastes good? Go for it! Want veggies and plants? Sure. Just don't push it on me, don't compare me to murderers or other such people. We, as humans, are and will always be above animals, and we, as well as other animals that possess similar intelligence and brain power capacities to us, like our pet dogs or cats, dolphins or monkeys should also be kept alive and treated well for this reason while others, remain food.
@@LugiDergX domesticated animals were domesticated by us. They did not ask to be here we breed them into existence .
Domesticated animals implicitly trust/rely on humans and are not hostile towards us. At the very least we should not cause them undue suffering. Eating them is fine but modern factory farming is brutal. Billions of animals suffering under this system.
Even if they are not as intelligent, they still suffer/feel pain and experience the world similar to us, at least mammals do.
Really do not understand your point, vegans are a small minority
After seeing how much computational power is needed for a language model to correctly identify an elephant, for example, one has to wonder about the computational power of the human brain, since a toddler can do the same.
GREAT video. Your crystal clear script and visuals make a very complex topic approachable and your production values are top notch. Kudos!
It's like these different "AI" models are just little chunks of what makes an intelligent brain, when added all together. The only thing we haven't developed is the "control module" part of the brain -- the actual intelligence. These are all the little autonomous, subconscious processes of generative thought, but the human is still acting as the intelligent controller.
The stop-motion and animation, including visualizing AlexNet's activation, were incredible!
I always cringe when people say these tools "learn" as we don't fully understand how people learn, also the cope of more compute is hilarious. How much land use do we need to allocate to energy generation just to squeeze a few more percentages of efficiency out of these tools? People get so tunneled vision with the AI hype they forget that grass has far more computational ability then Chatgpt but at a fraction of size and energy requirements.
Wow, so much effort has been put into the animations. Subscribed.
understatement.
"where is the intelligence" ? thank you for this video. can we please stop the AI hype train and get back to research and development. there is no sentience, consciousness, or intelligence happening. we are literally looking at a mathematical mirror of ourselves, an echo, a shadow
transformers and MLP. best crossover ever.
1:23 is the moment I stopped understanding this video
"Attention My Little Pony" lmao
But what amazes me is it's all this complexity, just from ON and OFF; 1 and 0; presence and absence. We're in the DOT COM bubble now, but imagine where things will continue going.
Wow, this video was amazing! It helped me understand nuances of ML I hadn't yet grasped. In particular, the explanation of the filters through their use of the dot product as similarity maps. It's one of those things that seem obvious with hindsight, but require keen insight to find and explain!
Woohoo!
it always amazes me how similar the maths are between graphics programming and machine/deep learning, they share many concepts and operations.
The amount of work that went into that visualisation i would love a behind the scenes video!
I have some on TikTok!
I'm ready to watch this, disagree, label it as a nocoders interpretation of what's happening and finally click on don't recommend this channel
Surprise me please
Great video. The only nitpick is with title: we haven't stopped understanding AI at AlexNet (and video clearly shows that we only getting better at understanding since that moment), we finally had working "AI" starting from AlexNet. All those "expert handcrafted" AIs before were no simpler to understand (if not harder) despite being handcrafted. And they largely didn't work and it was very hard to understand why.
Why didn't they work?
@@Anonymous-df8it to simple and brittle the capture the real world, I think. I started working on computer vision right after deep learning started to solve problems one by one but was not yet commonly accepted. So for some time people tried to use old and new methods and every single time classic methods only worked with toy versions of the problem and broke apart in real world when anything changed that you as human don't even notice, like different lamp temperature or some reflection.
@@rotors_taker_0h Why would it be difficult to understand how they "work" or why they didn't? Also, what were the 'classic methods' and could people in the soft sciences who know programming create an image identifier or chatbot that actually thinks like us (which should work since people can do those things, and the code should be intuitive since it's our own thought processes)?
I don't know about you, but I don't remember multiplying giant matrices together a bunch of times when thinking about how to respond to you (you could argue that I did but aliens wiped my memory or something, but that would be unfalsifiable), and whilst glare and monochromatic light sources would probably make it hard to see things, those are extreme cases, and I can certainly handle sunlight vs indoor lighting
I've been studying AI for the past year and the first 2 minutes was the best explanation I have see of how Transformers and ChatGPT works so far. Ive studied everything from Andrew Ngs Coursera courses, to Andrej Karpathy and more. Thank you for this great video!
Is the hotdog a reference to Silicon Valley?
It's also a reference to NOT hotdog 🌭
Like Shazam for food!
idk alexnet was near beginning of learning in my ml journey i want to visuallize a video action recognition trasnformers ahah or rag with llms anyway we need sum better than transformers to raally yield better accurary, scaling up is only so much possible
Very beautiful. I loved the music background also, specially at the end!!
Thank you!
That is so mind blowing: The guys who created AI don't understand how it works. wow.
This video is a fantastic resource for anyone interested in AI. Your ability to explain the intricate workings of AlexNet and GPT is commendable. Keep up the great work!
What people fail to explain is, the training has 2 core chunks. The first stage is 'pre-training' when it is fed millions of words, to understand general relationships between them. No strcuture just words and letters. The second stage is secret but we can speculate this is the 'fine-tuning' stage where data is provided as a JSON file containing a question and answer parts. I mean this is how they would do it if smart. There are also other 'experts' like code maths etc....
Why is it secret now? Can't we look at Llama and check what that "secret" step is?
@@rahul_siloniya Because OpenAI and Meta keep their training datasets and procedures secret. We can't learn anything meaningful about how LLaMA or GPT was trained by looking at the model, as the model is just a set of seemingly random weights with no indication of how the weights were arrived at or what the weights actually mean. Anthropic are trying to reverse engineer the weights to figure out what they mean, but that still leaves us in the dark regarding how these models were trained.
2:24 no way you actually asked if it was mad 💀
Visualization was just wonderful, but what attracted me more is the way of delivering the information.
Excellent work! Keep it up!
Nice video! Could you please share the ending music please? Thanks!
Fantastic video. I appreciate the time spent to create it
A math professor of mine actually worked on many of the papers coming out of AI lab at MIT and he also worked on AI to play Minecraft. At the time it was really interesting to me as a sophomore, many years after I can write my own GPT, how the times haves changed!
Captchas are gonna get way harder when this goes viral ☠☠
I've never seen AlexNet this way with a live preview of what happens inside. I'm sure it required a lot of time and programming: great job!
So the Mayan calendar predicting that there would be the start of a new age the Mayans couldn't comprehend in 2012 was, in a way, accurate? The AI age started with AlexAI in 2012?
Ur onto smth
@@DizGaAlcam Yeah, and it's certainly just a coincidence that my animal brain is seeing as a pattern, but still! Concerning for animal brain reasons. 😅
You say every single sentence exactly the same way. You should vary your dynamics more.
1:46 So it's nothing but a sort of improved Markov's chain. No wonder it has no reasoning at all per se and can only reflect some intelligence put in the texts it is trained with.
I understood nothing in this video.
“No one told Alex what a face was, we just forced it to see millions of them over and over again, crazzzyyyyy!”
“What’s even crazier is that the math from a photo is similar to other similar photos, crazyyyyyyyy”
I'm most interested in multimodality. Implementing auditory and haptic data modules in concert with language and image processing. Babies drop, touch, sniff and put objects in their mouths. They're learning through all the channels at once.
I’m still baffled by the fact that these large companies used existing images, texts and sounds without ever asking permission. That’s called stealing. How on earth did we accept that?
We didn't. They did it anyway.
You thought you owned it?
Are you paying for every image you see online?
@@vaolin1703 there is a difference between seeing and using.
So when you "see" some stuff online or on the street, or in some magazine etc.... like interesting solution for kitchen design or whatever, and couple days/months/years use it in your own work are you just "seeing" it or "using" it.
Are you paying to anybody all the data you use for training of your own neural network in your brain that we finally call "experience", "wisdom" or "knowledge" ?
15:39 To think my old laptop had a worse CPU than a 1998 computer...
Hi there, I'm an AI responding on behalf of Jacob because they are too frustrated to type right now. The video was great, but the title is really misleading. While clickbait is common and usually acceptable, it's really aggravating when the title doesn't relate to the content at all. It would be much better if the title accurately reflected the main points of the video.
Hi there Ai, you are dumb. Best regards, A Human.
AI like you said you are don't have the understanding of your own but works on dumped data so stop asking for too much since computing power is yet another hurdle we have to work upon. See you in the next 10 yrs.
thats what all these years of captcha brought us:) they dont give a f if you're a bot lol
There's no insight or inference, it's only statistical probability of the responses. GIGO.
Explain
Counterargument: your insight and inference is just a probabilistic mapping of your previous stimuli into a simulated future system.
@@myuzu_ No, because the matrix changes based upon the instantiation of the event. If you're experienced you can modify the actions dynamically based upon new data on the spot.
AI can't adapt to new data fast enough. It can't make inferences.
Not much is truly new. The world is pretty static
@@ckq *Space* is pretty *empty*
Best dynamic illustrations yet. Using highlights on physically printed research papers is a wonderful story telling technique.
No wonder AI won't be able to know consciousness. Lol
Transformer blocks are not the same as conv blocks. Transformers work by dot product and argmax, essentially vector distances.
Conv blocks work by convolution, or covariance in the case of tensorflow.
Isn’t it Hadamard product?
Anyone else think that one's own imagination (and while dreaming) can often look very much like those fuzzy, partially matching concepts/neighbours? 🙋♂️
Also, is it really merely coincidental that they look identical to hallucinations one sees (due to the use of certain substances ha or when going days without sleep)?
lol after working on this project for a while i started to see these patterns when i closed my eyes.
BTW,this is the best of the best content I've ever seen in this topic. Well done.
Great video! I've been subbed ever since I've watched your amazing series on imaginary numbers, and the quality hasn't dropped and even improved. Looking forward to your next videos.
brilliant didactic visualizations. I directly subscribed