Self Driving Cars [S1E2: ALVINN]
Вставка
- Опубліковано 11 чер 2024
- PATREON: / welchlabs
TWITTER: @welchlabs
MORE: www.welchlabs.com
CODE: github.com/stephencwelch/self...
Welch Labs on backpropogation
• Neural Networks Demyst...
Great 3B1B Video on backpropogation
• Backpropagation calcul...
FURTHER READING
Backpropagation: www.nature.com/articles/323533a0
ALVINN: www.cs.cmu.edu/afs/cs/Web/Peop...
ALVINN: pdfs.semanticscholar.org/df76...
ALVINN: papers.nips.cc/paper/95-alvinn...
VIDEO REFERENCES
Nvidia End-to-End: • NVIDIA Autonomous Car
Amnon Shashua: • Autonomous Car - what ...
Chris Urmso: • Chris Urmson: Perspect...
Navlab: • cmu Navlab, self drivi...
More Navlab: • NavLab 1 (1986) : Carn...
More more Navlab: • cmu Navlab, self drivi...
More more more navlab: • Navlab 97: Automated a...
You guess it, more navlab: • NavLab 1984 - 1994
Navlab on the news: • Navlab on KDKA
MUSIC
www.premiumbeat.com/royalty-f...
www.premiumbeat.com/royalty-f...
www.premiumbeat.com/royalty-f...
www.premiumbeat.com/royalty-f...
SPECIAL THANKS TO
Tony Fast
Krish Ravindranath
Karthik Naga
Dean Pomerleau
- And -
Sid Sarasvati
Ross Hanson
Yana Chernobilsky
Vin Soma
Antoine Pintout
Jaewon Jung
Raphael J Vasquez
AJ Englehardt
Nate Fuller - Наука та технологія
![Self-Driving Cars [S1E3: AI Failure?]](http://i.ytimg.com/vi/yaYER2M8dcs/mqdefault.jpg)
![Self-Driving Cars [S1E3: AI Failure?]](/img/tr.png)
![Self Driving Cars [S1E1: The ALV]](http://i.ytimg.com/vi/cExJbbwOfcw/mqdefault.jpg)
![Self-Driving Cars [S1E4: RALPH]](http://i.ytimg.com/vi/DIAKL0s4Mzs/mqdefault.jpg)
Stephen - I can't thank you enough for so clearly describing and demonstrating ALVINN. You've articulated the ideas behind it much better than I ever did! It is great to see that the early history of self-driving cars based on artificial neural networks captured for posterity.
Sir you are a genius
What a real life badasssss you are sir....today i am enjoying your brain child.....thanks a lot
Most underrated UA-cam channel, some of the highest quality content out there. Thanks a lot!
Definitely true.
Malheureusement les vidéos qui servent a rien attirent les bon à rien... Et ils sont nombreux on dirait !
Thanks for watching!
This is true, but only if you are interested in technical stuff! lol Even if u are interested, it may be hard to learn in just 10 or so minutes. Even I have to "go at a snail's pace" while watching his videos.
Yeah this guy's video about imaginary number i was a gold :)
I don't think I've ever felt more eager for a next episode on UA-cam than with this series.
6:41 I like how you can actually see the overfitting in the four weight pictures. The pixels above the skyline are not relevant to the actual driving, yet they have very strong weights, even stronger than the road pixels. The top pixels tell the AI where it is on the track by looking at the buildings and trees and decides to steer mostly based on that rather than the track.
I know that this problem would be fixed by a lot more training data, I just like how you can see it so clearly. And it certainly still shows that it has some understanding of the road, so it's good enough as a proof of concept.
I noticed that too, good eye!
Would it be "fixed" with more data though?
I feel like that overfitting is an artefact of the naive downscaling solution used. When a human looks at the view from their car, they don't downsample it to low resolution in their head then pay equal attention to every pixel. We somehow decide what's important and not, and we keep an eye on that.
There should be a neural network that downsamples the images. Think like a visual cortex providing information to a decision making centre.
@@Asdayasman Actually I think that the downscaling is more to mitigate computational costs. More data, but from different tracks, would fix it since for training they drove laps on just one track (and with one lane).
Thus the ANN learned to mark objects above the road and base its steering angle output based on that (which for that 1 track and that 1 lane resulted in small test error). Getting more tracks for your data would probably lower the weights of those pixels in the weight matrices.
Daniel Saranovic I feel like you didn't understand my point.
Downscaling and throwing more data at the problem is the AI equivalent of brute force. It can work, but is dumb. What you need to do is to describe the images as features, like we see them. Tree here, road here, lamppost here, etc. The way to do that would be a visual cortex, be that a classic algorithm, or another network.
Yes - it is easy for the NN to key off of features in the environment that just coincidently happen to correlate with the correct steering direction. That's why in the original ALVINN implementation I only sampled pixels from the ground plane directly ahead of the vehicle, avoiding everything above the horizon line or too far to the left or right of where the road is expected to be in the image.
I’ve always love Welch lab videos. My favorite is the complex number series. This series in my opinion is very well produced and definitely deserve more attention. You pulled a smart one by leaving us with more questions than answers. Hopefully all will be answered with the end of the series😁. Love your work man. Keep it up. 👍
BLACKBURNER same.
One of his best videos ! Great inspiration!!!!!!😀😀😀😀😀
Glad to see him back at it again!
Ok when this series is done I'm going to convert an RC car with a raspberry pi and a camera and program it to drive on its own around Campus.
I didn't start studying mechatronics engineering for nothing!
What about a self steering bicycle haven't seen anyone doing that before.
@@VR_Wizard The balance would be tricky... hm... did you mean while someone's on it?
@@VulpeculaJoy Yes. The bike could do micro adjustments to circumvent potholes or get the angle right when I cross rail tracks or if someone opens the door in front of me it could start to break automatically. The new E Bikes come with electric anti-lock braking system (ABS). Or look behind me when a car is to close it could make adjustments.
@@VR_Wizard Well the thing is, balancing will be pretty much impossible without making you look like a circus actor with weights dangling all over the place and since sudden steering inputs that you don't plan yourself would make you loose all stability too, I'd assume, there's only an automatic emergency breaking system left to be of actual use.
But on a bike you are a defensive driver anyway and have a much better FOV as well as sensual immersion in your movement so I doubt that it would make a big difference in rider safety.
For motor bikes however the physics change and you can save the rider from an impending collision at high speeds by a sudden but very precise yank in the opposite direction inexperienced riders would intuitively think. There I see much more potential but also at a development cost, risk and overall span of complexity that sadly greatly exceeds my capacity.
It's an interesting topic for a bachelor or master thesis paper however.
Why use a rpi when there are better SBCs from nvidea,odroid,atomicpi,rockpi4,etc
You’re killing me with these cliff hangers man, great vid!
These videos have the wrong order of magnitude of views, Dean Pomerleau himself is here watching!
Incredible work!
You're a master at explaining complicated topics. Kudos and keep it up!
No better videos! Please keep up the amazing work!!
Well-presented and accurate-really nice work! Looking forward to the next episodes.
I love the style of your videos, keep the good work up!
Your videos are awesome, this quality damnnn, someday you will grow big
I don't remember subscribing to this channel.
But I sure am glad I did! Great video!
probably from the imaginary numbers not being imaginary video
i was waiting for this episode ... keep going
Just backed you on patreon, great channel
Hey you're back. Hooray!! :)
The biggest problem with supervised learning for self-driving is the handling of exceptional situations. What was not part of the training data, is not part of the learned behaviour and may get mishandled. Think: New signage, animals or humans suddenly crossing the road, etc.
A combination with Reinforcement Learning is much more promising.
The major issue with ALVINN is that it can not properly handle the vehicle´s body motion, like pitching and rolling. These motions (among another plethora of issues with learned approaches) basically "smear" your input features in the weight map that you have shown. And once these "smearings" get too large the system can not derive useful steering angles any more. If you want to have a system that can handle this better you need something to estimate at least the vehicle pitch angle. This could be solved with either structure from motion or if you apply some modeling about the shape and appearance of the lane markings. If you see two lane markers on the left and the right side you can actually make a pretty goo estimate of the vehicle pitch, if you assume that the left and right marker are parallel. So this basic problem needs to be solved.... And we even did not start to talk about "mean" issues from the chaotic and "hostile" environment that the car is driving in ;-)
This is definately one of the most underrated channel. Atleast it should be slightly more popular than siraj channel considering the rigor.
This is so fascinating!
Thank you!
Damn ! Your videos are lit.
oh man your quality remains the best
Please continue!
Excellent video.
I love this series. When is the next one coming out?
This series is sick
Thanks for that, help a lot!
very interesting as always :)
thank you very much! Please, keep going :)
How is this at only 28k views. It's the best video about self driving cars I've come across by far. Also if you're reading this go check out his series on imaginary numbers. It was epic.
Thank you!!!
Thank you 😀
you are a great man
Hola puede hacer este video sobre alvinn subtitulado o doblado en español por favor es muy interesante lo de multiplicación de matrices y retropacion y sobre peso de la red neural
AMAZING!
Thanks for watching!
Darn these cliff hangers!!! 😋😂😂
Still waiting for part 3. :D
You are so great..... :-)
Will there be other videos in this series?
Here from Veritasium's video
Will you make "learning to hear" someday?
i love f(image) = theta :D
f(🖼️)=θ
Interesting to see it in 2024 with Tesla successfully demonstrated end-to-end approach and we expect robots to drive us around in 2 years.
Michael Talbott "Holographic Universe" argument is that all matter and brains Karl Pribram - holobrains works on continuous Fourier series (integral). Electron, atom, proton, molecule is just Fourier waves combination of vibrating strings, Schrodinger wave equation have not direct conformation of such assumption (duality Schrodinger wave equation versus solid particle Heisenberg matrix). Electronics camera works based on discrete scan mode and based on binary code thus not analogical number system analogical means continuous stream of information. Create android flesh or organic based robots no plans or project done. Computer can not produce continuous waves, even continuous celluloid tape roll analogical camera works based on frames per second on discrete numbers mode. Sorry, AI will be based on linear algebra (Heisenberg solid particle matrix) principle, of brutal force computation.
Only 52k views, tragic 😢 😔 😪
No more new videos? :(
What's the problem? Full speed ahead!
Yeah, I'm sure a self-driving tank would do just fine. Turn towards destination, drive straight.
MY_GUESS: End-to-end (was that the term?) is too scary trust lives to. If your logic is an arrow to a big black labeled "magic happens" you don't have much trust in your system, no matter how well trials behaved. You need a feedback generating system with discrete functional blocks so each aspect can be tested and trained independently. When a failure occurs (and it will) you want to be able to point at problem section and be able to fix/improve that subsystem. AI can certainly be involved in these subsystems, but there should also be alternative solutions running in parallel for higher trust in the system. We really want AI that can tell us what it is doing :) (eventually).
two problems
1st problem is that the algorithm receives a lot more data than it needs (clouds, trees, bystanders etc), and blindly includes them in the decision it makes. cutting a tree down, changing a traffic sign, initiating a road renovation or similar change would completely derail the ai.
2nd problem is that a neural network learns by example, and only work if it was properly taught. it will not cope well with situations never introduced to, which includes for example the 'i need to avoid this frontal collosion, shall i ram into the rock or the water instead' type of situation, of which i doubt there will be a lot of training examples 😃
@@speedseeder That is a problem with the black-box approach (feed it everything and expect useful driving instructions). AI can still be used in the components of a robust system for doing some of those tasks you mentioned. A classifier can be trained to detect clouds and trees. That information can then either be used or discarded as needed for the current task. A hybrid system will likely be more stable than either pure algorithm or pure AI/ML.
If the goal is driverless cars I think it would be easy to use sensors on the roads instead of this whole hassle. If the goal is something else then I am missing the target big time.
where is he?
This is the first time I've seen an example where the weights are actually meaningful to the human eye. So often when introducing neuro-nets, people will talk about what the weights could mean theoretically. Then when you actually see an example, the weights look completely meaningless to humans.
This is because the weights are directly mapped to pixels - the "neuron to neuron" weights will not look like something useful anymore. And also the feature we are looking for is a white marker - that means its something with a high grey value in the input image - and the weight map is meant to "boost" these features. The Problem is, that the system did not learn anything about the abstract concept of a lane marker nor of the "rules" of how they are shaped, just about its appearance in the training data set. And this is also one of the biggest caveats of all machine learning systems. They DONT actually understand anything from the content they are identifying. They produce a relation between a given input (i.e. a Image) and a related output (here a steering angle).
so this video is only a teaser for an other one? arrrrrr
as all good series
Why are all the math youtubers so hot :0
... what about mathologer? :P
These videos are super interesting, but the background music is a bit loud and distracting at times,
Keep up the good work!
10
veritasium sent me
Welch Labs best husbando
Couldn't a neural net learn the steering angle without training data by driving around in games?
Im guessing not back in the 80s, cause they didnt have anything as realistic as modern games to train on. But today, that can absolutely be done through something called reinforcement, or deep-Q, learning. Im fairly sure ive seen someone testing that idea out with GTA bout a year ago, you can probably find it On youtube.
Obviously Big data wasn't big enough
neural networks are great for automating repetitive jobs, driving (and adopting to changig driving conditions) isn't one of these jobs.
neural networks could be used to solve certain aspects of driving, but can not be used as an end to end solution
It would be so much easier if all road size and color are standardized smh
SO they are trained by humans and you are expecting them to do better.