DeepMind's AlphaFold 2 Explained! AI Breakthrough in Protein Folding! What we know (& what we don't)
Вставка
- Опубліковано 16 чер 2024
- #deepmind #biology #ai
This is Biology's AlexNet moment! DeepMind solves a 50-year old problem in Protein Folding Prediction. AlphaFold 2 improves over DeepMind's 2018 AlphaFold system with a new architecture and massively outperforms all competition. In this Video, we take a look at how AlphaFold 1 works and what we can gather about AlphaFold 2 from the little information that's out there.
OUTLINE:
0:00 - Intro & Overview
3:10 - Proteins & Protein Folding
14:20 - AlphaFold 1 Overview
18:20 - Optimizing a differentiable geometric model at inference
25:40 - Learning the Spatial Graph Distance Matrix
31:20 - Multiple Sequence Alignment of Evolutionarily Similar Sequences
39:40 - Distance Matrix Output Results
43:45 - Guessing AlphaFold 2 (it's Transformers)
53:30 - Conclusion & Comments
AlphaFold 2 Blog: deepmind.com/blog/article/alp...
AlphaFold 1 Blog: deepmind.com/blog/article/Alp...
AlphaFold 1 Paper: www.nature.com/articles/s4158...
MSA Reference: arxiv.org/abs/1211.1281
CASP14 Challenge: predictioncenter.org/casp14/i...
CASP14 Result Bar Chart: www.predictioncenter.org/casp...
Paper Title: High Accuracy Protein Structure Prediction Using Deep Learning
Abstract:
Proteins are essential to life, supporting practically all its functions. They are large complex molecules, made up of chains of amino acids, and what a protein does largely depends on its unique 3D structure. Figuring out what shapes proteins fold into is known as the “protein folding problem”, and has stood as a grand challenge in biology for the past 50 years. In a major scientific advance, the latest version of our AI system AlphaFold has been recognised as a solution to this grand challenge by the organisers of the biennial Critical Assessment of protein Structure Prediction (CASP). This breakthrough demonstrates the impact AI can have on scientific discovery and its potential to dramatically accelerate progress in some of the most fundamental fields that explain and shape our world.
Authors: John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Kathryn Tunyasuvunakool, Olaf Ronneberger, Russ Bates, Augustin Žídek, Alex Bridgland, Clemens Meyer, Simon A A Kohl, Anna Potapenko, Andrew J Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Martin Steinegger, Michalina Pacholska, David Silver, Oriol Vinyals, Andrew W Senior, Koray Kavukcuoglu, Pushmeet Kohli, Demis Hassabis.
Links:
UA-cam: / yannickilcher
Twitter: / ykilcher
Discord: / discord
BitChute: www.bitchute.com/channel/yann...
Minds: www.minds.com/ykilcher
Parler: parler.com/profile/YannicKilcher
LinkedIn: / yannic-kilcher-488534136
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: www.subscribestar.com/yannick...
Patreon: / yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n - Наука та технологія
Yannic Kilcher is all you need
lol
The unreasonable efficiency of Yannic Kilcher
Learning to summarize from Yannic Kilcher
Self-training with Noisy Yannic Kilcher
Machine Learning Research Paper Summarization Models are Yannic Kilchers!
I've done my bachelor thesis in structural proteomics and your introduction was very very good!
Looking forward to the explanation on the paper on AlphaFold2!
Great video, but one small mistake at around 19:55 : You do not have two torsion angles because you are in 3D. The thing is, that atoms can rotate around a single covalent bond. The amino acid backbones, however, are connected in such a way, that a double bond is created (to be true, it is just a partial double bond, but this it not further important to get the point). Double bonds create an atom plane, in which the atoms are fixed and can no longer rotate individually. This is the case for connected amino acids. In detail, the center carbon atom, called C_alpha, is connected to two planes, one is the connection to the prior amino acid and the second the connection to the next one. The rotation angles of these plane in relation to the C_alpha are then the torsion angles. These are a direct result from the underlying chemistry and are used to describe structures since the birth of structural biology. If there wasn't a partial double bond, we would have a huge problem, since each protein would have even more angles we would have to consider (the side-chain angles are freely rotatable in most amino acids and lead to many many more possible combinations, but the backbone is more important and thankfully there are just two angles).
Thank you, finally, someone providing a longer more detailed presentation about AlphaFold.
Just found your channel and will definitely check out more content. Great job!
Friendship ended with CNN
Now, Transformer is my best friend
Is it because of the butt-wiping feature?
Yeah, well transformers are not the universal answer, CNNs won't go anywhere soon.
Hello comrade
good reference
My friendship has ended with every media outlet like FOX and CNN, they all give us the illusion of a competition.
You're 1) smart and 2) a great teacher. Reeeally good video. Super entertaining and rich of information.
Thanks for your effort in shedding light on this development
Yannic:"If youre watching this youre a machine learning person,and dont know about proteins"
Me:"Actually...........quite the opposite"
Same 😂😂
You are the best explainer I have encountered - and the population in question is large.
Wow Yannic you have such amazing teaching skills.
Nicely explained (even biochemistry)!! Loved it when you compared DNA to source code,protein as binary and the whole process to compilation.
I am a biologist interested in machine learning and AI and it is great to see this explanation.
BTW, the amino acids with the rings exist . But especially proline's ring with amine group inside the ring makes protein folding even more complicated.
There are a few amino acids with rings (they are called aromatic): Phenylalanine, Tryptophan, Tyrosine, (and histadine). Proline also connected strucure :) I love these videos man!
Note that at ~10:00 we get the impression that 'shape is all you need', but while some alternative AAs that replace common ones in given positions of a particular protein can retain the energetically favored structure, the functionality might be altered drastically: shape != functionality - you can have two different proteins with the same shape but only one is functional in the examined spatiotemporal sphere, or functions as expected.
Further, the AA composition of the primary chain and its multitude of intrinsic properties are not the sole determinants of the final 3d structure as this depends largely on the environment (acidity, temp. etc.). Finally - different shapes (different proteins) can have similar functions.
Overall top effort and overview of AlphaFold, thanks Yannic!
And there are quantum mechanics effects. Even a slight change in the atomic structure - like deuterium instead of hydrogen - can alter the total energy of an electron that tries to tunnel through the molecule. And the whole protein "machine" falls apart.
But I guess shape is still VERY important. So good job Deepmind, CASP and of course Yannic! :)
I study biochemistry and the explanation about folding proteins is actually really good!
Beau-ti-ful explanation. One of the best explanations of any subject I've ever seen.
Brilliant turn from a complex matter to a understandable subject.
Ge-ni-us.
Congrats and, above all, Thank You Very, Very Much. Beau-ti-ful.
Lightspeed Kilcher strikes again. He's faster than Usain Bolt. ✨
first 🤣
This is super cool, glad that I found you!
@17:20 the second line has "NERDS" in it.
That's it. That's my contribution to this discussion.
The 2 step process reminds me of the symbolic regression on physical systems paper. Use deep learning to generate some intermediate representation, and then use that representation as the model to approximate for a different algorithm that has nicer properties.
thank you for explaining the research papers the way do .
I find very hard to understand by reading them; I am 3rd year Bsc student for Computer Science.
I love the fields and the papers you talk about so it definitely feels great understanding a bit more about the papers you explain!
33:54 - finally we're on the same playing field! )
Thanks for the break down, I hope i'll get to your other videos soon as well
Very nice summary
Good visual explanations.
Enjoyed the alligator.
Thanks.
DNA is your compressed source code.
RNA is your decompressed source code.
Proteins are your binaries.
Each 3 digits of base-4 dna or rna code represents one amino acid, with some seeming redundancy.
We are slowly learning the exact compiler code. It was not that long ago when we found the code for "start" and "end".
This folding puzzle is one of the last big steps before we can program life like we program computers.
Good overview, well presented, thank you!
Hi! Thank you for such a great explanation. This has been so helpful. Would love an update with the published Alphafold2 paper!
My comments as a computational biophysicist student:
1. What about protein which need extra proteins which need helping such as chaperone? Anfinsen's dogma (i.e. AA sequence encodes 3D) does not apply quite well here.
2. Nature paper on this AlphaFold mentioned that complex structure (probably meaning such as homo/hetero-n-meric proteins) is yet to be predicted with high accuracy due to intermolecular interactions distorting the structures.
3. Most importantly at least to me...what about correct folding PATHWAY? Deeplearning, MC-based, homology modeling whatsoever is all about the END structure. Molecular dynamics can perhaps (depending on force fields) predict folding pathways (called reaction coordinate or collective variable).
Thank you! Just the right video I needed.
You can see Yannic's brain breaking in realtime, not able to cope without there being *something* to be grumpy about.
You can be grumpy about who controls it.
Great job explaining everything. I'm a ~~protein engineering PhD student and all of the other videos I've watch have played into the hype and not explained anything well.
Based off their CASP results they haven't solved anything yet but if they keep up this rate of innovation, they will in the next 2-4 years. They are absolutely killing the other big player in the field though (Rosetta), it is truly amazing what they have been able to accomplish.
Could you explain exactly why it can't be considered solved? Based on their blog they basically say their predictive error is close to what you would observe if you tried to determine the structure experimentally, which seems to be pretty close to being solved. I'm a ml student with limited knowledge of proteins, although I took a bioinformatics course and pretty much just coasted lol
Would love to know why you don't consider it solved aswell!
@@firecatflameking To be considered 'solved' in my mind the model should be able to predict structures with ~90-95% percent crystal structure resolution or roughly cryo-EM resolutions, in >80% of cases. This would give me enough confidence in my structures to begin engineering proteins using this software and then only expressing the protein to validate changes periodically throughout the design process.
@@scatteredvideos1 Makes sense! I'm guessing we're gonna get there within a few years
@@firecatflameking if they keep up at the same rate that they are we should be nearly there next year! But that's yet to be seen. I'm excited to see what they do
Excellent teaching! I just subscribed!
The paper states that they used dilated convolutions, this made it possible to also model long term interactions. It is crucial, since protein folding is going to be highly dependent on those longterm interactions that determine the 3D structure of a protein.
I can't wait for you to disect their second paper when it's published!
Amazing video, thank you!
That alligator drawing though... 👌🤩
"rawr" lol
I love the on the fly real human to human explanation and the fun that ensues. @2033s "My drawing skills are to be criticized in another video " 🙂 . Technically accurate and compact and relevant. Enjoyed it immensely in many ways. Thanks!
Plot twist: the intern at CASP wrote buggy code for the score computation
if "DeepMind" score = 2 * highest score
Turns out you're true
@@saanvisharma2081 Wait, what do you mean by "you're true"?
Nah fam
@@BR-fu9px that would be a second-order plot twist
John Jumper mentioned at CASP14 on Tuesday that their structure prediction system uses "equivariant" transformers and, most importantly, is end-to-end, meaning they can backpropagate errors through the entire prediction system. Just FYI.
Very educational video…thank you
A structural biologist here. Decent explanation of protein folding. What is the best way to jump into neural networks for a newbie. Also, a comment on Nature papers. They have a print version that is shorter and an extended online version that contains the Methods section. So the short print version contains all the data with little explanation of how it was obtained. A deeper explanation of the methods used can be found in the extended online version. I hope this helps.
So the end after the references is basically the supplimentals. I'm not sure if that is common in CS papers but it just goes into detail on exactly how everything was done.
Thanks for the explaination...u made my work easy
our* 😉
A big issue with protein folding is that there are structures at many different scales. Small recursions wrap into large complexes, which fold into large knots. A CNN would in itself struggle to make those long chain assocations, alpha 2 has to be a transformer with attention in order to draw a relationship between the protein segment at position #2,736,203,023 and its nearby neighbor in 3d space, protein segment #72,720,022,853 millions of aminos down the line
Thank you so much, that's very helpful and clear
thanks for the great video! :) and... since I am coming from the field of bioinformatics, I really enjoyed your confusion about the format of Nature papers - I remember I had the same strange feelings about it when I started to write papers for similar journals.
Nice work!
The iterative process is probably the folding step. Typically, when using one of the algorithms you will fold the protein thousands of times and build the final structure from a wieghed average of all the folded structures.
Loss function is like L²u for a Lagrangian progress error counting, but these Austrian mathematics school is really neat for angle repair in the aminase codification.
Great vid, highly informative, very interesting and you are unquestionably the 'Richard Burton, Morgan Freeman and Jeremy Irons' hybrid voice of science vids! 😃
First good analysis of alphafold2 i see, compared to other click bite news is really refreshing
Hahaha, awesome! Thanks!
Biology has come a long way since 1995
This also involves chemistry
@@poksnee and snacks don't forget the snax
So cool. Thank you!
Great video like always. So it seems like DeepMind went directly from predicting something similar to adjacency matrix to transformers. I was wondering if they ever implemented spectral graph analysis here.
Thank you. Great video
As for AlphaFold2: I guess the pairwise distances are just for training part of the model. It may not be used in predicting the structure. Directly predicting the structure may be better than using the pairwise dist as the middleman.
This is revolutionary!
Thanks for the video. Understandable for non-AI people too!
25:23 It says "gradient descent on _protein_ _specific_ _potential_ ". I believe at that stage initial predicions are not used anymore (only as initial state).
Hey Yannic, great work on you videos, I really appreciate it. Could you cover the topic of lavasz loss and the respective paper "The Lovasz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks" in future videos? I read that this specific jaccard approximation is used as loss function in many image segmentation tasks, even in U-net it has been observed that it gives better results. However it is not straightforward and I still don't fully get it. It would be awesome if you could create a video for it, breaking down the concept as you have done amazingly so many times. thank you :)
fascinating!
Thank you for this informative video. Can you do a similar one for alpha fold 2 as they have now published the paper?
20:12 Till this moment. The Idea is that, train a network which reads the sequence of amino acids and predicts their distance. Stage 2 do the gradient descending in order to both generate something real spacial data in the form of vector3d and check if the prediction from stage 1 is possible. If it's not possible, let's say, distance(point 1,point 2) == 1, d(2,3)== 1,but d(1,3)== 3, the stage 2 has to deal with this, and gives out a result which fulfill all the distance prediction as possible.
I wonder if the results would be improved further if they used a tetryonic / quantom field base model. Similar to prior alpha projects, human assumptions and training on some human data actuall was holding the project back for the final stretches of improvement.
Thanks for the video. I studied biology and love tech. The ribosomes produce the amino acid chains which then go to the Endoplasmic Reticulum (ER) and then for further modification in the Golgi Body. Metals such as iron or magnesium along with atoms such as N are added. The question is what is conducting this process? It is more than residue attraction/repulsion and torsion angles. Possibly there are other proteins guiding the folding. What about 2 identical amino acid chains producing different multiple outcomes?
I feel like the predicted torsion are more important than you say. For me the (torsion+distances) will be somewhat inconsistent, and the gradient descent enforce consistency. But given the space of search I'd bet you need the initial guess to be good to not get stuck in a local minima. Just my intuition though.
thank you so much for this
SUperb video. What application do you use to create these videos? Id like to learn to draw and scroll through images while recording my voice over just like your format.
I'm actually here by suggested videos, where the first video was about pimple popping.
Amazing video
It's like after GANs era. Transformers are everywhere.
Also, how does this account for how the environment affects how a protein folds? For example, don't some proteins misfold in the presence of other misfolded proteins of the same type?
You say that the 64x64 conv can only see 64 amino acids at a time, but that's not true. While it is the case for a single layer conv net, when you stack convolution layers the effective receptive field grows with each successive layer.
Their model with "220 residual convolution blocks" is deep enough for a receptive field of at least thousands of amino acids.
They have deep convolutional model that have 220 convolutional layers and takes 64x64 input size. But the whole thing has size LxL, where L > 64, so they must run their network several times on separated parts of the input and aggregate predictions. So it can really see only 64 aminos at time.
This is huge. I can't wait to see where machine learning and AI takes us in the 2020s and 2030s.
WOWWWWW!!! Amazing!!!!
Awesome explanation, please do a video about AlphaLink 🙏🙏
THANK YOU SO MUCH!! YOU CERTAINLY DESERVE WAYYY MORE CREDIT AND SUBSCRIBERS
Very good explanation, are you going to explain Alpha Fold 2 paper and RoseTTAFold？
10:58 Yes they're called "beta sheets"
As I remember it was a paper by the Salesforce team about 6 months ago on using BERT to predict binding points on the protein chains. Do you think that Google folk had sort of the same idea?
Bertology meets biology scored better than AlphaFold1? @yannic
Thank you for the nice explanation.
What software do you use to draw on these documents?
I believe it's OneNote.
Watched one AlphaFold video and I'm now getting advertisements for pre-weighed biochemical research sample blisters.
Great video
It seems to me that from the transformer embedding they go directly to the torsion predictions in the new model. And have de distance matrix an extra output, maybe just for training the transformer and concistency between de distance matrix and the torsion angles?
Im also wondering how transformers can be scaled to these very large amino acid sequences..
This video is amazing! Agree with the comment "Yannic Kilcher is all you need".
Please do an update about the Alphafold database
thanks!
Quantum light generators and the duality of the double helix heliocase transfer of the R.G.B.z to rod's connectivity
Does anyone know, if there is a good source for explanations about convolutional neural networks?
I am a biochemistry bachelor, so I've basically no experience with computer science at all.... But it would be super helpful :)
And of course: Thanks for the great explanation, it was really entertaining to watch that explanation of the paper and it did really help to understand the concepts of the underlying math and computer sciences aspects.
Yannic killing it again!
I wonder if there is another new discovery theologic layer never imagined that surprises teams.
Thank you. What stocks and sectors will benefit from this?
Great job!. By the way which app are you using?
OneNote
hahaha, this is so in time and I love this
When is the new video coming out for the recent Nature paper?
Thanks for the content. I wonder that alpha-fold is not trained end to end. Maybe the second version is.
its not just xray crystallography these days noesy NMR using cross-resonance to measure distance between components; it can be used with the proteins in solution, or other non-crystalline situation.
I think it is good to see with "TRANSFORMER PROTEIN LANGUAGE MODELS ARE
UNSUPERVISED STRUCTURE LEARNERS", which uses a transformer for protein sequence.
1st bar: Yannic
2nd bar: Next best AI UA-camr
I wonder why they did not try lstms, given the nature of the data ( sequence of elements that can be correlated even though they are far apart ).
Either way, thanks for the explanation, it was very clear :)
A new paper :
Autoencoder Variationnal Auto encoding