Professor, your presentation was insightful. It reminded me of a quote by Professor Henry that says *"A model must be wrong, in some respects, else it would be the thing itself. The trick is to see where it is right"*
To Cynthia’s question at minute 17 about changing AlphaFold 3’s random seed... at around 1:01:20 in this other talk Sergey Ovchinnikov group found that for nanobody-protein binding it took testing about 20 random seeds for AlphaFold to confidently predict binding orientation. ua-cam.com/video/qjFgthkKxcA/v-deo.html
Antibodies are harder than nanobodies, and in AF3, for accuracy reports (e.g., Fig. 1c), "All scores are reported from the top confidence-ranked sample out of five model seeds (each with five diffusion samples), except for protein-antibody scores, which were ranked across 1,000 model seeds for both models (each AF3 seed with five diffusion samples)." So they needed 199-fold more seeds (and diffusion samples) to achieve an accuracy improvement from about 23% in AF2 to about 53% in AF3. (Measured as percentage DockQ > 0.23).
@@DonaldLab Thank you for the addition context! Is the thought that antibodies are more challenging than nanobodies because the training dataset is mostly composed of smaller proteins (ie antibodies binding to large protein complexes could be out of distribution) or is the challenge the greater degrees of freedom for larger structures?
Thank you so much Professor. I learnt understanding the subtle yet significant errors in computational protein modeling is essential for researchers pursuing drug discovery, as it can greatly inform and improve their research endeavors.
It seems like the root of Cynthia’s question around 10 minutes in was something like “are the ‘wrong’ structures hallucinated by AlphaFold still energetically plausible?”. Are the wrong structures in a local free energy minima? Do the wrong structures achieve free energies as low as the correct structures? For my own curiosity, why are only a subset of possible stable folding conformations observed? Chaperones? Evolutionary selection of proteins that fold to a single conformation?
There is no experimental evidence that the wrong structures in this video are in local free energy minima nor that they achieve free energies as low as the correct structures. There is no experimental evidence that these structures can be observed by biophysical methods, and we do not see them by Xtal or Cryo-EM. It could be possible to perform in-silico free energy calculations, but the models (and algorithms) are approximate, so this would not be definitive. It is not clear that only a subset of possible stable folding conformations are observed. Such claims are controversial. By NMR, you can observe minor populations of as low as 2%. However, the difficulties you mention could be due to flexibility. We have some ways of modelling and addressing this, see ua-cam.com/video/wMUhMZsqMtg/v-deo.htmlsi=7-Ip3OGh0kW1MCmI and ua-cam.com/play/PLeXxBB7xQ8dr4I65AhiyX6cNJg5PkJhI8.html&si=wag-VqP9SxYFHtZy
Thank you for your question. It definitely could. However, I believe our structures are cryo-EM stuctures, not Xtal, and they are Fab's, not Scvf. The structures in our Cell paper are PDB Ids 8GAJ, 8FLW, and, 8FL1.
There are predictions that AF is good at, and there are things where it is not. The latter may improve with time, or new methods may have to be developed. My personal opinion is that the Nobel Prize to Deepmind/Demis Hassabis & John Jumper is well deserved. When I have corresponded with John about PSP he has always been a good scientist and a perfect gentleman.
![[TALK 21] AlphaFold: Use and Applications - Sami Chaaban](http://i.ytimg.com/vi/yJKfn6rvHmg/mqdefault.jpg)
Professor, your presentation was insightful. It reminded me of a quote by Professor Henry that says *"A model must be wrong, in some respects, else it would be the thing itself. The trick is to see where it is right"*
To Cynthia’s question at minute 17 about changing AlphaFold 3’s random seed... at around 1:01:20 in this other talk Sergey Ovchinnikov group found that for nanobody-protein binding it took testing about 20 random seeds for AlphaFold to confidently predict binding orientation. ua-cam.com/video/qjFgthkKxcA/v-deo.html
Antibodies are harder than nanobodies, and in AF3, for accuracy reports (e.g., Fig. 1c), "All scores are reported from the top confidence-ranked sample out of five model seeds (each with five diffusion samples), except for protein-antibody scores, which were ranked across 1,000 model seeds for both models (each AF3 seed with five diffusion samples)." So they needed 199-fold more seeds (and diffusion samples) to achieve an accuracy improvement from about 23% in AF2 to about 53% in AF3. (Measured as percentage DockQ > 0.23).
@@DonaldLab Thank you for the addition context! Is the thought that antibodies are more challenging than nanobodies because the training dataset is mostly composed of smaller proteins (ie antibodies binding to large protein complexes could be out of distribution) or is the challenge the greater degrees of freedom for larger structures?
@@andrewmaier1995 It could be both. But there is recent progress! See for example: www.cell.com/cell-reports/fulltext/S2211-1247(23)00722-2
Thank you so much Professor. I learnt understanding the subtle yet significant errors in computational protein modeling is essential for researchers pursuing drug discovery, as it can greatly inform and improve their research endeavors.
Thank you for sharing your knowledge and this zoom meeting with us
It's very helpful :)
It seems like the root of Cynthia’s question around 10 minutes in was something like “are the ‘wrong’ structures hallucinated by AlphaFold still energetically plausible?”. Are the wrong structures in a local free energy minima? Do the wrong structures achieve free energies as low as the correct structures?
For my own curiosity, why are only a subset of possible stable folding conformations observed? Chaperones? Evolutionary selection of proteins that fold to a single conformation?
There is no experimental evidence that the wrong structures in this video are in local free energy minima nor that they achieve free energies as low as the correct structures. There is no experimental evidence that these structures can be observed by biophysical methods, and we do not see them by Xtal or Cryo-EM.
It could be possible to perform in-silico free energy calculations, but the models (and algorithms) are approximate, so this would not be definitive.
It is not clear that only a subset of possible stable folding conformations are observed. Such claims are controversial. By NMR, you can observe minor populations of as low as 2%. However, the difficulties you mention could be due to flexibility. We have some ways of modelling and addressing this, see ua-cam.com/video/wMUhMZsqMtg/v-deo.htmlsi=7-Ip3OGh0kW1MCmI and ua-cam.com/play/PLeXxBB7xQ8dr4I65AhiyX6cNJg5PkJhI8.html&si=wag-VqP9SxYFHtZy
Prof I noticed that the crystal complex contains Scfv, whereas in AF3 it is Fab. Would this affect the AF3 precision?
Thank you for your question. It definitely could.
However, I believe our structures are cryo-EM stuctures, not Xtal, and they are Fab's, not Scvf. The structures in our Cell paper are PDB Ids 8GAJ, 8FLW, and, 8FL1.
Thanks for sharing this video
My pleasure
I wonder if the Nobel committee has seen this presentation
There are predictions that AF is good at, and there are things where it is not. The latter may improve with time, or new methods may have to be developed.
My personal opinion is that the Nobel Prize to Deepmind/Demis Hassabis & John Jumper is well deserved. When I have corresponded with John about PSP he has always been a good scientist and a perfect gentleman.