Hi Steve, thank you for your amazing videos. They spark pure joy in relearning control systems! A topic that I've studied in university and unfortunately didn't enjoyed at the time. I'm looking forward to read your book "Data-Driven Science and Engineering"! Hope you and your family are safe and please keep up with the amazing work!
The shape of things to come! I worked for many years in research labs doing computer vision applied to biomedical engineering. When I discovered deep learning in 2016, my jaw just dropped. DL provides such a different way to do computer vision and NLP. Not only do we get better at what we do, we also better understand how it is done. Image segmentation and classification were hard problems then; it is less of an issue now. This new approach on data-based physical modeling produced the same jaw-dropping moment on me. During my PhD studies, I often wondered/dreamed about ways to do it instead of using the standard approaches from basic principles. The few people in history who could get the right PDEs this way are in the pantheon of physics for the right reasons. I’s not easy. Getting the DEs or PDEs purely from observations and measurements will fasten the pace of developments in physics, chemistry, biology, etc. What also fascinates me is the possibility of discovering new terms in the PDEs already known. An example? Think about the Maxwell equations without the last, weird term for displacement currents. Without it, Maxwell could not have predicted that light is an electromagnetic wave phenomenon! No radio or TV. Who knows what else we will find. Particle physics is another field where interesting discoveries await. Maybe some missing terms in the PDEs of field theory will shed light on new fundamental forces and on the nature of dark matter. I have to stop here because I’ll get carried away. It’s a good time to be a scientist and witness the rise of a new approach that will boost the power of mathematical modeling. Bravo!
steve, having come from a deep learning background, this topic is so refreshingly transparent and elegant. thanks so much for the beautifully presented material. I'm excited to start exploring SINDy for the problems im looking at.
I approached the person who underwrote the second connectionist summer of the 1980s, Systems Development Foundation executive Charles Sinclair Smith, with the original SINDy paper when it first came out. He and I had been going back and forth about Algorithmic Information as opposed to Shannon Information model selection for a number of years. When he read the SINDy paper he said I had finally gotten him to reorient his thinking. His original motivation for financing Hinton, Werbos, Rumelhart, McClelland, etc. was dynamical systems modeling of the energy economy, as he had co-founded the Energy Information Agency of the DoE under President Carter. However, he was unaware of the history of Algorithmic Information at that time, and his statistical approach to data-driven policy analysis originating with his professor John Tukey had gotten him rather stuck in that mode of thinking -- along with the rest of the social sciences to the present day. It may be premature to hope for the social sciences in general to realize they've been barking up the wrong tree, even after all these decades of Moore's Law during which Ray Solomonoff's proof has been known, that Algorithmic Information approximation is the optimal model selection criterion. But let's hope its not too late when they finally do "reorient their thinking". Literally hundreds of billions of dollars per year are at stake in social policy and perhaps more than that in increasingly polarized political rancor. Think about it like this: If you had a billion dollars or so to spend per year, and access to the Federal Government's data, what kind of machine learning infrastructure could be brought to bear to generate better macrosocial models?
I have a few questions. What is the use of Neural networks in this framework? I mean, if you define the polynomials in advance, why do we need NNs? Why cannot we find the parameters by using the standard regression techniques, like in established methods? How is this different from standard well-established system identification methods? How does this compare with the methods in the literature? What are its advantages over the other methods developed so far in the literature?
Sir, please make a video on how to plot bifurcation diagrams and poincare plots for the continuous dynamical system. I couldnt it find it anywhere online.
This is great. I was essentially tackling this problem, basically building this Ax=b system, using nonlinear features. I did not know about lasso algorithm, and I found your previous video on that; I was only using a naive Least Square (bc of the simplicity of the solution haha, I was aware of Tikhonov, but then I thought how much difference it would really make). I would definitely try out LASSO and SINDy on my time series
Waiting for the success of SINDy in large-scale real world problems with highly nonlinear and unknown dynamics. I am planning to try it out for problems related to Water Distribution Systems modelling and Environmental Flows
This is very beautiful and useful methodology! I think, it has potential applications to controlling undesirable or pathological chains of chemical reactions in a human body. Just speculating, may be it can be applied to understand deeper cancer or prion diseases. It would be also interesting to know if the method is resilient to noisy measurements. Looking forward to your amazing lectures!
Dr. Brunton, these videos continue to amaze. I suspect I'll have many questions for you in the upcoming days and weeks about how to implement SINDy. Pondering how to design some sort of project wherein I capture experimental data (perhaps using an arduino?) and using these techniques to verify the governing equations. Perhaps something having to do with beam deflection or fluid flow..
I’m so happy I met this video, I’m currently exploring SINDy as a method of equation discovery and I am actually having trouble with the Python package pysindy. There’s a consistent error of np.maths.factorial or list.T, how do I get the correct or stable version of the package
Could you please link the papers that use SINDy technology to learn non-linear dynamics from noisy data or data having stochastic dynamics? Many thanks!
Could sindy be used for non time series data but sort of sequential data? For example data where there are forces acting on a particular region and layers of laminates (of different angles at each layer) chosen? Could it find a relation between the layers chosen and the forces?
Steve sir can you tell me what is the future most growing jobs in the world. In feild of automation,machine learning,digital technology,energy technology and food technology I mean how will people going to select a job in populating world
Hello Steve, thank you for sharing this knowledge. I have a question, have you ever seen this applied to system dynamics (in the field of what JayW. Forrester proposed), like how from data we could learn the structure of the system (diagrams of flows and levels) or how we can exract the behaviour patterns of the system from data to find the archetypes(i.e. Limits to growth, tragedy of commons,Escalation, that are explained in some works of Donella Meadows) that system is experiencing. Thank you!
13:05 It's still not clear how to get time derivatives of the states. If we could know them, then the problem would be solved by the least squares and no new methods needed. It seems, I missed something.
One way to approximate the derivatives is to compute the finite differences of the time series. In my applications of SINDy I’ve used fourth order centered differences
@@milesmedina6193 of course you can estimate derivatives numerically, and the finite difference is not the best way. The only trouble is the derivative estimation is an ill-posed problem. But Steve told that the requirement can be relaxed. I didn't understand how. On the other hand, if u know derivatives, then no need to use SINDy. U can use the usual least squares instead. Actually, the problem at 15:15 looks like least squares, doesn't it? What the point of SINDy then?
@@maksim-surov SINDy helps you find the governing equations (relation between derivatives and measurements). For fitting the library of functions, you can choose least squares preferrably a sparse version of least squres. Usually governing equations are short, at least relative to a huge library of terms.
@@maksim-surov the main goal is not to estimate the parameters, but to receive the actual model dynamics and their equations with a sparse fitting! You do not know the equations! If you fit a non-sparse model, you could receive many terms from the library describing the equations. You want the smallest possible set of terms from the library to describe the data and form the dynamics equations.
I understood after reading documentation. The SINDy finds a "simplest" polynomial which fits the rhs. Actually, in the considered example the least squares will give the same result despite the objective function is different. Probably, it would be more representative to compare least squares fitted and SINDy fitted dynamics for a non-polynomial system.
Hi Professor Steve, LOVE YOUR VIDEOS. Have you considered forming a discord group or something like that? There's quite a following on your channel, I am sure there are many people like me would like a discussion on control/ Sparsity alike topics
Hmm, you take the humor out of Einstein’s quote. We want things as simple as possible, but not the impossible! But I get it. Too sparse does and it does not work.
![AI/ML+Physics Part 1: Choosing what to model [Physics Informed Machine Learning]](http://i.ytimg.com/vi/ARMk955pGbg/mqdefault.jpg)
Hi Steve, thank you for your amazing videos. They spark pure joy in relearning control systems! A topic that I've studied in university and unfortunately didn't enjoyed at the time. I'm looking forward to read your book "Data-Driven Science and Engineering"! Hope you and your family are safe and please keep up with the amazing work!
Amazin brother! Now new Syndy hardware becomes demanding on the market
Just noticed this video was uploaded a couple of weeks ago. Look forward to the following videos. Thanks, Prof. Brunton.
Your work is really exceptional and deserve a salute...great work!!
The shape of things to come!
I worked for many years in research labs doing computer vision applied to biomedical engineering. When I discovered deep learning in 2016, my jaw just dropped. DL provides such a different way to do computer vision and NLP. Not only do we get better at what we do, we also better understand how it is done. Image segmentation and classification were hard problems then; it is less of an issue now.
This new approach on data-based physical modeling produced the same jaw-dropping moment on me. During my PhD studies, I often wondered/dreamed about ways to do it instead of using the standard approaches from basic principles. The few people in history who could get the right PDEs this way are in the pantheon of physics for the right reasons. I’s not easy. Getting the DEs or PDEs purely from observations and measurements will fasten the pace of developments in physics, chemistry, biology, etc.
What also fascinates me is the possibility of discovering new terms in the PDEs already known. An example? Think about the Maxwell equations without the last, weird term for displacement currents. Without it, Maxwell could not have predicted that light is an electromagnetic wave phenomenon! No radio or TV. Who knows what else we will find. Particle physics is another field where interesting discoveries await. Maybe some missing terms in the PDEs of field theory will shed light on new fundamental forces and on the nature of dark matter.
I have to stop here because I’ll get carried away. It’s a good time to be a scientist and witness the rise of a new approach that will boost the power of mathematical modeling. Bravo!
steve, having come from a deep learning background, this topic is so refreshingly transparent and elegant. thanks so much for the beautifully presented material. I'm excited to start exploring SINDy for the problems im looking at.
Awesome Thanks! Got the hard copy of the book Data Driven Science and Engineering, very helpful
Where can I get the book, I’m really in need of it
Absolutely love yours and Nathan work. Going thru your book on DDSE. The animations are always great!! Thank you for sharing!!
I approached the person who underwrote the second connectionist summer of the 1980s, Systems Development Foundation executive Charles Sinclair Smith, with the original SINDy paper when it first came out. He and I had been going back and forth about Algorithmic Information as opposed to Shannon Information model selection for a number of years. When he read the SINDy paper he said I had finally gotten him to reorient his thinking. His original motivation for financing Hinton, Werbos, Rumelhart, McClelland, etc. was dynamical systems modeling of the energy economy, as he had co-founded the Energy Information Agency of the DoE under President Carter. However, he was unaware of the history of Algorithmic Information at that time, and his statistical approach to data-driven policy analysis originating with his professor John Tukey had gotten him rather stuck in that mode of thinking -- along with the rest of the social sciences to the present day. It may be premature to hope for the social sciences in general to realize they've been barking up the wrong tree, even after all these decades of Moore's Law during which Ray Solomonoff's proof has been known, that Algorithmic Information approximation is the optimal model selection criterion. But let's hope its not too late when they finally do "reorient their thinking". Literally hundreds of billions of dollars per year are at stake in social policy and perhaps more than that in increasingly polarized political rancor. Think about it like this: If you had a billion dollars or so to spend per year, and access to the Federal Government's data, what kind of machine learning infrastructure could be brought to bear to generate better macrosocial models?
Great video, Steve! Can't wait to see the rest of the series :)
I have a few questions.
What is the use of Neural networks in this framework? I mean, if you define the polynomials in advance, why do we need NNs?
Why cannot we find the parameters by using the standard regression techniques, like in established methods?
How is this different from standard well-established system identification methods?
How does this compare with the methods in the literature? What are its advantages over the other methods developed so far in the literature?
Sir, please make a video on how to plot bifurcation diagrams and poincare plots for the continuous dynamical system. I couldnt it find it anywhere online.
This is great. I was essentially tackling this problem, basically building this Ax=b system, using nonlinear features. I did not know about lasso algorithm, and I found your previous video on that; I was only using a naive Least Square (bc of the simplicity of the solution haha, I was aware of Tikhonov, but then I thought how much difference it would really make). I would definitely try out LASSO and SINDy on my time series
I think you might need ARX with AIC and BIC.
Was Eagerly waiting for this prof. Thanks✨
Waiting for the success of SINDy in large-scale real world problems with highly nonlinear and unknown dynamics. I am planning to try it out for problems related to Water Distribution Systems modelling and Environmental Flows
This is very beautiful and useful methodology! I think, it has potential applications to controlling undesirable or pathological chains of chemical reactions in a human body. Just speculating, may be it can be applied to understand deeper cancer or prion diseases. It would be also interesting to know if the method is resilient to noisy measurements. Looking forward to your amazing lectures!
Cant wait professor! Send them over, quick! :))
Another great video, thank you for the knowledge
Hi steve, thank you for such fantastic videos and give me purpose as a mechanical engineer on what to do in future
Dr. Brunton, these videos continue to amaze. I suspect I'll have many questions for you in the upcoming days and weeks about how to implement SINDy. Pondering how to design some sort of project wherein I capture experimental data (perhaps using an arduino?) and using these techniques to verify the governing equations. Perhaps something having to do with beam deflection or fluid flow..
Wow! Now that is a good use of machine learning.
Cannot wait for the next video!
Can't wait for your next video
How effective is SINDy to identify Saturation Nonlinearities?
I'd be interested to see this applied to cosmology, especially in the mond vs gr+ dark matter debate
ahhh! Just found that you are the author of SINDY!
I’m so happy I met this video, I’m currently exploring SINDy as a method of equation discovery and I am actually having trouble with the Python package pysindy.
There’s a consistent error of np.maths.factorial or list.T, how do I get the correct or stable version of the package
Could you please link the papers that use SINDy technology to learn non-linear dynamics from noisy data or data having stochastic dynamics? Many thanks!
Could sindy be used for non time series data but sort of sequential data? For example data where there are forces acting on a particular region and layers of laminates (of different angles at each layer) chosen? Could it find a relation between the layers chosen and the forces?
Steve sir can you tell me what is the future most growing jobs in the world.
In feild of automation,machine learning,digital technology,energy technology and food technology I mean how will people going to select a job in populating world
Steve, might a discrete dynamical system be an appropriate application for the SINDy technique or is SINDy more optimized for continuous systems?
I kind of miss the marker-on-glass videos. Make more great lectures :)
Can you comment on the scale of the Xi coefficients? Are they naturally finding the `\sigma(y - x) = 10*(y-x)` (etc.) that you seeded the system with?
How does this differ from AI feynman??
Amazing talk!
Sir can you tell me what is future of software engineers.
beautiful algorithm
Very cool! This seems like it could work well with financial time series data in various ways too.
Is this an assumption or why do you think so?
Aren't the Financial time series are mostly nonlinear and multimodal ? So what are you exactly asking 🤔
Hello Steve, thank you for sharing this knowledge. I have a question, have you ever seen this applied to system dynamics (in the field of what JayW. Forrester proposed), like how from data we could learn the structure of the system (diagrams of flows and levels) or how we can exract the behaviour patterns of the system from data to find the archetypes(i.e. Limits to growth, tragedy of commons,Escalation, that are explained in some works of Donella Meadows) that system is experiencing. Thank you!
Has anybody tried to reverse engineer Kepler's laws out of Tycho🎉 Brahe's data?
I’m pretty sure I remember Miles Cranmer and co. doing something like this
13:05 It's still not clear how to get time derivatives of the states. If we could know them, then the problem would be solved by the least squares and no new methods needed. It seems, I missed something.
One way to approximate the derivatives is to compute the finite differences of the time series. In my applications of SINDy I’ve used fourth order centered differences
@@milesmedina6193 of course you can estimate derivatives numerically, and the finite difference is not the best way. The only trouble is the derivative estimation is an ill-posed problem. But Steve told that the requirement can be relaxed. I didn't understand how. On the other hand, if u know derivatives, then no need to use SINDy. U can use the usual least squares instead. Actually, the problem at 15:15 looks like least squares, doesn't it? What the point of SINDy then?
@@maksim-surov SINDy helps you find the governing equations (relation between derivatives and measurements). For fitting the library of functions, you can choose least squares preferrably a sparse version of least squres. Usually governing equations are short, at least relative to a huge library of terms.
@@maksim-surov the main goal is not to estimate the parameters, but to receive the actual model dynamics and their equations with a sparse fitting! You do not know the equations! If you fit a non-sparse model, you could receive many terms from the library describing the equations. You want the smallest possible set of terms from the library to describe the data and form the dynamics equations.
I understood after reading documentation. The SINDy finds a "simplest" polynomial which fits the rhs. Actually, in the considered example the least squares will give the same result despite the objective function is different. Probably, it would be more representative to compare least squares fitted and SINDy fitted dynamics for a non-polynomial system.
How do you make your videos? I always wondered how.
Hi Professor Steve, LOVE YOUR VIDEOS. Have you considered forming a discord group or something like that? There's quite a following on your channel, I am sure there are many people like me would like a discussion on control/ Sparsity alike topics
That's great!!!!
Thanks a lot
Thanks a lot !
i am really interested. thanks for the videos,professor.
First principles----->Occam's Razor---->Sparse model
So, God, human, and machines, they converge now?
Amazing!
Hmm, you take the humor out of Einstein’s quote. We want things as simple as possible, but not the impossible! But I get it. Too sparse does and it does not work.
very interesting
Waw.. this is the ultimate science and technology.. can we reverse engineer and rediscover Schrodinger, Navier-Stocks and probably GR!!!
brilliant
_👏👏👏👏👏👏👏_
YEAH ~!
hype!
Very interesting