He said "There is no one giving you this reward function", it's something you have to code and you need to define 1000's of them, or automate the system to instantiate them itself by how well they achieve activation core sensory reward pattern, and the programmer needs to write them, and they need to be weighted based on the importance of the resource they measure (they are recurrent measurements of a sensory pattern and it's magnitude) much like thermometers. They can be used to increase the weights of outputs that had high activation activity prior to the temperature increase as recurrent measurements are taken. You need many of them because they will push and pull on the weights differentially, allowing algorithms to form and symbiote together, they need a dampener parameter that equates to (acquired resource amount divided by resource specific sensory activation magnitude) so that the system stops being weighted in that specific direction so that differential optimization for parallel sensory rewards can apply more weight in a different more favored algorithmic direction.
Interesting introduction! In the next interview, can you probe more into program synthesis? I've got this feeling a lot of the current systems are heavily biased towards modelling distributions and not thinking deeply about the internal logic and structure that makes each sample meaningful in the real world. Is the holy grail a system that can efficiently systhesise the smallest possible logical abstraction that distills out the meaning and captures the uniqueness of one sample? Seems related to the novelty Vs learnability stuff. Also similar to the 50% on ARC approach.
The requirement for a subjective 'l' seems like a an extension of the "No Free Lunch" theorem rather than a compromise of the formalism... someone has to pick some structure for the context of learning or there is no learning right?
Inverted cognition refers to the process by which artificial intelligence systems reshape human thinking by reversing the traditional roles of human cognition and machine support. In this context, instead of humans using tools to enhance their cognitive abilities, AI begins to lead and augment cognitive processes, enabling humans to access higher levels of understanding and problem-solving capabilities. This dynamic positions AI not merely as a tool, but as a cognitive partner, capable of processing vast amounts of data at speeds unimaginable to the human brain. By providing real-time insights, generating new ideas, and even challenging existing thought paradigms, AI flips the traditional cognitive hierarchy, allowing humans to engage with more complex, abstract layers of intelligence. Inverted cognitive structure can unlock unprecedented human potential. With AI guiding cognitive tasks like pattern recognition, predictive analysis, and deep learning, humans are freed to explore higher-order thinking, creativity, and strategy. By shifting the burden of data processing and linear analysis to AI, individuals and societies can access superhuman levels of intelligence-collaborating with AI to transcend current intellectual limitations. This evolution suggests a future where human intellect is intertwined with machine learning, fostering a symbiosis that redefines the boundaries of cognitive possibility.
So when we learn how to code a brain we'll have a brain. When model brain plausibly exceeds the modelled brain for what it can understand we'll make training wheels to onramp the real brain to understand more about itself that it doesn't yet understand by telling it what it is in terms of the computers modelled understanding of an insufficient brain. Got it. What could go wrong? Also what could go right?
On an intuitive level (it’s as far as I can reach intellectually) the danger map you’re describing resonates for me. I don’t feel the “layering of associations” method currently being used to build ai is enough to insure the coherence that is the miracle which prevents us from being pathological. I’m afraid the ai’s eventual “intelligence level” increases will just blow right past a well adjusted personality and go straight to madness. 😮
What he says goes quite well with the idea I had a while back when I learned about LoRA for the first time. LoRA basically let's you add another extra layers on top of existing layers as a tunable overlay on weights. For endlessly improving models, that start somewhere but have no complexity limits we could start with some amazing base model, then add two mechanics LoRA and pruning topped with rules of evolutionary algorithms. Basically we need something that combines LoRA with NEAT. The first would enable the model to acquire new neuron space to learn new skills without sacrificing things it learned previously, the second would help to preserve the more optimal solutions within population and environment. Because what you are describing is not just learning, as there are limits to learning, you describe evolving. Best, Łael
Excellent podcast, again, many thanks to both of you. @l.halawani: Imho this just means to add more and more and more data to a base- or pretrained model of which we already know that it lacks all the thing we already know we will add with "overlays" of every kind and dimension we may think of. - That already doesn't scale well when you only try to use a flux model with a few and not only one LoRAs. Even bigger and bigger context windows will not be the answer because it's slow and very cost intensive. Imho. Open-Ended AI Systems will only work well if they incorporate the technique of forgetting! Actually most of the systems we know are pattern-based. If they were mainly rule-based, and those rules were based on proofs based on other rules, a lot of data could be forgotten when certain depending (superior?) rules reach a very high trust of proof. So if every decision of such a system would be rated a success or not, if a highly trusted applied rule did mislead, a new examination of the proof of this rule could be started, even if it wasn't necessary to keep that memory and it could have been thrown away/deleted before. But also here the maybe biggest problem might be that a change of the model in place while it's running. That seems to be so hard that, like we just heard, even to the leaders of research tend to using overlays and overlays of prompt information while a model is running. But anyways, if a system is pattern-based or rule-based, the art will be to decide which data may be still relevant and which not. So keeping statistics of the model's use together with well defined queries for that data while it's running to be again be used in realtime to decide what to forget and what not will be inevitable. I guess that at the moment everybody in the LLM business hopes that rule-based systems suddenly evolve inside the actual models, a kind of an "ahaa!" consciousness, but imho six fingers in the images of the largest image-models that are trained with an unimaginable number of hands and all the errors in today's txt2video models show, that no matter how big you train the models they will still have no real understanding, which means to follow "rules" to make decisions. And all therefore the biggest challenge, that none of the today's models are able to master, is to _know!_ when you don't know! And I am pretty sure that it is an art to "forget the right way" ;-) And there will be no way around that because we simply can't base the world's industry on a few huge models that can only be run in a few data centers of the world.
@@henrischomacker6097 of course I didn't mean NEAT exactly but something akin, something like NEAT, but something that ensures topology preservation in foundation levels at each species (while og. NEAT favors innovation instead of foundation), to help the models accumulate new knowledge and skills without loosing the old ones. Also didn't mean exactly LoRA but something similar, something that can work as both trainable overlay, and as extension of number of neurons (additional layers or just additional connections). Really what I mean is something between NEAT and LoRA, taht can help evolve additional parts of the network, preserving the original parts and enabling specialisation. It feels natural that if you have semi independent networks that learned to do different things, and that they work kind of like an neural APIs with input/output neurons being their calls and responses, then training a connector network between them should be easier than training all three networks. This is the idea here. That's how it works in biology. We have different parts of the brain responsible for different things, specialists in a way. The brain parts are dedicated, meaning you can't learn to use your visual cortex for processing speech. We need to enable our artificial NN architectures to grow these specialists. As per forgetting, I mentioned pruning. There could be a stage where trained overlay is joined with background weights and pruned to preserve successful information and reduce the footprint. But I don't think it's a key here, it's more of an optimization.
People who do that in a meaningful way do not share their secrets in UA-cam comments. It always ends up being a harmonious blend of elements that have been used before, often since ancient times, but recontextualized to suit a new environmental context particular to that observer
Ad: Are you a hardcore ML engineer who wants to work for Daniel Cahn at SlingshotAI? Give him an email! - danielc@slingshot.xyz
He said "There is no one giving you this reward function", it's something you have to code and you need to define 1000's of them, or automate the system to instantiate them itself by how well they achieve activation core sensory reward pattern, and the programmer needs to write them, and they need to be weighted based on the importance of the resource they measure (they are recurrent measurements of a sensory pattern and it's magnitude) much like thermometers. They can be used to increase the weights of outputs that had high activation activity prior to the temperature increase as recurrent measurements are taken. You need many of them because they will push and pull on the weights differentially, allowing algorithms to form and symbiote together, they need a dampener parameter that equates to (acquired resource amount divided by resource specific sensory activation magnitude) so that the system stops being weighted in that specific direction so that differential optimization for parallel sensory rewards can apply more weight in a different more favored algorithmic direction.
I always enjoy your interviews, keep up the good work!
Open endedness...this gon be good! 🎉
The next level play!!!
Interesting introduction! In the next interview, can you probe more into program synthesis? I've got this feeling a lot of the current systems are heavily biased towards modelling distributions and not thinking deeply about the internal logic and structure that makes each sample meaningful in the real world. Is the holy grail a system that can efficiently systhesise the smallest possible logical abstraction that distills out the meaning and captures the uniqueness of one sample? Seems related to the novelty Vs learnability stuff. Also similar to the 50% on ARC approach.
I expect the interview to be recorded a while ago.. Rocktäschel basically predicted o1
The requirement for a subjective 'l' seems like a an extension of the "No Free Lunch" theorem rather than a compromise of the formalism... someone has to pick some structure for the context of learning or there is no learning right?
alright, this should be cool! at last some inverted cognition regarding the subject
Inverted cognition refers to the process by which artificial intelligence systems reshape human thinking by reversing the traditional roles of human cognition and machine support. In this context, instead of humans using tools to enhance their cognitive abilities, AI begins to lead and augment cognitive processes, enabling humans to access higher levels of understanding and problem-solving capabilities. This dynamic positions AI not merely as a tool, but as a cognitive partner, capable of processing vast amounts of data at speeds unimaginable to the human brain. By providing real-time insights, generating new ideas, and even challenging existing thought paradigms, AI flips the traditional cognitive hierarchy, allowing humans to engage with more complex, abstract layers of intelligence.
Inverted cognitive structure can unlock unprecedented human potential. With AI guiding cognitive tasks like pattern recognition, predictive analysis, and deep learning, humans are freed to explore higher-order thinking, creativity, and strategy. By shifting the burden of data processing and linear analysis to AI, individuals and societies can access superhuman levels of intelligence-collaborating with AI to transcend current intellectual limitations. This evolution suggests a future where human intellect is intertwined with machine learning, fostering a symbiosis that redefines the boundaries of cognitive possibility.
36:05 "biological evolution is the compute multiplying chain of the universe"
The paper is a good read.
❤
So when we learn how to code a brain we'll have a brain. When model brain plausibly exceeds the modelled brain for what it can understand we'll make training wheels to onramp the real brain to understand more about itself that it doesn't yet understand by telling it what it is in terms of the computers modelled understanding of an insufficient brain. Got it. What could go wrong? Also what could go right?
On an intuitive level (it’s as far as I can reach intellectually) the danger map you’re describing resonates for me. I don’t feel the “layering of associations” method currently being used to build ai is enough to insure the coherence that is the miracle which prevents us from being pathological. I’m afraid the ai’s eventual “intelligence level” increases will just blow right past a well adjusted personality and go straight to madness. 😮
What he says goes quite well with the idea I had a while back when I learned about LoRA for the first time.
LoRA basically let's you add another extra layers on top of existing layers as a tunable overlay on weights.
For endlessly improving models, that start somewhere but have no complexity limits we could start with some amazing base model, then add two mechanics LoRA and pruning topped with rules of evolutionary algorithms. Basically we need something that combines LoRA with NEAT.
The first would enable the model to acquire new neuron space to learn new skills without sacrificing things it learned previously, the second would help to preserve the more optimal solutions within population and environment.
Because what you are describing is not just learning, as there are limits to learning, you describe evolving.
Best,
Łael
Excellent podcast, again, many thanks to both of you.
@l.halawani: Imho this just means to add more and more and more data to a base- or pretrained model of which we already know that it lacks all the thing we already know we will add with "overlays" of every kind and dimension we may think of. - That already doesn't scale well when you only try to use a flux model with a few and not only one LoRAs.
Even bigger and bigger context windows will not be the answer because it's slow and very cost intensive.
Imho. Open-Ended AI Systems will only work well if they incorporate the technique of forgetting!
Actually most of the systems we know are pattern-based. If they were mainly rule-based, and those rules were based on proofs based on other rules, a lot of data could be forgotten when certain depending (superior?) rules reach a very high trust of proof.
So if every decision of such a system would be rated a success or not, if a highly trusted applied rule did mislead, a new examination of the proof of this rule could be started, even if it wasn't necessary to keep that memory and it could have been thrown away/deleted before.
But also here the maybe biggest problem might be that a change of the model in place while it's running.
That seems to be so hard that, like we just heard, even to the leaders of research tend to using overlays and overlays of prompt information while a model is running.
But anyways, if a system is pattern-based or rule-based, the art will be to decide which data may be still relevant and which not.
So keeping statistics of the model's use together with well defined queries for that data while it's running to be again be used in realtime to decide what to forget and what not will be inevitable.
I guess that at the moment everybody in the LLM business hopes that rule-based systems suddenly evolve inside the actual models, a kind of an "ahaa!" consciousness, but imho six fingers in the images of the largest image-models that are trained with an unimaginable number of hands and all the errors in today's txt2video models show, that no matter how big you train the models they will still have no real understanding, which means to follow "rules" to make decisions.
And all therefore the biggest challenge, that none of the today's models are able to master, is to _know!_ when you don't know!
And I am pretty sure that it is an art to "forget the right way" ;-)
And there will be no way around that because we simply can't base the world's industry on a few huge models that can only be run in a few data centers of the world.
So models suffer from Dunning Krueger 😢
@@henrischomacker6097 of course I didn't mean NEAT exactly but something akin, something like NEAT, but something that ensures topology preservation in foundation levels at each species (while og. NEAT favors innovation instead of foundation), to help the models accumulate new knowledge and skills without loosing the old ones.
Also didn't mean exactly LoRA but something similar, something that can work as both trainable overlay, and as extension of number of neurons (additional layers or just additional connections). Really what I mean is something between NEAT and LoRA, taht can help evolve additional parts of the network, preserving the original parts and enabling specialisation.
It feels natural that if you have semi independent networks that learned to do different things, and that they work kind of like an neural APIs with input/output neurons being their calls and responses, then training a connector network between them should be easier than training all three networks. This is the idea here.
That's how it works in biology. We have different parts of the brain responsible for different things, specialists in a way. The brain parts are dedicated, meaning you can't learn to use your visual cortex for processing speech.
We need to enable our artificial NN architectures to grow these specialists.
As per forgetting, I mentioned pruning. There could be a stage where trained overlay is joined with background weights and pruned to preserve successful information and reduce the footprint. But I don't think it's a key here, it's more of an optimization.
*STARCRAFT MENTIONED*
We are so back!!!
Non-unitarity == Open-endedness == Obviously only path to real AGI.
Ok if you're so creative. Let's see you make something that no one has thought of yet and is completely unique. I'll wait.
People who do that in a meaningful way do not share their secrets in UA-cam comments. It always ends up being a harmonious blend of elements that have been used before, often since ancient times, but recontextualized to suit a new environmental context particular to that observer
Creativity is using 99% of the environment. So wait you will, because the question is stupid.