Yoshua Bengio - Towards Neural Nets for Conscious Processing and Causal Reasoning
Вставка
- Опубліковано 10 жов 2022
- A major gap between current state-of-the-art deep learning and human generalization abilities regards out-of-distribution scenarios, where our best AI systems suffer a significant drop in accuracy, compared with us. Interestingly, when humans are confronted with new or surprising situations, they tend to switch from system-1 types of behaviors relying on quick habitual responses to system-2 types of cognitive processes, which are slower, require conscious attention and generate verbalizable thoughts. This form of computation seems to rely on a modular decomposition of knowledge into pieces that can be recombined in novel ways on-the-fly using attention mechanisms to sequentially generate these pieces forming the elements of our thoughts, suggesting that this provides a form of more powerful systematic generalization than the system-1 habitual responses. In this presentation, we will describe a research plan and inductive biases for introducing this type of system-2 knowledge representation, inference and learning in neural networks, as well as early results on the neural machinery we propose for this, called GFlowNets.
Recognized worldwide as one of the leading experts in artificial intelligence, Yoshua Bengio is most known for his pioneering work in deep learning, earning him the 2018 A.M. Turing Award, “the Nobel Prize of Computing,” with Geoffrey Hinton and Yann LeCun. In 2019, he was awarded the prestigious Killam Prize and in 2021, became the second most cited computer scientist in the world. He is a Fellow of both the Royal Society of London and Canada and Officer of the Order of Canada. - Наука та технологія
This guy is a gift. Also, he could make it big in ASMR! Pleasant voice. 😊
"There's no executive here, it's just a piece of network that is trained" ... Really important words!
Such an amazing talk, always a pleasure to hear the intuition of an AI veteran
Awesome talk, thank you very much!
mind blowing
Both System 1 and System 2 are conscious ; the difference is that System 1 is non-declarative - i.e. it occurs out of the scope of language and thought. Consciousness is not thought - it's the sense of reality, like one has a sense of sound or sight. The sense of reality is central to cognition because it establishes the spatio-temporal frame of reference we call 'now'. Machine don't have a 'now', their processes happen in what amounts to an eternal present because computer memory is a spatial embedding, whereas the living creature's memory is a functional embedding.
Can you please share a link for the slides?
‘Thought’, ‘mind’, ‘intelligence’ & ‘consciousness’ are all information-related phenomena and it is not difficult to show that one of the principal (& completely inexcusable) reasons why we have not so far come to any good & proper - nor fully verifiable - understanding of these otherwise greatly sought-after yet still highly mysterious phenomena is due in great part to the simple fact that we do not presently also have a good & proper - that is, we do not presently also have a clear & fully verifiable - understanding/science of ‘information’ itself.
Although I have personally had the (altogether dubious) fortune of having been able to figure out ‘information’s’ correct (& fully verifiable) ontological identity, and although I’m not going to divulge its formalistic definition here in this UA-cam comment (without which formalistic definition it is not possible to establish a full & accurate science of the phenomenon, but with it it is) nevertheless I can assure you that with it in hand - that is, with ‘information’s’ correct ontological identity within one’s investigative arsenal - the exercise of determining the ontological identities of all of the other directly information-related phenomena such as ‘thought’, ‘mind’, ‘intelligence’ & ‘consciousness’ (to far less than exhaust the list) becomes one of no great difficulty.
Obversely, once ‘information’s’ correct (verifiably correct) ontological identity is properly recognised, not only do the correct ontological identities of all of its most closely related cousins (as above) become nicely elucidated, but so also does the woeful incorrectness - the hopeless & excruciatingly embarrassing incorrectness - of all of information’s current imposters, along with ‘consciousness’s’ own struggling wanna-bees too. So much so that it becomes fully & quite verifiably obvious that (i) digits are not information, that (ii) thinking is not a computable phenomenon, & (iii) that computers do not because they cannot, think. Let alone do so either intelligently or consciously. Even less so with full cognitive self-conscious awareness.
And (iv) our own nature-built, real live flesh & blood, internal thinking machine is not a computer.
Although it pertains to millions & millions of different things - things which we ourselves call colour, sound, taste, odour, texture, temperature, balance, love, hate, joy, happiness, the feeling of the need to micturate & defecate, vomit, sneeze, cough, choke etc, etc, etc in its generic form ‘information’ turns out to be a completely knowable, identifiable, measurable, quantifiable phenomenon & it is also simple. And our universe is chockablock full of it. It’s also something staring at you right in your face. Hiding in plain sight.
Knowing information’s correct ontological identity allows any kind & amount of it to be both identified, & to be traced & tracked, when- & wherever any of it resides & moves, here in our universe, including any of it being operated on inside our own internal, nature-given, flesh & blood thinking machine.
Performing this identifying//tracing-&-tracking exercise on any of the information that eventually makes it into our own conscious awareness is not only a fully doable task, but it is the one which readily highlights the exact ontological identity of all of our mental phenomena - including ‘thought’, ‘mind’, ‘intelligence’ & ‘consciousness’…
Can we also get the slides?
1:04:09 so interventions are verbs and therefore llm have built in structure, i.e. grammar, that map to bayesian reasoning? thus ilya is right in that if llm can generate correct output iff it understands and reason about, to a certain extent, strucutre of the world. another word, llm is enough to achieve human level intelligence.
💓
Good boy, Yoshua 😏
Just remember that the Turing Award should have also gone to Tomáš Mikolov for hiw word2vec for Google and more