Відео

You're welcome! We want this to benefit the world. :)

Hey @SkillsToLearn, currently we are in research mode, trying out new approaches, so we haven't been focused on the efficiency of the CMP wire transfer yet.

That's the plan!

Thanks! We do have some written tutorials in our documentation here: thousandbrainsproject.readme.io/docs/tutorials and just posted a code walkthrough here: ua-cam.com/video/x0e5SBY2nu8/v-deo.html We are also organizing a meetup on December 4th where we present about the whole project and you can ask questions: www.meetup.com/thousand-brains-project/events/304416178/?eventOrigin=group_upcoming_events We would be happy to have you join! More interactive workshops are in the planning :)

@@thousandbrainsproject Thanks I'll check them our asap . Thanks again

Glad you enjoyed it!

Thanks!

Well put!

Thanks for the tip! We used tsne since it was an easy out-of-the-box sklearn functionality and we just wanted to get a quick impression of how the representations cluster. If we take a more detailed look at this we will definitely give PaCMAP a try :)

Let us know how it goes!

You're welcome! We think it's critical that every aspect of this is open-source, not just the models.

Thank you! A lot of the underpinning technology of HTM will almost certainly make it in to the Thousand Brains Project.

Thank you! And today it's live - github.com/thousandbrainsproject/tbp.monty

Please do, we'd love your participation and feedback.

Hi @medwards1086, it's an interesting question - do you have a specific part of the video you're referring too?

Good question! While we agree with the basic premise that building spatial intelligence is the way to go, I think we have very different ways of going about this. We don’t think that to acquire true spatial intelligence we can simply train a large neural network on a new, more spatially focused, dataset. To get to the capabilities and efficiency of the human brain, we have to build fundamentally different models that use reference frames to model the spatial structure of the world. There isn’t a lot of detailed information out there on the approach the World Labs company will take but from what I can gather, it sounds like it is not the through and through sensorimotor learning the brain is doing. The conventional way of doing vision based interaction with the world is using an ANN for several layers of visual processing and then using the result of this to decide the next action/train a policy. However, in the brain, motor input and output is found everywhere, not just in the motor cortex. Every column receives movement information and sends outputs to subcortical motor areas. I don’t want to get into too many details here but just for another flavor of the differences: On their website they state that “Initially we will focus on generating 3D worlds without limits”. While this may sound cool, it is definitely not what our brains evolved to do. In fact, the only output of the brain are motor outputs. We are still watching Fei Fei Li’s and her team’s work with anticipation and think that there are already exciting intersections with her work (such as this cool behavior benchmark which we would like to use to train and test our system on at some point behavior.stanford.edu/ ).

@vivianeclay Thank you very much for your detailed response Viviane, things are happening so fast today that its a challenge just to try and keep up, I feel very fortunate that so many scientists are sharing the work their doing in an open media environment for the curious to try and keep up, have a wonderful day, and congratulations on the continued progress your team has been able to achieve, peace

We think it's a new era too. More to come.

@FalcoOnline we appreciate your support.

Thanks!

Yep, the audio is an issue. You can turn on comments which helps as they're manually curated.

:)

@@thousandbrainsproject Real intelligence is coming, come on！！👍👍

We love how interested and engaged the community is already. So much more to come!

It’s very on topic! We believe that the thalamus is responsible for translating the sensors location in space with reference to the body to a reference frame in the object’s space. The AI Bus (now called Cortical Messaging Protocol or CMP) sends information between columns in the reference frame of the body whereas internally the columns need to look at poses in the reference frame of their internal model. While we don’t need a thalamus to do that in Monty, we believe that is where the translation from one “coordinate” system to another is happening in the human brain.

@@thousandbrainsproject That function of the thalamus of translating space coordinates is inspired by the thalamus storing long term spatial memories using the place cells ? Is that what you think gives it the ability of doing all thes spation translations?

Currently, our implementation encapsulates both the where and what concepts in one learning module. Neurologically, they are separated in the human cortex but we are not 100% sure if we'll need to implement them this way. Stay tuned!

Haha yes, we do have Chapter II of the Legend of Monty :)

Thank you

Good question! This system is different than deep learning approaches and because this is a completely new form of AI there aren't a lot of parallels like gradient descent or other optimizations used in that field. Monty moves its sensors to explore an object and as it explores an object it builds a representation in the sensor's attached learning module. The common communication protocol (now called Cortical Messaging Protocol, CMP) has one goal in common with a joint embedding space, namely to allow multimodal transfer, but it also allows learning modules to vote together to form consensus about what is being sensed (as well as a host of other functionality). If you like, you can also ask more questions about the CMP over on our discourse forum - thousandbrains.discourse.group

As I understand it, it is associating patterns of displacements with objects based on evidence. As she mentioned in the video, I think they are still doing that supervised, but at some point it will be unsupervised. Right now, or back in 2022, they were focusing on the initial object recognition part of it. There is no gradient descent. Communication protocol is the voting protocol between mini-columns (each of which covers a little window on the object). They all have their own perception of the object based on evidence and they vote. The highest vote wins and that's perceived to be the object. At least that's what I understood. I may be wrong.

Sorry, I wrote a response but didn't post it! Good question! This system is very different than deep learning approaches and because this is a completely new form of AI there aren't a lot of parallels like gradient descent or other optimizations used in that field. Monty moves its sensors to explore an object and as it explores an object it builds a representation in the sensor's attached learning module. The common communication protocol (now called Cortical Messaging Protocol, CMP) has one goal in common with a joint embedding space, namely to allow multimodal transfer, but it also allows learning modules to vote together to form consensus about what is being sensed (as well as a host of other functionality). You can also ask more questions about the CMP over on our discourse forum - thousandbrains.discourse.group

Glad you liked it!

Good question! These are shorthands for the representations. Sensor to Object, Object to Body, Sensor to Body

There is an overview of it on the Numenta site, and there is a link to the full paper on that page too. www.numenta.com/resources/research-publications/papers/a-theory-of-how-columns-in-the-neocortex-enable-learning-the-structure-of-the-world/

We are glad you're finding it interesting! So much more to come.

Yeah, we're sorry about that, the recordings are quite old. We did transcribe the audio so the captions should help.

@next_phase yes it's an interesting question to examine how these relate and differ. At a high-level, both of these rely on hierarchy, however there are several key differences. In CNNs, and deep-learning systems more generally, there is often a lack of “object-centric” representations, which is to say that when processing a scene with many objects, the properties of these tend to be mixed up with one another. This is in contrast to humans, where we understand the world as being composed of discrete objects with a degree of permanence, and where these objects have the ability to interact with one another - an understanding that emerges at a very young age. Furthermore, any given object in our brain is represented spatially, where the shape of the object - i.e. the relative arrangement of features - is far more important than low-level details like a texture that might be present. Again, this is different from how CNNs and other deep-learning systems learn to represent objects. So while there is hierarchy in both CNNs and the human visual system, the former can be thought of as more of a bank of filters that detect things like textures and other correlations between input pixels and text. We believe that in the brain however, every level of the hierarchy is representing discrete objects with their own structure and associated motor policies. These can be rapidly composed and recombined, enabling a wide range of representations and behaviors to emerge. Hope that's helpful, let me know if I can clarify anything.

Thank you!

Thank you!

I just added an image of the whiteboards over in the forum post associated with this video - thousandbrains.discourse.group/t/2021-11-intro-to-the-ai-bus/96

Good point! We have posted pictures of the whiteboard on our discourse channel: thousandbrains.discourse.group/t/2021-11-intro-to-the-ai-bus/96 A nice member of the community, moe, also transcribed the whiteboard pictures into text!

Good feedback! We have a lot of back-catalog to publish, but I promise our production values will increase. :)

​@@thousandbrainsproject 😅 Thank you so much, in my view its very important content.

So much more to come!

Thanks!

Thanks 🤗