Thank you so much. Honestly for this one. I spent a lot of time and I was quite skeptical if I can explain well. Thanks for sharing your feedback and support 🙏🏾
Holy shit we might be able to do mind uploading with techniques like this. If you can recover your architecture by doing inference over all your skills. Then write down and encode all your memories into a memory system. Then use color blind tests and hearing tests to encode your qualia parameters. Then run inference tests over all your motor outputs in different eye tracking vr inference tasks. You could theoretically create a near perfect copy of your architecture. Damn i might start a cult lol
Great job! Thank you so much! These paper videos are the your most valuable contributions at least in my opinion. You're not afraid to go into the details like 2 minute papers (his videos have their own type of value, bringing attention from the broader public to AI research)
The hidden layer dimensions of Ada and Babbage and also the undisclosed dimensions of GPT 3.5 and also the weight matrices. All these were previously unknown
I think the definition of a black box model would be that we do not have access to the model's weights and its architectural information, etc., instead of not being able to understand the weights of the model because I think this paper falls into the category of adversarial machine learning. To my limited knowledge, there are 4 categories of attacks in AML, which are: 1) Evasion attack 2) Data poisoning attack 3) Extraction attack 4) Inference attack And this particular attack falls into the category of an extraction-based attack, which means we are trying to extract information about the model.
There is, it is called SpikeGPT, but it had not got any traction yet, because spiking neural networks are very specific to implement and many of our technollogies don't fit well with them. I think that if your plan is to just translate the normal neural networks trained with backpropagation into spiking neural networks (or just train them directly using surrogate gradients), then you are already losing all the supposed benefits of a different architecture like SNN's, so it is kinda pointless, except for the better cost and possibly the scalability on specialized hardware.
That doesn't have anything to do with AI black box effect. Black box means that we don't understand what happens INSIDE the AI model, how they think or how they make their decisions. The thing that would make it not a black box is interpretability research. Im sorry, but thats clickbat.
I quoted that from the paper. And feel it's a great achievement in terms of knowing the inner workings of DNN just from the API call. I'm sorry if you feel it's a clickbait!
@@1littlecoder yeah, but its not what black box usually means, much better term would be closed-source, the reason black box is called that is because NO ONE knows whats inside. not because only some people know whats inside. For instance, Windows is a closed-source system, but people know what goes in inside of it, while AI can be open-source but people would still have no idea what exactly happens inside after the input and before the output, that's why AI interpretability field exists.
@@TheManinBlack9054 tbh, even before LLMs were popular, the black box model has been a common tech term to use. We used to call Random forest and Xgboost as black box models because they were not easily interpretable unlike linear models
do u understand that the same solution also work for interpretability? AI model understood and decode how another AI mode work, wit some effort u can create one that translate that for us to understand.
You make research papers sound so easy. Thank you for explaining them in simple terms.
Thank you so much. Honestly for this one. I spent a lot of time and I was quite skeptical if I can explain well. Thanks for sharing your feedback and support 🙏🏾
You have real talent in exlaining hard scientific papers easy. Thanks from Poland!
Thanks so much for the kind words. I'm glad this one worked out well
@@1littlecoder you got new subscriber, I binge watch your videos in the evening. Great job mate
@@ineffige Thanks mate!
Holy shit we might be able to do mind uploading with techniques like this. If you can recover your architecture by doing inference over all your skills. Then write down and encode all your memories into a memory system. Then use color blind tests and hearing tests to encode your qualia parameters. Then run inference tests over all your motor outputs in different eye tracking vr inference tasks. You could theoretically create a near perfect copy of your architecture. Damn i might start a cult lol
Hehe 😂
Love it
Isn't that what Facebook does :) ?
Great explanation for such a complex paper ❤
Thanks very much. Glad it turned out well
Great job! Thank you so much! These paper videos are the your most valuable contributions at least in my opinion. You're not afraid to go into the details like 2 minute papers (his videos have their own type of value, bringing attention from the broader public to AI research)
Wow, thank you! I appreciate your kind words for the efforts
Imagine Elon musk stealing the source from OpenAIs models and open sourcing them XD
Elon RobinHood Musk
Giving people's intelligence back to people 😅
First he will do some publicity stunt
@@cig_in_mouth3786then he will fail to deliver what he promised.
Finally he’ll wait another few weeks and promise something new.
I think it'll probably be the oposite. CloseAI and ShalowMind stealing from Claude or Mistral
Does this research reveal anything about OpenAI's models that was previously unknown?
The hidden layer dimensions of Ada and Babbage and also the undisclosed dimensions of GPT 3.5 and also the weight matrices. All these were previously unknown
Nicely covered. Thank you
Damn :-) I would've leaked everything to the dark web. Would love to force the 'open' part in OpenAI..
I wonder if we can use something similar to get the dimension of our simulation (Wolfram) ruliad.
Nice research
Kudos to the researchers!
I am looking for Devin AI software engineer
Sad this didn't come from someone a little more gray hat-ish that would actually release all the details.
I think the definition of a black box model would be that we do not have access to the model's weights and its architectural information, etc., instead of not being able to understand the weights of the model because I think this paper falls into the category of adversarial machine learning. To my limited knowledge, there are 4 categories of attacks in AML, which are:
1) Evasion attack
2) Data poisoning attack
3) Extraction attack
4) Inference attack
And this particular attack falls into the category of an extraction-based attack, which means we are trying to extract information about the model.
you just send an empty prompt “ “
Stealing is all you need.
Probably someone else will work on this paper title 🤣
lmao
Why there is no LLMs based on spiking neural networks?
Given that RNNs are making a comeback. we never know!
There is, it is called SpikeGPT, but it had not got any traction yet, because spiking neural networks are very specific to implement and many of our technollogies don't fit well with them. I think that if your plan is to just translate the normal neural networks trained with backpropagation into spiking neural networks (or just train them directly using surrogate gradients), then you are already losing all the supposed benefits of a different architecture like SNN's, so it is kinda pointless, except for the better cost and possibly the scalability on specialized hardware.
What is 1- bit LLM
ua-cam.com/video/Gtf3CxIRiPk/v-deo.html
That doesn't have anything to do with AI black box effect. Black box means that we don't understand what happens INSIDE the AI model, how they think or how they make their decisions. The thing that would make it not a black box is interpretability research. Im sorry, but thats clickbat.
I quoted that from the paper. And feel it's a great achievement in terms of knowing the inner workings of DNN just from the API call. I'm sorry if you feel it's a clickbait!
@@1littlecoder yeah, but its not what black box usually means, much better term would be closed-source, the reason black box is called that is because NO ONE knows whats inside. not because only some people know whats inside. For instance, Windows is a closed-source system, but people know what goes in inside of it, while AI can be open-source but people would still have no idea what exactly happens inside after the input and before the output, that's why AI interpretability field exists.
@@TheManinBlack9054 tbh, even before LLMs were popular, the black box model has been a common tech term to use. We used to call Random forest and Xgboost as black box models because they were not easily interpretable unlike linear models
do u understand that the same solution also work for interpretability? AI model understood and decode how another AI mode work, wit some effort u can create one that translate that for us to understand.