First I’ve gotta say thank you for making this video. I’ve just read the paper, enjoyed it immensely, and wanted to find an implementation. And bang! here you are, with an in-depth guide on making it work. Also, please, do Never Give Up as well!
Thank you very much for giving an example of how to implement ICM. I'm looking forward to try it myself, and also to make my own further experiments with it. I could perhaps give one tip: as a strong adherent of separation of concerns, I believe it would be better to focus less on other parts of the code, which are less relevant to the core topic, and perhaps just take already written components. I believe that would save a lot of time :)
Thank you very much for such a detailed ICM explanation! I was trying to implement it some time ago but with gym envs like cart pole or lunar lander I found it doesn't perform as expected, probably due to absence of "states encoding" part which I thought is a very important part of the work. I also didn't use a3c for my experiments but rather used a2c. In the end, I found that "Random Network Distillation" algorithm works way better for the same purpose and also free of "TV on the wall" defect like ICM.
Hi Phill, thank you very much for this tutorial ! As I understood the ICM, the Inverse model should be trained together with the encoder NN (which we do not use here) in order to inform the encoder about the parts of the states that controllable by the agent. So if we dont need the encoder here, we also dont need the Inverse model, isnt ? Expecting to some answers, thanks again!
I worked on this last year and ended up developing it, but my full solution never quite held up to my expectations. I always wondered if implemented it correctly, time to verify against yours. Thanks.
Hey, I was working on a genetic NEAT like algorithm but I don’t how to crossover two neural networks with different topology. Is there a procedure to doing this or do you just choose a random one when this happens?
First of all, I have to thank you for making this video. I have made some necessary modification to apply "MountainCar-v0" problem, which is a real "sparse reward" environment. However, it doesn't work. Can you check the code if it is successful in this environment? Thanks!
Hello there Can we save the trained model in this example? Then is it possible to test the model we trained for another environment? How are we going to do? Thus, we can see the success and performance of the trained model more clearly. Could you help?
I'm pretty sure we can compact everything you said with fewer words and fewer domain-specific words. At some points, I can follow you, but the jargon is exploding my face.
![[UA] Team Vitality проти NAVI | IEM Cologne 2024](http://i.ytimg.com/vi/wttO1zVRTC0/mqdefault.jpg)
First I’ve gotta say thank you for making this video. I’ve just read the paper, enjoyed it immensely, and wanted to find an implementation. And bang! here you are, with an in-depth guide on making it work.
Also, please, do Never Give Up as well!
Nice! Some weekend activity, thanks!
Thank you very much for giving an example of how to implement ICM. I'm looking forward to try it myself, and also to make my own further experiments with it. I could perhaps give one tip: as a strong adherent of separation of concerns, I believe it would be better to focus less on other parts of the code, which are less relevant to the core topic, and perhaps just take already written components. I believe that would save a lot of time :)
Thank you very much for such a detailed ICM explanation! I was trying to implement it some time ago but with gym envs like cart pole or lunar lander I found it doesn't perform as expected, probably due to absence of "states encoding" part which I thought is a very important part of the work. I also didn't use a3c for my experiments but rather used a2c. In the end, I found that "Random Network Distillation" algorithm works way better for the same purpose and also free of "TV on the wall" defect like ICM.
This is great video and thanks for sharing !
Thanks for watching
Hi Phill, thank you very much for this tutorial !
As I understood the ICM, the Inverse model should be trained together with the encoder NN (which we do not use here) in order to inform the encoder about the parts of the states that controllable by the agent.
So if we dont need the encoder here, we also dont need the Inverse model, isnt ?
Expecting to some answers, thanks again!
I was thinking the same thing. Did you try removing the inverse model?
I worked on this last year and ended up developing it, but my full solution never quite held up to my expectations. I always wondered if implemented it correctly, time to verify against yours. Thanks.
Let me know if you've any improvements
@@MachineLearningwithPhil you are the best
@@MachineLearningwithPhil can you do ai course beginner to expert
Hey, I was working on a genetic NEAT like algorithm but I don’t how to crossover two neural networks with different topology. Is there a procedure to doing this or do you just choose a random one when this happens?
If you look at the neat paper it explains specifically how to do it. It has to do with keeping track of the topological history I believe.
I was just reading about this!
It's an awesome topic.
How would it perform compare with HER (hindsight experience replay)?
First of all, I have to thank you for making this video. I have made some necessary modification to apply "MountainCar-v0" problem, which is a real "sparse reward" environment. However, it doesn't work. Can you check the code if it is successful in this environment? Thanks!
thanks a lot
Hello there
Can we save the trained model in this example? Then is it possible to test the model we trained for another environment? How are we going to do? Thus, we can see the success and performance of the trained model more clearly. Could you help?
I'm pretty sure we can compact everything you said with fewer words and fewer domain-specific words. At some points, I can follow you, but the jargon is exploding my face.
feature vector, featur map. We have so many terms.
This seems really similar to the entropy of SAC.
HI, did anyone run this in google colab? is there any problem with spawning