THANK YOU SO MUCH, I spent a day working on this because it would show " File "D:\py4e\myenv\lib\site-packages l\callbacks.py", line 8, in from tensorflow.keras import __version__ as KERAS_VERSION ImportError: cannot import name '__version__' from 'tensorflow.keras' (D:\py4e\myenv\lib\site-packages\keras\api\_v2\keras\__init__.py) " with the tensorflow keras and so when I used the specific versions that you stated to use it started working perfectly afterwards. btw just pip install tensorflow==2.12.0, pip install keras-rl2==1.0.5, and pip install gym==0.25.2 for anyone wondering how to install the versions.
right off the bat you start off with extremely useful information .. very nice. you can only do so many request before getting flagged and blocked. Proxies rule , Bots rule, VMs Rule, Tacos Rule
Thank you for this video - would def. be down to dive deeper into the theory of what is happening behind the scene as well to gain a deeper understanding - thank you!
Hey I just tried you code but when I try to run it I have the error : ValueError: Error when checking input: expected flatten_input to have shape (1, 4) but got array with shape (1, 2) Can Someone Help me ?
here is the fix: inside: \venv\Lib\site-packages l\core.py change: action = self.forward(observation) to if first_time: action = self.forward(observation[0]) first_time = False else: action = self.forward(observation) ofc init the first_time = True at the top of the method
why do you set the activation of the output layer to "linear"? wouldnt it make more sense to set it to a sigmoid function, since it is basically a classification?:)
Hey I'm an industrial engineer in master degree and for the year's project a friend and I need to code an IA that is self driving. For me it could be really interesting to have a theoretical apprach for all of this so it could be nice that you make a serie on it. By the way if someone that's reading has some tehoretical resources on the subject I'm intersested in it.
hello , thank you for you amazing work !!i tried implementing it but ikeep getting the message that env is not defined can you help me on how to fix that? iwould be very grateful !
i think there some compatibilities issues with the Adam optimizer, what is the version for tensorflow you've been using in the video, that would be appreciated
If anyone is watching now or anytime in the future and gets a "ValueError: too many values to unpack" error. Try adding another . _ to the part where is says = env.step(action) that helped for me at least
That's a good recommendation. Up to a certain package version, the step function only got four variables back but this is changed to five. Your suggestion works perfectly. Thank you.
NeuralNine shows in his video that you need to install keras-rl2. Once installed, you can access this package by 'import rl' as you can see in the header section of his code. Don't confuse the last part 'rl2' with rl12'....easy to mistake but difficult to spot.
Could you show some example on how to traine chat gpt model opensource stuff for natural language handling (not sure of correct techincal terms and costumize to have a specific conversation) lets say about cars we could give him some extra dataset with conversations about cars or blogs content. You have great content keep it going
Las matemáticas de Q-learning tuve la oportunidad de verlas, y la verdad están raras. Si le sumas las matemáticas de la red neuronal del modelo de Keras. Con eso se complica aún más. Realmente, vería interesante un video próximo orientado a la librería Gym en si. Sus herramientas y sus límites.
i am having this trouble if anybody can help File "C:\Users\alann\AppData\Local\Programs\Python\Python39\lib\site-packages l\util.py", line 86, in __init__ super().__init__(optimizer._name) AttributeError: 'Adam' object has no attribute '_name'
Try changing the import of adam to the legacy version #from tensorflow.keras.optimizers import Adam from tensorflow.keras.optimizers.legacy import Adam
@@mdcandaswhen importing Adam do: from tensorflow.keras.optimizers.legacy import Adam instead of: -from tensorflow.keras.optimizers import Adam- and: agent.compile(Adam(learning_rate=0.001), metrics=["mae"]) instead of: -agent.compile(Adam(lr=0.001), metrics=["mae"])- worked for me
DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`. (Deprecated NumPy 1.24) if not isinstance(terminated, (bool, np.bool8)): Traceback (most recent call last): File "C:\Users shub\MLprojects\ReinforcementLearning\main.py", line 15, in _, reward, done, _ = env.step(action) ValueError: too many values to unpack (expected 4) I am getting error bro. Can yo help me?
Here is a tutorial on how to run the code on a windows 10 machined with python 3.10 and all of the modules ua-cam.com/video/cED5IfKwSHg/v-deo.htmlsi=jWuWHUOm1QEtfIBh
@@ruthvikas bro this was done weeks ago get with the program /s A little sarcasm here because AI developments are moving really fast. That said I still wouldn't expect a youtuber to do that as a side project >
i keep getting this error that i cant fix: Traceback (most recent call last): File "C:\Users\andre\PycharmProjects\pythonProject\main.py", line 14, in _, reward, done, _ = env.step(action) ^^^^^^^^^^^^^^^^ File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\time_limit.py", line 50, in step observation, reward, terminated, truncated, info = self.env.step(action) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\order_enforcing.py", line 37, in step return self.env.step(action) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\env_checker.py", line 37, in step return env_step_passive_checker(self.env, action) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\utils\passive_env_checker.py", line 233, in env_step_passive_checker if not isinstance(terminated, (bool, np.bool8)): ^^^^^^^^ File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages umpy\__init__.py", line 424, in __getattr__ raise AttributeError("module {!r} has no attribute " AttributeError: module 'numpy' has no attribute 'bool8'. Did you mean: 'bool'?
Please help! This is my code: import random import gym import numpy as np from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Flatten from tensorflow.keras.optimizers import Adam from rl.agents import DQNAgent from rl.policy import BoltzmannQPolicy from rl.memory import SequentialMemory env = gym.make('CartPole-v1') states = env.observation_space.shape[0] actions = env.action_space.n model = Sequential() model.add(Flatten(input_shape=(1, states))) model.add(Dense(24, activation='relu')) model.add(Dense(24, activation='relu')) model.add(Dense(actions, activation='linear')) agent = DQNAgent( model=model, memory=SequentialMemory(limit=50000, window_length=1), policy=BoltzmannQPolicy(), nb_actions=actions, nb_steps_warmup=10, target_model_update=0.01 ) agent.compile(Adam(lr=0.001), metrics=["mae"]) agent.fit(env, nb_steps=100000, visualize=False, verbose=1) results = agent.test(env, nb_episodes=10, visualize=True) print(np.mean(results.history['episode_reward'])) env.close() # episodes = 10 # for episode in range(1, episodes+1): # state = env.reset() # done = False # score = 0 # while not done: # action = random.choice([0,1]) # _, reward, done, _ = env.step(action) # score += reward # env.render() # print(f'Episode {episode}, Score: {score}') and this is error message: PS D:\Antek 2023\Python\Path_finding_for_AICAT> & "d:/Antek 2023/Python/Path_finding_for_AICAT/AICAT/Scripts/Activate.ps1" (AICAT) PS D:\Antek 2023\Python\Path_finding_for_AICAT> & "d:/Antek 2023/Python/Path_finding_for_AICAT/AICAT/Scripts/python.exe" "d:/Antek 2023/Python/Path_finding_for_AICAT/gymtest.py" D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\tensorflow\__init__.py:29: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives import distutils as _distutils D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\gym\core.py:317: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future. deprecation( D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\gym\wrappers\step_api_compatibility.py:39: DeprecationWarning: WARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future. deprecation( WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam. 2023-06-11 19:56:08.722599: W tensorflow/c/c_api.cc:300] Operation '{name:'dense_1_1/bias/Assign' id:130 op device:{requested: '', assigned: ''} def:{{{node dense_1_1/bias/Assign}} = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](dense_1_1/bias, dense_1_1/bias/Initializer/zeros)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session. Traceback (most recent call last): File "d:\Antek 2023\Python\Path_finding_for_AICAT\gymtest.py", line 36, in agent.compile(Adam(lr=0.001), metrics=["mae"]) File "D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages l\agents\dqn.py", line 175, in compile optimizer = AdditionalUpdatesOptimizer(optimizer, updates) File "D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages l\util.py", line 86, in __init__ super().__init__(optimizer._name) AttributeError: 'Adam' object has no attribute '_name'. Did you mean: 'name'? (AICAT) PS D:\Antek 2023\Python\Path_finding_for_AICAT>
To make the code work you need specific versions of the libraries:
tensorflow: 2.12.0
gym: 0.25.2
keras-rl2: 1.0.5
Thanks man, i could’t get it working but this fixed every thing
THANK YOU SO MUCH, I spent a day working on this because it would show " File "D:\py4e\myenv\lib\site-packages
l\callbacks.py", line 8, in
from tensorflow.keras import __version__ as KERAS_VERSION
ImportError: cannot import name '__version__' from 'tensorflow.keras' (D:\py4e\myenv\lib\site-packages\keras\api\_v2\keras\__init__.py) "
with the tensorflow keras and so when I used the specific versions that you stated to use it started working perfectly afterwards.
btw just pip install tensorflow==2.12.0, pip install keras-rl2==1.0.5, and pip install gym==0.25.2
for anyone wondering how to install the versions.
you are a lifesaver man
Thank you so much! life saver :D
Is there a reason why the suggested tensorflow version doesn't exist
Math / theory sounds great! I'd also really appreciate practical use case examples 😊
Yes please, I would really like to learn the theory and mathematics behind this. Thanks in advance!
would love to see more detail (math and theory) on how these work
right off the bat you start off with extremely useful information .. very nice. you can only do so many request before getting flagged and blocked. Proxies rule , Bots rule, VMs Rule, Tacos Rule
Hey, nice video, I really enjoyed that one. Could you maybe show us some theory and implementation of SAC or at least actor critic methods of RL?
I would love to watch video about math explanations of NN
Thank you for this video - would def. be down to dive deeper into the theory of what is happening behind the scene as well to gain a deeper understanding - thank you!
Omg! Awesome! I just needed to learn this!
Hey I just tried you code but when I try to run it I have the error :
ValueError: Error when checking input: expected flatten_input to have shape (1, 4) but got array with shape (1, 2)
Can Someone Help me ?
same issue here
here is the fix:
inside: \venv\Lib\site-packages
l\core.py
change:
action = self.forward(observation)
to
if first_time:
action = self.forward(observation[0])
first_time = False
else:
action = self.forward(observation)
ofc init the first_time = True at the top of the method
Yea it would be great if u made a series
Do you have some videos about setting up custom Gym envoriment?
I know its a hassle setting up the environment with the right compatible modules I keep getting errors
why do you set the activation of the output layer to "linear"? wouldnt it make more sense to set it to a sigmoid function, since it is basically a classification?:)
Hey I'm an industrial engineer in master degree and for the year's project a friend and I need to code an IA that is self driving. For me it could be really interesting to have a theoretical apprach for all of this so it could be nice that you make a serie on it. By the way if someone that's reading has some tehoretical resources on the subject I'm intersested in it.
Really satisfied my curiosity,
Book recommendation: "A Primer to the 42 Most commonly used Machine Learning Algorithms (With Code Samples)."
need more advance tutorial OpenAI Gym
Indeed!!
hello , thank you for you amazing work !!i tried implementing it but ikeep getting the message that env is not defined can you help me on how to fix that? iwould be very grateful !
make sure to add:
env=gym.make('CartPole-v1')
before commiting any task on env
11:00 I would like to hear the explaination
Please provide complete tutorials for reinforcement learning basic to advance
Bro pls add the videos on the theory explaining the theory and mathematics behind it.
i have some problems with the tensor flow and the keras can you tell me what version of those 2 libraries r u using and what python version?
i think there some compatibilities issues with the Adam optimizer, what is the version for tensorflow you've been using in the video, that would be appreciated
same
would be greawt if you can include a requirements.txt file in your github repo to save some time during setup
If anyone is watching now or anytime in the future and gets a "ValueError: too many values to unpack" error.
Try adding another . _ to the part where is says = env.step(action)
that helped for me at least
That's a good recommendation. Up to a certain package version, the step function only got four variables back but this is changed to five. Your suggestion works perfectly. Thank you.
How did you manage to install rl? It seems like it doesn't support windows, so pip install rl always fails
NeuralNine shows in his video that you need to install keras-rl2. Once installed, you can access this package by 'import rl' as you can see in the header section of his code. Don't confuse the last part 'rl2' with rl12'....easy to mistake but difficult to spot.
I've trained a new model and it's saved it in my hard disk. How do I load this one to test?
Yes the math but with your approach would be epic in the industry.
Could you show some example on how to traine chat gpt model opensource stuff for natural language handling (not sure of correct techincal terms and costumize to have a specific conversation) lets say about cars we could give him some extra dataset with conversations about cars or blogs content. You have great content keep it going
How do you export the model and be re-use?
Las matemáticas de Q-learning tuve la oportunidad de verlas, y la verdad están raras. Si le sumas las matemáticas de la red neuronal del modelo de Keras. Con eso se complica aún más. Realmente, vería interesante un video próximo orientado a la librería Gym en si. Sus herramientas y sus límites.
i am having this trouble if anybody can help File "C:\Users\alann\AppData\Local\Programs\Python\Python39\lib\site-packages
l\util.py", line 86, in __init__
super().__init__(optimizer._name)
AttributeError: 'Adam' object has no attribute '_name'
Could you finally fix it? Thanks
Try changing the import of adam to the legacy version
#from tensorflow.keras.optimizers import Adam
from tensorflow.keras.optimizers.legacy import Adam
@@mdcandaswhen importing Adam do: from tensorflow.keras.optimizers.legacy import Adam
instead of: -from tensorflow.keras.optimizers import Adam-
and: agent.compile(Adam(learning_rate=0.001), metrics=["mae"])
instead of: -agent.compile(Adam(lr=0.001), metrics=["mae"])-
worked for me
@@mdcandas any fixes?
@@jhevishramphul6331 go to the file the error raises in and change "._name" to ".name"
new to this, why doesn't my python looks like the program he got open?
Tbh, math behind it is not so interesting but video about what type of what is usefull for what type of AI would be really cool
I mean what type of layers is used in specific programe, when do ide used ect.
saved me, thanks
Gym not as part open ai with 2021 year . Now they call gymnasium
what IDE do you use? I thought Visual Studio Code but it`s not, or?
It's pycharm
How to resolve module not found 'gym' error
How would you save the agent in a file
Im not able to install "rl" library, --on windows please help me.. asap
the library to install is 'keras-rl2'
Tensorflow version?
up
error if
from rl.agents import DQNAgent
help
Would love to learn about the theory please
thank you
Damm hell yeah would like theory
More RL videos please
Can you try roulette game? But humans must play. May be with Q learning.
it will not work because no one can predict roulette game (it's random)
DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`. (Deprecated NumPy 1.24)
if not isinstance(terminated, (bool, np.bool8)):
Traceback (most recent call last):
File "C:\Users
shub\MLprojects\ReinforcementLearning\main.py", line 15, in
_, reward, done, _ = env.step(action)
ValueError: too many values to unpack (expected 4)
I am getting error bro.
Can yo help me?
_, reward, done, _, _ = env.step(action)
this is the fix, since step() function returns one more value. (some sort of dictionary as fith return value)
@@hahaok_ thanks buddy 😊
@@hahaok_ would you like to connect with me?
@@magrathea8797 sure, whats your @?
any fixes?
finally a tutorial that does not use google collab notebook..
you doing well, but I need to understand from SCRATCH can you?
can you do a hide and seek in gym ?
Explain the theory too
Nope, won’t install
getting value error
more RL vids!!
Yes please, to math explanations.
Here is a tutorial on how to run the code on a windows 10 machined with python 3.10 and all of the modules ua-cam.com/video/cED5IfKwSHg/v-deo.htmlsi=jWuWHUOm1QEtfIBh
Make a video on how to build own ChatGPT on local computer that is intelligent and has memory!
U really think it can be done on computers ??
U need Terabytes of storage and goddamn data centers for that.
@@ruthvikas bro this was done weeks ago get with the program /s
A little sarcasm here because AI developments are moving really fast.
That said I still wouldn't expect a youtuber to do that as a side project >
Outdated
Nice stolen content from Nicholas Renotte. GJ mate
just because they had the same libraries to import doesnt mean its stolen
i keep getting this error that i cant fix:
Traceback (most recent call last):
File "C:\Users\andre\PycharmProjects\pythonProject\main.py", line 14, in
_, reward, done, _ = env.step(action)
^^^^^^^^^^^^^^^^
File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\time_limit.py", line 50, in step
observation, reward, terminated, truncated, info = self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\order_enforcing.py", line 37, in step
return self.env.step(action)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\wrappers\env_checker.py", line 37, in step
return env_step_passive_checker(self.env, action)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages\gym\utils\passive_env_checker.py", line 233, in env_step_passive_checker
if not isinstance(terminated, (bool, np.bool8)):
^^^^^^^^
File "C:\Users\andre\PycharmProjects\pythonProject\.venv\Lib\site-packages
umpy\__init__.py", line 424, in __getattr__
raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool8'. Did you mean: 'bool'?
Please help! This is my code:
import random
import gym
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.optimizers import Adam
from rl.agents import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory
env = gym.make('CartPole-v1')
states = env.observation_space.shape[0]
actions = env.action_space.n
model = Sequential()
model.add(Flatten(input_shape=(1, states)))
model.add(Dense(24, activation='relu'))
model.add(Dense(24, activation='relu'))
model.add(Dense(actions, activation='linear'))
agent = DQNAgent(
model=model,
memory=SequentialMemory(limit=50000, window_length=1),
policy=BoltzmannQPolicy(),
nb_actions=actions,
nb_steps_warmup=10,
target_model_update=0.01
)
agent.compile(Adam(lr=0.001), metrics=["mae"])
agent.fit(env, nb_steps=100000, visualize=False, verbose=1)
results = agent.test(env, nb_episodes=10, visualize=True)
print(np.mean(results.history['episode_reward']))
env.close()
# episodes = 10
# for episode in range(1, episodes+1):
# state = env.reset()
# done = False
# score = 0
# while not done:
# action = random.choice([0,1])
# _, reward, done, _ = env.step(action)
# score += reward
# env.render()
# print(f'Episode {episode}, Score: {score}')
and this is error message:
PS D:\Antek 2023\Python\Path_finding_for_AICAT> & "d:/Antek 2023/Python/Path_finding_for_AICAT/AICAT/Scripts/Activate.ps1"
(AICAT) PS D:\Antek 2023\Python\Path_finding_for_AICAT> & "d:/Antek 2023/Python/Path_finding_for_AICAT/AICAT/Scripts/python.exe" "d:/Antek 2023/Python/Path_finding_for_AICAT/gymtest.py"
D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\tensorflow\__init__.py:29: DeprecationWarning: The distutils package is
deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
import distutils as _distutils
D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\gym\core.py:317: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
deprecation(
D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages\gym\wrappers\step_api_compatibility.py:39: DeprecationWarning: WARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
deprecation(
WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
2023-06-11 19:56:08.722599: W tensorflow/c/c_api.cc:300] Operation '{name:'dense_1_1/bias/Assign' id:130 op device:{requested: '', assigned: ''} def:{{{node dense_1_1/bias/Assign}} = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](dense_1_1/bias, dense_1_1/bias/Initializer/zeros)}}' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session.
Traceback (most recent call last):
File "d:\Antek 2023\Python\Path_finding_for_AICAT\gymtest.py", line 36, in
agent.compile(Adam(lr=0.001), metrics=["mae"])
File "D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages
l\agents\dqn.py", line 175, in compile
optimizer = AdditionalUpdatesOptimizer(optimizer, updates)
File "D:\Antek 2023\Python\Path_finding_for_AICAT\AICAT\lib\site-packages
l\util.py", line 86, in __init__
super().__init__(optimizer._name)
AttributeError: 'Adam' object has no attribute '_name'. Did you mean: 'name'?
(AICAT) PS D:\Antek 2023\Python\Path_finding_for_AICAT>
from tensorflow.keras.optimizers import Adam
change this to
from tensorflow.keras.optimizers.legacy import Adam
make sure to have the correct version of gym installed.
pip uninstall gym
pip install gym==0.25.2