This is the easily the most enjoyable series for me as well. Been a while since I woke up early b/c I wanted to get out of bed and see how a model was doing :P
I know this is old but, as someone else mentioned, if you did if 'A' in keys: output[0] = 1 if 'D' in keys: output[2] = 1 if 'W' in keys: output[1] = 1 instead of doing the if, elif, else, then you would have the combinations of those buttons too. Ex: You pressed WA then the output would be [1,1,0].
Enjoyed watching this series up untill now so before watching this video i made a python script that could play the chrome dinosaur run game (although it was an older version without the pterodactyls) and i quickly went from getting a score of around 700 to some code that managed to get a score of 10870. Also, it seems quite new so it may not have been available at the time of recording but there is a python module calle mss which allowed me to get screen grabs at 60 FPS (if i drew the screen grabs back to the screen using cv2.imshow i got around 25-30 FPS)
Hm. Looking back at my driving, it seems NN will pick up bad manners: when I was driving uphill on that same road segment as you are, first several attempts near the cottage at the start I drove at the center of the lane or on the outer side of the road, which is very slow(thus providing a lot of data points). During the last attempts I've figured out that coming near center of the road, to the left part of the lane is much quicker, thus providing fewer data points. My future self should probably record data it to several files so data can be discarded selectively. Also I've just used `if 'Q' in keys: break` to stop recording instead of alt-tab, ctrl-c.
Hey, I am working for similar project but in FIFA, but the problem is it cannot detect the key pressed(keyboard) when the window if focused on FIFA ,but detects the key if it is focused on Pycharm or the opencv window. Any help regarding that issue?
@@bhairajagautam3505 pypi.org/project/PyDirectInput/ Try the above link once. I had one game which did not react to pyautogui but gave response to pydirectinput.
how about the data with just lines and coordinates. there are many things trees, safety fences, grass etc. that are redundant for line following. I also think that when the data consist of just lines, weights of miscalculation lines in the network will be easily decreased. I mean they are finally ineffective to the output. thanks.
Great video man I had a question though. For getkey.py is there a way to add mouse click as well to the character list so that even that is given as a key input ?
How can I use this pywin32api on linux or raspbian. Can pywin32api work for linux or I need some library like pylin32api. Not sure if that's a library. Any library with same commands for linux.
wouldn't it be better to train the c-nn by feeding it the lines got from the Hough lines algorithm? i mean why the whole image if the data is too large for the nn?
Can anybody help me with recording key presses.I have tried the code which was there on sentdex Github(grabkeys.py) but the pywin32 module does not work.I have also tried msvcrt.getch() even that does not work.Using python 3.5.1 with Pycharm Community Edition.
I'm trying to apply what you're doing in the GTA videos to the browser game agario. Do you have a script that would record mouse movement instead of key presses?
Help needed!!! I tried the same on Need for Speed, and I got an issue. The controls of the game use arrow keys and not W, A, S, D. Right now, I changed the controls to WASD but I'm not able to play it good as I am very much used to the arrow keys. Can someone help me in logging the arrow keys instead of WASD, pls... Thanks!
I hope this isn't too late but I believe you would just change the WASD variables in directkeys to the arrow keys (meaning you would also have to change the variables when you import it). According to www.gamespp.com/directx/directInputKeyboardScanCodes.html, the key codes are: Up arrow: 0xC8 Left arrow: 0xCB Right arrow: 0xCD Down arrow: 0xD0
'training_file.npy' stops storing data after some point of time. Like if length of training_data hit 1000... And after 15-20 minutes of playing the game, 'training_file.npy' shows 0 bytes. Any suggestions?
I am trying to implement this in another game called Euro Truck Simulator, but the keystrokes do not register in mine and all I get is a list of 0s for each image captured. Anyone has any ideas on why is this happening. It works fine outside of the game when I test it with say a text editor but it doesn't work in game.
those who are on mac , for image grabbing you can use mss library , it can record with high fps , edit your code a lil , its launched in 2019 link for eg : python-mss.readthedocs.io/examples.html
Is there a way to record mouse movement from a game in python? win32api GetCursorPosition returns x,y coordinates of the mouse, and in game if you move you mouse infinitely to the left and up, it will rotate the camera, but cursor will just get to 0,0 and stay there, and that's what pyautogui and GetCursorPosition return. In games it's more like offset. So, is there a way to record this data? I actually don't think that it's a good idea to try and let neural network learn how to control a mouse in a 3d shooter game, lol, but who knows! For science! I wanna try to play for a few hours, record both keyboard and mouse actions, and see if CNN can figure out how to play a game :D The only thing stopping me right now is how to get mouse movement.
Nope, I did not. And from what I think now, it will probably require a separate neural network to control the mouse, or at least a fair amount of modifications to current one
Hi sentdex maybe this video is not right to ask this question. I want to count all orange from this image : goo.gl/ViGTJZ, I habe been triying it with Python and Opencv but I have not could do it. Could you give some advice? It's possible do it with matching learning, namely, training some algoritm with image? Thanks in adavance.
peter parker I have the same doubt. Right now, the only solution I found is to actually change your in-game controls from arrow keys to some other alphabet...
Love this project. I'm working on my own AI for Live For Speed auto pilot racer running on Linux (Ubuntu) via wine. For those looking to the linux analog of windows key press query I used this: rosettacode.org/wiki/Keyboard_input/Keypress_check ``` import keyboard #Using module keyboard def get_keys_output(): output = [0, 0, 0, 0] # up, down, left, right if keyboard.is_pressed('up'): #if key 'up' is pressed.You can use right,left,up,down and others output[0] = 1 if keyboard.is_pressed('down'): output[1] = 1 if keyboard.is_pressed('left'): output[2] = 1 if keyboard.is_pressed('right'): output[3] = 1 return output ``` NOTE: You'll need to run your training python code as sudo. This is required by linux. Cheers to all!
The taxi's code uses the in game engine. It doesn't actually know how to drive, they just have coded rules that knows inherently where the roads are, where to go, how to act...etc. The in-game taxis, for example, could never leave GTA V's city and drive anywhere else.
grab screen is not reading any screen data just it is showing a grey window, please help me i am using visual studio and i.ve tried it in idle but same result.
"Gotta keep that youtube ad revenue" cracked me up xD great series btw
Love the series! This is keeping me happy and satisfied with the progress of mankind! :)
All your videos are amazing, but these are probably the most entertaining!
This is the easily the most enjoyable series for me as well. Been a while since I woke up early b/c I wanted to get out of bed and see how a model was doing :P
I know this is old but, as someone else mentioned, if you did
if 'A' in keys:
output[0] = 1
if 'D' in keys:
output[2] = 1
if 'W' in keys:
output[1] = 1
instead of doing the if, elif, else, then you would have the combinations of those buttons too. Ex: You pressed WA then the output would be [1,1,0].
Yeah! I have the same thought.
Yes but I don't think a classifier model can predict two values at one time
@@datarachit agreed
just in case nobody told you today: you are GREAT!
+sentdex Great guy. That came a little earlier then expected. Thumbs up!
Happy to beat your expectations!
Loved this series so much
It makes me irrationally angry that you resized 800x640 to 80x60 and not 80x64
lmao :)
Excuse the late comment; only just found this but it's an 800x600 screen that's offset by 40 to hide the top bar, no?
Bakonoe yes
Amazing series. I love the concept. Subbed!
Thanks for the support!
Enjoyed watching this series up untill now so before watching this video i made a python script that could play the chrome dinosaur run game (although it was an older version without the pterodactyls) and i quickly went from getting a score of around 700 to some code that managed to get a score of 10870. Also, it seems quite new so it may not have been available at the time of recording but there is a python module calle mss which allowed me to get screen grabs at 60 FPS (if i drew the screen grabs back to the screen using cv2.imshow i got around 25-30 FPS)
aww yeah I've been waiting for this! :)
Hm. Looking back at my driving, it seems NN will pick up bad manners: when I was driving uphill on that same road segment as you are, first several attempts near the cottage at the start I drove at the center of the lane or on the outer side of the road, which is very slow(thus providing a lot of data points). During the last attempts I've figured out that coming near center of the road, to the left part of the lane is much quicker, thus providing fewer data points. My future self should probably record data it to several files so data can be discarded selectively.
Also I've just used `if 'Q' in keys: break` to stop recording instead of alt-tab, ctrl-c.
Hey, I am working for similar project but in FIFA, but the problem is it cannot detect the key pressed(keyboard) when the window if focused on FIFA ,but detects the key if it is focused on Pycharm or the opencv window. Any help regarding that issue?
I've same issue, have you figured out??
@@m_squre No workaround. I had to give up the project. If you ever find it out, please reply me to this thread
@@bhairajagautam3505 pypi.org/project/PyDirectInput/
Try the above link once. I had one game which did not react to pyautogui but gave response to pydirectinput.
Great little series :)
I use np.vstack to append numpy arrays
pywin32 module is now available to download via pip under an alias name pypiwin32. So, just run pip install pypiwin32
My network predicts only zeros. Any idea what could be the possible problem?
how about the data with just lines and coordinates. there are many things trees, safety fences, grass etc. that are redundant for line following. I also think that when the data consist of just lines, weights of miscalculation lines in the network will be easily decreased. I mean they are finally ineffective to the output. thanks.
Awesome video series !
You shouldn't have used elseif in the keys output function, since key can be combined (both A and W are pressed for example)
That part made me cringe so hard. :(
Great video man I had a question though. For getkey.py is there a way to add mouse click as well to the character list so that even that is given as a key input ?
I love you sentdex you're my boi
Is there a way to emulate an analogue input (i.e the triggers and sticks on a x360 pad) so the AI can make more subtle turns and accelerations?
+Oliver Hilton not easily enough for me to have figured out how to implement it. If you figure it out, let me know!
Oliver Hilton I saw a code on github which shows how to use the controller and read the input data. So I think you can easily find it.
@@sentdex There is a way to link it to Robot operating System and do that. Its called ROS bridge.
How can I use this pywin32api on linux or raspbian. Can pywin32api work for linux or I need some library like pylin32api. Not sure if that's a library. Any library with same commands for linux.
I am getting an error around 50,000 frames... "EOFError: Ran out of input". Any fixes?
wouldn't it be better to train the c-nn by feeding it the lines got from the Hough lines algorithm? i mean why the whole image if the data is too large for the nn?
Can anybody help me with recording key presses.I have tried the code which was there on sentdex Github(grabkeys.py) but the pywin32 module does not work.I have also tried msvcrt.getch() even that does not work.Using python 3.5.1 with Pycharm Community Edition.
12:47 best part
I'm trying to apply what you're doing in the GTA videos to the browser game agario. Do you have a script that would record mouse movement instead of key presses?
I do not currently, BUT, if YOU find one, please share with me :)
Absolutely. Thanks for the quality content. :)
this is so good and interesting!
Hi sentdex, why did you remove all the line detection code, am I missing something? Or was it because you found a better approach?
Neural network is being added and it will detect the lines I guess
Couldn't you have the script pause when you press esc by using key press?
Please put some kind of vibration dampening on your mic.. The vibration coming though the mic from your hard key presses is crazy.. Thanks
Masterism88 I think it sounds since, gives the video a nice ambient sound.
Do you have the download of all the new Python programming you've done?
Help needed!!!
I tried the same on Need for Speed, and I got an issue. The controls of the game use arrow keys and not W, A, S, D. Right now, I changed the controls to WASD but I'm not able to play it good as I am very much used to the arrow keys. Can someone help me in logging the arrow keys instead of WASD, pls...
Thanks!
I hope this isn't too late but I believe you would just change the WASD variables in directkeys to the arrow keys (meaning you would also have to change the variables when you import it). According to www.gamespp.com/directx/directInputKeyboardScanCodes.html, the key codes are:
Up arrow: 0xC8
Left arrow: 0xCB
Right arrow: 0xCD
Down arrow: 0xD0
'training_file.npy' stops storing data after some point of time. Like if length of training_data hit 1000... And after 15-20 minutes of playing the game, 'training_file.npy' shows 0 bytes. Any suggestions?
How to fix Memory error when writint to training data file?
I know I'm a little late, but were you able to figure it out?
I am trying to implement this in another game called Euro Truck Simulator, but the keystrokes do not register in mine and all I get is a list of 0s for each image captured. Anyone has any ideas on why is this happening. It works fine outside of the game when I test it with say a text editor but it doesn't work in game.
Anyone know of a way to do the same with a controller?
It would be easier to drive the vehicle which means better training data.
those who are on mac , for image grabbing you can use mss library , it can record with high fps , edit your code a lil , its launched in 2019
link for eg : python-mss.readthedocs.io/examples.html
You wrote right = right[:len(right)] instead of right = right[:len(forwards)]
But it's fixed on Github, so it's fine...
Can someone tell what is key name for space, up, down, ctl, etc in win32api
What if I'm using Ubuntu. How do I deal with pywin32 ?
Is there a way to record mouse movement from a game in python? win32api GetCursorPosition returns x,y coordinates of the mouse, and in game if you move you mouse infinitely to the left and up, it will rotate the camera, but cursor will just get to 0,0 and stay there, and that's what pyautogui and GetCursorPosition return. In games it's more like offset.
So, is there a way to record this data? I actually don't think that it's a good idea to try and let neural network learn how to control a mouse in a 3d shooter game, lol, but who knows! For science! I wanna try to play for a few hours, record both keyboard and mouse actions, and see if CNN can figure out how to play a game :D The only thing stopping me right now is how to get mouse movement.
Nope, I did not. And from what I think now, it will probably require a separate neural network to control the mouse, or at least a fair amount of modifications to current one
hello sir ,,my training data has [0,1,0] only i m using vice city as Game
did u got a fix for that? if yes then please let me know
Yash Shrivastava create training data without leaving The pressed if w then without leaving w press a ....in the same way generate the data
@@aakashshukla7501 sorry can u please elaborate... it's urgent please
can I add mouse output !!!!
nice!
Hi sentdex maybe this video is not right to ask this question. I want to count all orange from this image : goo.gl/ViGTJZ, I habe been triying it with Python and Opencv but I have not could do it. Could you give some advice? It's possible do it with matching learning, namely, training some algoritm with image? Thanks in adavance.
how to detect left arrow ??
peter parker I have the same doubt. Right now, the only solution I found is to actually change your in-game controls from arrow keys to some other alphabet...
genial!
strating fresh.... lol
.🤙
I noticed that I swear a lot after playing GTA 5. I noticed the same sss... thing back when GTA San Andreas came out.
you look like Snowden!
>keylogger.py
Fixed that for you.
what does it do ?
Unregistered Hypercam2
Love this project. I'm working on my own AI for Live For Speed auto pilot racer running on Linux (Ubuntu) via wine.
For those looking to the linux analog of windows key press query I used this: rosettacode.org/wiki/Keyboard_input/Keypress_check
```
import keyboard #Using module keyboard
def get_keys_output():
output = [0, 0, 0, 0] # up, down, left, right
if keyboard.is_pressed('up'): #if key 'up' is pressed.You can use right,left,up,down and others
output[0] = 1
if keyboard.is_pressed('down'):
output[1] = 1
if keyboard.is_pressed('left'):
output[2] = 1
if keyboard.is_pressed('right'):
output[3] = 1
return output
```
NOTE: You'll need to run your training python code as sudo. This is required by linux.
Cheers to all!
just take the taxi's code
The taxi's code uses the in game engine. It doesn't actually know how to drive, they just have coded rules that knows inherently where the roads are, where to go, how to act...etc. The in-game taxis, for example, could never leave GTA V's city and drive anywhere else.
grab screen is not reading any screen data just it is showing a grey window, please help me
i am using visual studio and i.ve tried it in idle but same result.
Your not the only one. I use spyder as python compiler and grabscreen is processing no screen data.