Hey everyone! There are two MAJOR updates that make this guide much easier: 1) Installing TensorFlow on the Pi is now much easier. Rather than downloading and installing the wheel file at the beginning of Step 2 of this video, you can simply issue "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" to install it. You still need to issue "sudo pip3 install pillow lxml jupyter matplotlib cython" and "sudo apt-get install python-tk" after installing TensorFlow. 2) You don't have to build protobuf from source anymore! In Step 4, simply use "sudo apt-get install protobuf-compiler". Now you can skip everything from 6:05 to 10:28 . Also, it seems opencv-python has one more package requirement now. In Step 3, when doing the apt-get commands, add "sudo apt-get libatlas-base-dev". This guide works on both the Raspberry Pi 3 and Raspberry Pi 4 (Raspbian Stretch and Raspbian Buster).
@@sbhurke1 I just posted a video showing what the FPS is running TensorFlow (and TensorFlow Lite) object detection on the Pi 4. Check it out! ua-cam.com/video/TiOKvOrYNII/v-deo.html
Same issue with protoc, with "apt-get install protobuf-compiler" 3.0.0 was installed. The latest version as on 11/nov/19 is "3.10.1" . I had to remove all files and compile the latest version from source. So before you do pip3 install check for the latest version.
I have been using your tutorials for the past 3 weeks and they have been really helpful. I just wanted to say thanks I really appreciate the work you have done.
Just a question in your best knowledge what is the fastest detection model that should be used for tensorflow on Raspbbery Pi. I am using ssdlite_mobilenet_v2_coco but the FPS is 1 or less and if it could be faster than this I would appreciate that. I have also looked for tensorflow-gpu version for Raspbbery Pi but I don't think it is supported as of now. @@EdjeElectronics
EXCELLENT TUTORIAL and just what I needed. Perfectly paced and very clear. Thank you. Now onto Pet Detector and making my own model :-) Keep up the good work.
Hi everybody! Two updates: 1) Installing TensorFlow on the Pi is now much easier. Rather than downloading and installing the wheel file at the beginning of Step 2 of this video, you can simply issue "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" to install it. You still need to issue "sudo pip3 install pillow lxml jupyter matplotlib cython" and "sudo apt-get install python-tk" after installing TensorFlow. 2) My Pet Detector Camera video is out! It provides an example of how to find the names of detected objects, determine their locations, and trigger actions if certain objects are detected. You can view it here: ua-cam.com/video/gGqVNuYol6o/v-deo.html Good luck with your projects!
I followed the new way of installing tensorflow, the confirmation said that it was successful, it even installed all the dependencies like protobuf. But when I typed protoc, it said command not found.
For me 'type protoc' shows 'protoc is hashed (/usr/local/bin/protoc)'. Try: ls -la /usr/local/bin/protoc. If it's not there, you may have missed one of the protoc install commands.
Everything worked for me on Windows when training and running my self trained model. But not on my Raspberry Pi. This model you used here, does work on my Pi. Error im getting: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 418, in import_graph_def graph._c_graph, serialized, options) # pylint: disable=protected-access tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'T' not in Op selected_indices:int32>; NodeDef: {{node Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/NonMaxSuppressionV3}} = NonMaxSuppressionV3[T=DT_FLOAT](Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/unstack, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Reshape, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Minimum, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/iou_threshold, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/score_threshold). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.). EDIT: I figured it out, just me being stupidos. i tried to run a 1.12 model on a 1.11 tensorflow installation...
Hi, I'm having an issue after installation. when i run Object_detection-picamera.py Object detection.py, line 137 value error: cannot set WRITEABLE flag to True of this array Any ideas how to fix?
Great tutorial. I'm thankful for the replacement of step 2 instructions, they looked pretty arduous. I added a sleep(300/1000) after each image analysis to avoid the Pi overheating and it seems to keep the same frame rate as if you run it continuously.
This was my error. I had the sleep statement after t2 = cv2.getTickCount() and it should have been before. This was effectively calculating the FPS of the analysis and excluding the sleep. I found I needed to sleep for 750mS to avoid the temp alert on a Pi with a heatsink in a case and this resulted in a 0.6 FPS after the correction.
Thanks for the tutorial, I'm working on a project to install pi security cameras around my house and this should let me set it to send an alert to my phone if a person is detected on my property while I'm away.
Have you also encountered this error? select timeout VIDIOC_DQBUF: Resource temporarily unavailable Traceback (most recent call last): File "Object_detection_picamera.py", line 192, in feed_dict={image_tensor: frame_expanded}) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run run_metadata_ptr) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1121, in _run np_val = np.asarray(subfeed_val, dtype=subfeed_dtype) File "/home/pi/.local/lib/python3.5/site-packages/numpy/core/numeric.py", line 501, in asarray return array(a, dtype, copy=False, order=order) TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Hey everyone, quick heads up: the newest release of OpenCV-Python doesn't work on the Raspberry Pi! Instead of using "pip3 install opencv-python" (which gets the newest version) in Step 3, use "pip3 install opencv-python==3.4.6.27". If you already installed it, just uninstall OpenCV using "pip3 uninstall opencv-python" and then install the older version using "pip3 install opencv-python==3.4.6.27".
Great video! Thanks for sharing!! I just followed the steps and I was able to configure TF on my RPi. When I ran initially, I got a couple of errors. But the fixes were easy. 1. Had to upgrade numpy: pip3 install numpy --upgrade 2. Made few changes in Object_detection_picamera.py. Diff below: 82,83c82,83 < od_graph_def = tf.compat.v1.GraphDef() < with tf.compat.v2.io.gfile.GFile(PATH_TO_CKPT, 'rb') as fid: --- > od_graph_def = tf.GraphDef() > with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid: 88c88 < sess = tf.compat.v1.Session(graph=detection_graph) --- > sess = tf.Session(graph=detection_graph)
@Harsha Kiran, that means the Raspberry Pi is unable to access/recognize the webcam. Most likely, there is something wrong with your camera. Try unplugging it and replugging it in, or try using a different one. I hope that fixes it!
The pet detector video is out! I promise no cats or dogs were harmed during the making of this video (although they may have been slightly annoyed) 😺😺: ua-cam.com/video/gGqVNuYol6o/v-deo.html . Thanks everyone for being awesome viewers!
Hello thank you so much all works as shown in the video! One question, where are all the dependencies installed? How can i delete everything we installed?
Ashutosh Kumar I didn’t get it at first either, but tensorflow is a really good way to work with neural networks and AIs, where opencv is basically just a way to quickly and efficiently edit properties of videos and pictures through python (colors, size, resolution etc...)
Beautifully explained! I managed to get my detector running finally :) now for the part of optimizing the speed by moving to lite versions of tensorflow and yolo... Guys make sure you use a heatsink and a fan on the rpi... once that is done, next mission is to move to Intel NCS2...
@@barskorkmaz8323 i mean the error it was from Objectdetectioncamera.py and when i want to run it i have issue like 'tensorflow'has no attribute 'gfile'
Thank you for this tutorial, I want to ask you question, how can I transfer the real time video to another computer for faster processing and then return result to raspberry pi.
Hey all, I just tested this with my new Raspberry Pi 4, and it does work! It runs a little faster, too: I get about 2.1 FPS using the SSDLite-MobileNet model. However, you have to build OpenCV-Python manually rather than using "pip3 install opencv-python". This is because the new Rasbpian Buster OS only has Python 3.7 installed, and the opencv-python package available through pip still only has the Python 3.5 version. I think the 3.7 package will be available in a few weeks, once PiWheels gets around to uploading it. Until then, instead of doing Step 3 from my guide, you can follow this guide to build/install OpenCV: www.pyimagesearch.com/2018/09/26/install-opencv-4-on-your-raspberry-pi/ .
Yes. I'm using Rasbpian buster. But in pyimagesearch tutorial, he mentioned the installation of opencv by using virtual environment , Any suggestion about installing Opencv without using Virtual environment.?
@@samjithraj7467 My recommendation is to NOT use a virtual environment. You can skip the parts where he talks about setting up the virtual environment. If you do use a virtual environment, you will have to make sure all the TensorFlow packages are installed in that environment too.
@@EdjeElectronics I have installed opencv without using virtual environment, but getting error when i import cv2 that "No module named cv2". How to solve this issue ?
Hi, I'm getting the following error "ImportError: /usr/local/lib/python3.7/dist-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8 ". Any idea? Thx in advance
I reinstalled with another opencv version: pip3 install opencv-python==3.4.6.27 and I am getting the following error: "import tensorflow as tf ModuleNotFoundError: No module named 'tensorflow'". Any help. Thx
@@supermicky7903 I downgraded tensorflow to 1.4 instead of version 2.0 and got "label_map_util.load_labelmap Error: module 'tensorflow' has no attribute 'gfile'"
@@supermicky7903 worked for me using tensorflow 2 with the following changes: tf.io.gfile instead of tf.gfile in file label_map_util.py line 138 tensorflow.compat.v1 instead of tensorflow in object_detection_picamera.py line 27 let me know.
Thank you for your content. I have seen your content about custom model of card using pc, but i still confuse because i use raspi. Would you mind creating custom model content with raspberry?
You're welcome! That's definitely doable, but it will take some significant changes to the code, which I unfortunately don't have time to help with. You might get a better idea of how to do it if you watch my Pet Detector video, which explains how the "scores" and "classes" variables work, and how to do certain things (like increase a variable count) if certain objects are detected. Good luck! ua-cam.com/video/gGqVNuYol6o/v-deo.html
The reason that you may have to reboot when done is that when installing shared libraries by hand you would usually issue: sudo ld_config command...this refreshes the shared library cache. Rebooting does the same thing!
This is a great project , thank u Edje. I need a small help sir. I am doing a project for the visually impaired. And i need to send a audio output of whatever object is being detected. Pls can someone suggest me a way out of it. Thank u :)
Hi, I am working on something similar and need to get an audio output aswell. Did you ever figure out how to get an audio response of what is being detected? Any help would be appreciated!
@@sikhallday HI GUYS MEE TOO WORKING ON SAME PRODUCT, LET'S MAKE THE PRODUCT SUCCESSFUL BY COMBINING TOGETHER, IF YOU GUYS ARE INTERESTED, DON'T HESITATE TO CONTACT ME. SEND YOUR WHAT'S APP NUMBER TO MY MAIL akashyanakandla29@gmail.com
Juan David Barrero L when you run the program and camera it crashes? Are you using USB camera? Mine would crash but I tried a different usb camera I had and it worked.
This is a great tutorial, well-explained and detailed. I built it and made it work, and -- to give you a hint to my noobie-ness -- I don't think I have ever successfully built ANYTHING from source before! Thanks. I have had in mind a "bus detector" project to get my kids involved (and to keep from driving them to school most days),; and this may put me over the really hard part of that project. If you have advice on tuning to get the software to pick up a bus (as opposed to a mug, ball, person... which it does extremely effectively!), please share. I haven't tried, yet. Cheers, Drew
Awesome, glad you were able to get it to work! I like the bus detector idea. You're right, the detection accuracy is pretty finicky. I think the easiest way would be to add code to make it so the program ignores other objects and only reacts if a bus is detected, see my other video ( ua-cam.com/video/gGqVNuYol6o/v-deo.html ) for an example of how to do that. You can also try training your own neural network that ONLY detects school buses, but that takes a lot more work (like at least a weekend or two if you're starting from scratch). I have a different video that shows how that works. It's getting a little outdated, so you might run in to errors. ua-cam.com/video/Rgpfk6eYxJA/v-deo.html
Hello, take a look at this website, it shows several options for doing text-to-speech. pythonprogramminglanguage.com/text-to-speech/ . My Pet Detector video can show you how to get the names of the detected objects: ua-cam.com/video/gGqVNuYol6o/v-deo.html
@@EdjeElectronics Yes sir I have seen the website but the codes did not work because of that text file format in picamera code is pbtxt the file name is "mscoco_label_map.pbtxt"
@@mohammedalshatri9728 Look at the 'gTTS text to speech portion' of the page I linked. It shows you the code for getting it to say "Good morning". You just need to substitute "Good morning" with the name of the detected object. You can see how to find the name of detected objects in my Pet Detector video: ua-cam.com/video/gGqVNuYol6o/v-deo.html
Excellent tutorial my friend! I did it to read snapshots from an IP camera. It works fine but after a while a red thermometer shows and after a while again the raspberry freezes... Do you know why?
I ran in to the same problem too! It means the Raspberry Pi’s processor is overheating and shutting down. Running the neural network takes lots of processing power, so the CPU generates heat more quickly than it can dissipate it. If you put a heat sink on the processor and set up a fan to blow air over the heat sink, that should keep it cool enough.
Hey, thank you for your tutorial! I got this after i wan to start some object-detecting tensorflow.python.framework.errors_impl.NotFoundError: /home/pi/tensorflowl/models/research/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb; No such file or directory How do I fix this? Thank you in advance for any help given!
@@ivantodorov267 It looks like the frozen_inference_graph.pb file is missing. Did you download and unpack the SSDLite-MobileNet model as instructed at the end of Step 5?
Hey Edje, Thanks again I have successfully done the installation of "SSDLite-MobileNet model" but wants to do my own model of objectdection so I follow the step 6 17:26 but when it opens the script is empty nada no code do u know the problem? I use VNC view to code
You can use the Object_detection_image.py script from my other GitHub repository ( github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10/blob/master/Object_detection_image.py ) with the Raspberry Pi. You just need to change the MODEL_NAME, PATH_TO_LABELS, and NUM_CLASSES variables to match what you are using on the Pi.
Thank you. Can I ask based on your experience, do tensorflow can detect objects base on its color? (I was worried it because I see many articles about opencv detect object's color but none of them tell about tensorflow).
Hey great tutorial. Once I have the program running that detects things, how do I get it to output useful information about what it detects so I can control motors or something similar? In other words, I want to detect an object, then based on what it detects make a motor perform an operation. Any help would be appreciated. Thanks in advance.
Thanks, sounds like a fun project! I think my Pet Detector code will help you get going in the right direction. It shows how to trigger an action if a certain object is detected. Check out the video at: ua-cam.com/video/gGqVNuYol6o/v-deo.html
@@EdjeElectronics Thanks for the tip. I'm trying to work on a Lego classifier at the moment. I am trying to figure out how to make the program output the name of the class that it detects. I have 20 Lego types, so once it detects a Lego I want it to output the class. I can't figure out how the program spits out a class for you to see. I just want it as a simple string output.
Somehow I get an error I dont understand by typing: "python3 Object_detection_picamera.py --usbcam" in "~/tensorflow1/models/research/object_detection". The error is: Traceback (most recent call last): File "Object_detection_picamera.py", line 23, in import cv2 File "/home/pi/.local/lib/python3.7/site-packages/cv2/__init__.py", line 3, in from .cv2 import * ImportError: /home/pi/.local/lib/python3.7/site-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8 I am using a clean installation of Raspbian Buster 10 on a pi3 b+. Had someone a simmilar issue and know where I made an error? As the next step im going to redo everything linke in the written gitub tutorial. Thanks
That's just because the newer versions of OpenCV still have errors. Try installing an older version using: pip3 install opencv-python==3.4.6.27 . Then, re-run the script. Hope that helps!
Great video. I am trying to send webcam video over my local network from the pi to a PC which should perform object detection and send the output stream back to the pi over the local network. I hope this will increase fps even after networking overhead. Do you have any tips or experience on how to implement that?
Hi everyone, I fulfilled each and every step successfully except the compiling one : "python3 Object_detection_picamera.py " Once I had tried to compile it, raspi warn me about exceeding memory requirements, it says kinda : "Allocation of 5031936 exceeds 10% of system memory" I have raspberry pi 3 model b+ and have 32 gb class 10 sandisk sd card to operate. I dont know what is wrong and how can I fix it, pls suggest smt to improve my allocation strategy :(
You can ignore that warning, the object detection program should still run. Make sure to close all other applications when you run Object_detection_picamera.py so you can free up as much memory (RAM) as possible.
@@EdjeElectronics thank you soo much for your fast response, but furter more, it is rebooting raspberry pi automatically. so can't move on with it :( The exact error before it reboots itself : W0723 02:39:30.401308 1996098256 deprecation_wrapper.py:118] From /home/pi/tensorflow1/models/research/object_detection/utils/label_map_util.py:137: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead. tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 5660928 exceeds 10% of system memory. EDIT: solved. I replaced my power source (I used to use apple charger before) with samsung fast charger. I don't know how but it affects the performance. In my project I was using raspberry on an unmanned aerial vehicle, once I feed it directly from Lipo battary of the drone using BEC voltage regulator, It worked fine too interestingly. I hope It helps.
I have fast the same problem, only there are some other deprecated functions from tf and after that i can't allocate also.... WARNING: Logging before flag parsing goes to stderr. W0731 12:40:06.483510 1995642432 deprecation_wrapper.py:119] From /home/pi/tensorflow1/models/research/object_detection/utils/label_map_util.py:137: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead. W0731 12:40:06.636931 1995642432 deprecation_wrapper.py:119] From Object_detection_picamera.py:82: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead. W0731 12:40:25.970835 1995642432 deprecation_wrapper.py:119] From Object_detection_picamera.py:88: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2019-07-31 12:40:41.656910: W tensorflow/core/framework/allocator.cc:107] Allocation of 11059200 exceeds 10% of system memory. 2019-07-31 12:40:41.672656: W tensorflow/core/framework/allocator.cc:107] Allocation of 11059200 exceeds 10% of system memory. 2019-07-31 12:40:41.779372: W tensorflow/core/framework/allocator.cc:107] Allocation of 8640000 exceeds 10% of system memory. : cannot connect to X server
Hi, just wonder if you are able to solve the allocation exceeds system memory issue? i am facing this issue and I closed all irrelevant applications before running the Object_detection_picamera.py command but still cannot. Apparently it is just a warning, but picamera still couldn't on for object detection:(
KUA MEI SAN Hi, after I replaced my power source (I used to use apple charger before) with samsung fast charger, it affects the performance in an interesting way. In my project I was using raspberry on an unmanned aerial vehicle, once I feed it directly from Lipo battary of the drone using BEC voltage regulator, It worked fine too. All related suggestions on the internet is about changing power supply or its cable. So, please try to feed your device with better alternative. I hope It helps. Let me know the result.
Hi, Edje. Thank you so much for your awesome tutorial. Can I ask a question? I am a Ph.D. student in biology. I plan to use object detection to count the dead or living insects under some treatments. I want to monitor the survival rate of these insects under some treatments. So, do you have any recommended tutorials with Raspberry Pi? Thank you
Hey all, while executing the statement 'python3 Object_detection_picamera.py' in the last step (detecting objects) I got the following messages. Does anyone know what went wrong? Thanks a lot: pi@raspberrypi:~/tensorflow1/models/research/object_detection $ python3 Object_detection_picamera.py Traceback (most recent call last): File "Object_detection_picamera.py", line 23, in import cv2 File "/home/pi/.local/lib/python3.7/site-packages/cv2/__init__.py", line 3, in from .cv2 import * ImportError: /home/pi/.local/lib/python3.7/site-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8
I have a question. At 11:50 he said that you have to save and exit the menu but there are different options to save it (Dos and Mac). Which one do I have to use?
Hello , thank's for the tuto , it seems that the object detection script will be running on its own otherwise the raspi will stop , could you specify cpu and memory consumption , thank you
Got an error at very end saying there's no matplotlib after using updated step 2. Fixed with "pip3 install matplotlib" which at first did not work but in error it mentions 2 other things to install which can be done using "sudo apt-get" followed by their respective names. Do "pip3 install matplotlib" again and it should work
Awesome tutorial! Do you think this can work on a flying drone? My drone has a Picamera and I wanted to install this software, could you give me any tips/advice on making this software run smoothly during flight?
I have to detect and sort three color based objects my question is how can i connect a servomotor with this in order to detect and sort in accordance with the relevant color match?
That's definitely do-able! However, it might be easier to use OpenCV to determine the color of the object. This link might help you: www.pyimagesearch.com/2014/08/04/opencv-python-color-detection/ . Also, check out my pet detector camera video, which shows an example of how to trigger actions if certain objects are detected: ua-cam.com/video/gGqVNuYol6o/v-deo.html
please help me out im stuck at Traceback (most recent call last): File "Object_detection_picamera.py.2", line 23, in import cv2 File "/usr/local/lib/python3.5/dist-packages/cv2/__init__.py", line 3, in from .cv2 import * ImportError: libQtGui.so.4: cannot open shared object file: No such file or directory
I had the same problem, Edje helped by saying the solution to the problem is to try use the command "sudo pip3 install opencv-python". Hope it was the solution:)
i have a problem with my screen....I got this to work and massiveee thankss to your tutorial here!! but im using a 3.5" screen and the tab is too big and i cant change the size. is there any way to solve this? ive tried changing the framebuffer_width and length and still nothing (ps thats 320 and 280) Ty!
Hello thanks for the tutorial, i have a question. I'm able to use this project on raspberry 2 with raspberry pi cam v2. However, i can only get 0.8 fps. How can i make my fps 60? Do i need to buy other versions of raspberry such as 3,4?
Hi, you're welcome! Please check out this video to get an idea of what FPS you'll get with various configurations of the Pi 3 and 4. You won't be able to get 60FPS, but you can get 30FPS with a Pi 4 and Coral USB Accelerator! ua-cam.com/video/TiOKvOrYNII/v-deo.html
Is there an endpoint that you can query to know what the camera is detecting? This would be extremely useful for creating an alarm system in home assistant.
Hi. During I execute the "make" command in the protobuf directory i get this error of "libtool: error: 'google/protobuf/compiler/csharp/csharp_generator.lo' is not a valid libtool object" I need your consultancy since I couldnt figure out what to do.
Hi Evan, I have managed to install all the steps up until point 15:51 in the video . After typing python3 Object_detection_picamera.py I get the messages listed below: ImportError: numpy.core.multiarray failed to import Traceback (most recent call last): File "Object_detection_picamera.py", line 23, in import cv2 File "/home/pi/.local/lib/python3.5/site-packages/cv2/__init__.py",line 3, in from .cv2 import * ImportError: numpy.core.multiarray failed to import In truth, I have no idea what any of this means. Any assistance with this would be greatly appreciated. I am using a Pi NoIR Camera V2 on my Raspberry pi 3 Model B+. Thanks in advance, hoping to hear from you
Hi Nicolas, I wrote a script that will play a video if a certain object is detected in the image. You can use it as an example for how to perform some kind of action (like turning on a GPIO) when an object is detected. Here's a link to the script: www.dropbox.com/s/xlpiup19vu8qqla/Object_detection_webcam_notification.py?dl=0 . Look at lines 132 - 176.
Hi Edje, what aspect of this project do you think can be improved on? For example, could the framerate be improved by changing the code (other than the camera resolution)?
You could slightly increase the framerate by reading the camera in a threaded way. At the moment, the camera is read and the detection is performed in the same loop. Not sure how much fps you'd gain though!
@Michail Vanis I did try creating a separate thread for the camera as described here: www.pyimagesearch.com/2015/12/28/increasing-raspberry-pi-fps-with-python-and-opencv/ . However, it didn't play nicely with TensorFlow! I got lots of errors. I think TensorFlow just requires too much processing bandwidth for doing multithreading.
@Immanuel Ng There are lots of ways to improve this project. You could train your own model that has more accurate detection (the pre-trained SSDLite-MobileNet I'm using here isn't very accurate at identifying objects correctly). You can add more functionality to it, like causing a robotic hand to wave when a person is detected in the image. As far as improving framerate goes, there may be some small improvements that can be made to the code, but I don't think they will make a significant difference. You can always keep an eye on the TensorFlow model zoo ( github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md ) to see if they release any models that are faster than SSDLite-MobileNet.
@@EdjeElectronics Interesting! I've got a handy Python class for doing the camera stuff here: github.com/mikevanis/CameraController I might have time to give it a go in a few weeks.
Hey Allanrey! Unfortunately, there aren't really any good ways to improve the frame rate. You can try looking into an Intel Modivius NCS (a USB device that increases processing power: ua-cam.com/video/p8o284gBSqo/v-deo.html ), but they don't work with TensorFlow models.
I wanted to share an issue that I had in case someone else has the same issue. When you install pip3 install tensorflow, no matter the version, it does not compile protobuf automatically, you still have to do that part yourself. I thought it did and I kept getting bash protoc command not found. It was straight twisting my brain around.
Hi Jassiel, I haven't tried using the Movidius NCS. It may be possible, but it takes some work. Take a look at Mark Jay's video series on using the NCS with the Pi (ua-cam.com/video/p8o284gBSqo/v-deo.html). Note that he shows how to use YOLO rather than TensorFlow. This page shows you how to compile TensorFlow models for the NCS: movidius.github.io/ncsdk/tf_modelzoo.html . Please let me know if you get it to work!
Grat tutorial thank you. Hopwever when using wget the coco models i get the follow regardless of which model i use 2019-02-16 19:03:27 (14.7 MB/s) - Read error at byte 5655576/51025348 (Connection reset by peer). Retrying.
thanks. it works fine. although i don't like the barely 2 fps. Is there any other small hardware that can give better performance as i am trying to work on an image processing project? thanks
Hello, thanks for the good question! That's a tough one. The OpenCV algorithm is much more reliable and accurate than the TensorFlow model. However, it won't work if the cards or overlapped or if the cards are against a non-uniform background. Ultimately, I think the TensorFlow model is better because it provides much more flexibility in the application, and its accuracy can be improved by giving it more training data. Actually, I just finished training a YOLO model that works very well at identifying cards. I'll be posting a video on it this month! I based the work off this guy's excellent video, which shows just how good the machine learning model can be: ua-cam.com/video/pnntrewH0xg/v-deo.html . This is the approach I'll be using for my blackjack robot!
is there any chance of getting the text as a voice feedback after the object is detected?? because i would like to use this as a project for the blind with detecting objects and hence alerting them with a voice feedback..
Hi, thanks for great tutorial i've manged to install evereything. And it is working! But at first I faced the problem of overheating (small termometer icon appeared on desktop and raspberry pi just hanged) when i run python3 Object_detection_picamera.py --usbcam (I use cheap usb camera). I've placed a cooler on top of pi and the problem with temperature was solved (no termometer icon anymore). But now Object_detection_picamera.py just hangs my Pi, after a minute or two of work. Can you advice me how can I debug the issue? I'm using Raspberry Pi 3B (without plus) and lateset Raspbian.
How much memory is left on your sd card? Try issuing df -h in the terminal , maybe that's causing problems although unlikely... And maybe you need to download fwusb ( not v sure of the name) library so that usb camera runs well , tge library was mentioned in a previous comment it's for supporting external usb cameras . For me , it made the code ran perfectly after that , I only had problems when I tried to turn it off though .. I'm also using raspberry pi 3 B+ model
Thanks for the video. I woke up this morning, and thought to myself "I wonder if my Pi can run tensorflow?" And here I am. One question, why didn't you use make -j 4 to speed up compile time?
Thanks! It makes for a good weekend project 😄. It seemed to fail more frequently if I used the make -j4 option, so I decided not to use it for the video. Fortunately we don't have to build it anymore and can just use "sudo apt-get install protobuf-compiler" to install it now!
I am trying now to follow the guide. When installing the .WHL file, the PI reports error: .WHL is not a supported wheel on this platform. Any idea why? I am using PI 4, and tried latest version, as well as 1.8.0
At the last step when trying to run the code to activate the camera, I get an error saying ImportError: No module named 'matplotlib' Any ideas what to do here? For step 2 I used the shortcut Edje Electronics mentioned in the comments by issuing "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" , I wonder if that has anything to do with it?
Use "pip3 install matplotlib" which at first did may not work but in error it mentions 2 other things to install which can be done using "sudo apt-get" followed by their respective names. Do "pip3 install matplotlib" again once you've installed the other parts and it should work
Thaks for the video it works perfectly, i have one question, the fps in my raspberry is 0.7 sometimes it's 0.8, is there a way that I can rise like yours maybe 1 or 1.2 ?
Hmm, what Raspberry Pi model are you using? Mine is a Pi 3B+, which is the fastest one available. That might be what causes the difference. Also, make sure all your other applications are closed while you run the script.
Thanks for the video! So could I train my own model on a certain detection model from the model zoo (ssd_resnet) and then run it with a different detection model (ssd_mobilenet)?
No, the models aren't interchangeable like that. When you train a model, it generates a frozen_inference_graph.pb file which contains the detection graph for the particular model you trained for. When you run a program (like my Object_detection_picamera.py) script, it loads that frozen_inference_graph.pb and runs it. So in other words, whichever model you trained is also the model you run. I hope that makes sense!
Hi everyone. Excellent tutorial. I desperately need help. I am following all the steps of the tutorial but when I try to train my own model I always get an error related to the tf.contrib library regardless of the version of tensorflow. Do I need to make any additional changes? I would greatly appreciate a help. (I have been using TF 1.14, TF 2.0 and TF 2.1 +Python 3.7.4 with the same result)
Hey Dominic, check out the "object_detection_video.py" script I created for my TensorFlow on Windows tutorial. Here's a link to the code. You might have to tweak some things to make it work on the Raspberry Pi. github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10/blob/master/Object_detection_video.py
Hey Edje Electronics, thank you so much for your video. We followed all the steps and are able to get the object detection working successfully on our raspberry pi 4. We are actually trying to get an audio output for the object our pi camera detects. For example, if a chair is detected by the pi it should give an audio like "there is a chair in front of you". We are trying to do this for a project and it would be really great if you could give us some tip for it. Thanks.
Hi thanks for this tutorial, I am getting a error "ImportError: libImath-2_2.so.12: cannot open shared object file: No such file or directory". What could be the issue??
Hey everyone! There are two MAJOR updates that make this guide much easier:
1) Installing TensorFlow on the Pi is now much easier. Rather than downloading and installing the wheel file at the beginning of Step 2 of this video, you can simply issue "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" to install it. You still need to issue "sudo pip3 install pillow lxml jupyter matplotlib cython" and "sudo apt-get install python-tk" after installing TensorFlow.
2) You don't have to build protobuf from source anymore! In Step 4, simply use "sudo apt-get install protobuf-compiler". Now you can skip everything from 6:05 to 10:28 .
Also, it seems opencv-python has one more package requirement now. In Step 3, when doing the apt-get commands, add "sudo apt-get libatlas-base-dev".
This guide works on both the Raspberry Pi 3 and Raspberry Pi 4 (Raspbian Stretch and Raspbian Buster).
Can we achieve higher fps by using Raspberry Pi 4B
if it is so what will be the maximum FPS for the same
@@sbhurke1 I just posted a video showing what the FPS is running TensorFlow (and TensorFlow Lite) object detection on the Pi 4. Check it out! ua-cam.com/video/TiOKvOrYNII/v-deo.html
Does it work on pi zero
"pip3 install tensorflow" is installing a older version. Better to follow the video
OR
pip3 install tensorflow==1.14.0
Same issue with protoc, with "apt-get install protobuf-compiler" 3.0.0 was installed. The latest version as on 11/nov/19 is "3.10.1" . I had to remove all files and compile the latest version from source.
So before you do pip3 install check for the latest version.
I have been using your tutorials for the past 3 weeks and they have been really helpful. I just wanted to say thanks I really appreciate the work you have done.
Thanks man, I'm glad to know it has been useful! Good luck on your projects.
Just a question in your best knowledge what is the fastest detection model that should be used for tensorflow on Raspbbery Pi. I am using ssdlite_mobilenet_v2_coco but the FPS is 1 or less and if it could be faster than this I would appreciate that. I have also looked for tensorflow-gpu version for Raspbbery Pi but I don't think it is supported as of now. @@EdjeElectronics
I can't thank you enough to explain everything step by step. Liked and subbed.
After two days and going though couple of errors, I finally got it working. Thank you for the tutorial, I learned a lot during this project.
Man you should be a professor. You explain everything easily
Edje - You are the best. Tried couple of other instructions, but yours is the only one working! THANK YOU
EXCELLENT TUTORIAL and just what I needed. Perfectly paced and very clear. Thank you. Now onto Pet Detector and making my own model :-) Keep up the good work.
Hi everybody! Two updates:
1) Installing TensorFlow on the Pi is now much easier. Rather than downloading and installing the wheel file at the beginning of Step 2 of this video, you can simply issue "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" to install it. You still need to issue "sudo pip3 install pillow lxml jupyter matplotlib cython" and "sudo apt-get install python-tk" after installing TensorFlow.
2) My Pet Detector Camera video is out! It provides an example of how to find the names of detected objects, determine their locations, and trigger actions if certain objects are detected. You can view it here: ua-cam.com/video/gGqVNuYol6o/v-deo.html
Good luck with your projects!
I followed the new way of installing tensorflow, the confirmation said that it was successful, it even installed all the dependencies like protobuf. But when I typed protoc, it said command not found.
For me 'type protoc' shows 'protoc is hashed (/usr/local/bin/protoc)'. Try: ls -la /usr/local/bin/protoc. If it's not there, you may have missed one of the protoc install commands.
Everything worked for me on Windows when training and running my self trained model. But not on my Raspberry Pi. This model you used here, does work on my Pi. Error im getting:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 418, in import_graph_def
graph._c_graph, serialized, options) # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'T' not in Op selected_indices:int32>; NodeDef: {{node Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/NonMaxSuppressionV3}} = NonMaxSuppressionV3[T=DT_FLOAT](Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/unstack, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Reshape, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Minimum, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/iou_threshold, Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/non_max_suppression/score_threshold). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
EDIT: I figured it out, just me being stupidos. i tried to run a 1.12 model on a 1.11 tensorflow installation...
@Reinier, thanks for letting us know about the resolution to your error!
Hi, I'm having an issue after installation. when i run Object_detection-picamera.py
Object detection.py, line 137
value error: cannot set WRITEABLE flag to True of this array
Any ideas how to fix?
Great tutorial. I'm thankful for the replacement of step 2 instructions, they looked pretty arduous. I added a sleep(300/1000) after each image analysis to avoid the Pi overheating and it seems to keep the same frame rate as if you run it continuously.
Thanks, I'll definitely incorporate that into my code for my next projects! Any ideas why adding the sleep command doesn't reduce frame rate?
This was my error. I had the sleep statement after t2 = cv2.getTickCount() and it should have been before. This was effectively calculating the FPS of the analysis and excluding the sleep. I found I needed to sleep for 750mS to avoid the temp alert on a Pi with a heatsink in a case and this resulted in a 0.6 FPS after the correction.
Great job. You have helped me so much with this tutorial, keep up the great job it's extraordinary !!!!!
Thanks for the tutorial, I'm working on a project to install pi security cameras around my house and this should let me set it to send an alert to my phone if a person is detected on my property while I'm away.
Hi, I just followed your tutorial and it works! Thank you so much!
Hello :) I just want to ask if you are using a raspberry pi camera module or usb webcam?
Have you also encountered this error?
select timeout
VIDIOC_DQBUF: Resource temporarily unavailable
Traceback (most recent call last):
File "Object_detection_picamera.py", line 192, in
feed_dict={image_tensor: frame_expanded})
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1121, in _run
np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
File "/home/pi/.local/lib/python3.5/site-packages/numpy/core/numeric.py", line 501, in asarray
return array(a, dtype, copy=False, order=order)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
@@markjasonong5668 i used pi camera
@@markjasonong5668 no I didnt encounter this error
@@xinyueyang1962 ok thanks for replying :)
many many thanks for the video and instructions guys, also for the great comments!
Hey everyone, quick heads up: the newest release of OpenCV-Python doesn't work on the Raspberry Pi! Instead of using "pip3 install opencv-python" (which gets the newest version) in Step 3, use "pip3 install opencv-python==3.4.6.27". If you already installed it, just uninstall OpenCV using "pip3 uninstall opencv-python" and then install the older version using "pip3 install opencv-python==3.4.6.27".
Edje Electronics when I install tensorflow on raspberry pi zero it downloads 99 percent not 100 please help
Edje Electronics help
EDJE Help please
Great video! Thanks for sharing!!
I just followed the steps and I was able to configure TF on my RPi. When I ran initially, I got a couple of errors. But the fixes were easy.
1. Had to upgrade numpy:
pip3 install numpy --upgrade
2. Made few changes in Object_detection_picamera.py. Diff below:
82,83c82,83
< od_graph_def = tf.compat.v1.GraphDef()
< with tf.compat.v2.io.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
---
> od_graph_def = tf.GraphDef()
> with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
88c88
< sess = tf.compat.v1.Session(graph=detection_graph)
---
> sess = tf.Session(graph=detection_graph)
dude, you rock !
i followed your steps and it works fine on RPI 3 model B
does it really work? i am getting an error saying cant open camera by index 0
@@soothmaster9905 yes it works on my rpi3 model B, the downside is that the FPS is 0 7~ 0.8, pretty slow in detecting
@Harsha Kiran, that means the Raspberry Pi is unable to access/recognize the webcam. Most likely, there is something wrong with your camera. Try unplugging it and replugging it in, or try using a different one. I hope that fixes it!
The pet detector video is out! I promise no cats or dogs were harmed during the making of this video (although they may have been slightly annoyed) 😺😺: ua-cam.com/video/gGqVNuYol6o/v-deo.html . Thanks everyone for being awesome viewers!
Hi
i want to know how to send signal digital to pin on rassberry pi when it find objects
Hello thank you so much all works as shown in the video!
One question, where are all the dependencies installed? How can i delete everything we installed?
Hey dude I am a beginner at this. Can you please tell me why are we using both tensor flow and open cv.
Ashutosh Kumar I didn’t get it at first either, but tensorflow is a really good way to work with neural networks and AIs, where opencv is basically just a way to quickly and efficiently edit properties of videos and pictures through python (colors, size, resolution etc...)
@@maxthecatoonist1206 Thanks brother
Beautifully explained! I managed to get my detector running finally :) now for the part of optimizing the speed by moving to lite versions of tensorflow and yolo... Guys make sure you use a heatsink and a fan on the rpi... once that is done, next mission is to move to Intel NCS2...
Hi,
when I want to run the Object_detection_picamera.py i have some issue from module 'tensorflow' has no attribute 'gfile'?
same error :(
@@barskorkmaz8323 i mean the error it was from Objectdetectioncamera.py and when i want to run it i have issue like 'tensorflow'has no attribute 'gfile'
@@ThePhobosPL I think we got this error because of the Tensorflow 2.0 I installed exact versions and it works
i got same error is there any solution on it
Thank you for this tutorial, I want to ask you question, how can I transfer the real time video to another computer for faster processing and then return result to raspberry pi.
Hey all, I just tested this with my new Raspberry Pi 4, and it does work! It runs a little faster, too: I get about 2.1 FPS using the SSDLite-MobileNet model. However, you have to build OpenCV-Python manually rather than using "pip3 install opencv-python". This is because the new Rasbpian Buster OS only has Python 3.7 installed, and the opencv-python package available through pip still only has the Python 3.5 version. I think the 3.7 package will be available in a few weeks, once PiWheels gets around to uploading it. Until then, instead of doing Step 3 from my guide, you can follow this guide to build/install OpenCV: www.pyimagesearch.com/2018/09/26/install-opencv-4-on-your-raspberry-pi/ .
Yes. I'm using Rasbpian buster. But in pyimagesearch tutorial, he mentioned the installation of opencv by using virtual environment , Any suggestion about installing Opencv without using Virtual environment.?
@@samjithraj7467 My recommendation is to NOT use a virtual environment. You can skip the parts where he talks about setting up the virtual environment. If you do use a virtual environment, you will have to make sure all the TensorFlow packages are installed in that environment too.
@@EdjeElectronics I have installed opencv without using virtual environment, but getting error when i import cv2 that "No module named cv2". How to solve this issue ?
ah yes, python 3.7 is there now
www.piwheels.org/simple/opencv-python/opencv_python-3.4.3.18-cp37-cp37m-linux_armv7l.whl
Works like a charm! Had to install the 'fswebcam' usbcam first.
Hi,
I'm getting the following error "ImportError: /usr/local/lib/python3.7/dist-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8
".
Any idea?
Thx in advance
I reinstalled with another opencv version: pip3 install opencv-python==3.4.6.27 and I am getting the following error:
"import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'". Any help. Thx
I have the exact same problem, could you please help us?
@@supermicky7903 I downgraded tensorflow to 1.4 instead of version 2.0 and got "label_map_util.load_labelmap Error: module 'tensorflow' has no attribute 'gfile'"
@@supermicky7903 worked for me using tensorflow 2 with the following changes:
tf.io.gfile instead of tf.gfile in file label_map_util.py line 138
tensorflow.compat.v1 instead of tensorflow in object_detection_picamera.py line 27
let me know.
Thank you for your content. I have seen your content about custom model of card using pc, but i still confuse because i use raspi. Would you mind creating custom model content with raspberry?
Object detection.py, line 137
value error: cannot set WRITEABLE flag to True of this array
Make sure you edit with the python ide, nano is finicky.
You have to use spaces instead of tabs
Hi thank you so much for making this video. I was wondering how I could display the number of counted "person"s instead of displayed the frame rate?
You're welcome! That's definitely doable, but it will take some significant changes to the code, which I unfortunately don't have time to help with. You might get a better idea of how to do it if you watch my Pet Detector video, which explains how the "scores" and "classes" variables work, and how to do certain things (like increase a variable count) if certain objects are detected. Good luck! ua-cam.com/video/gGqVNuYol6o/v-deo.html
Can I build Currency recognition and giving output via speaker using raspberry pi I need to build this project sir please help
hi @Prateek have you solve the output audio problem?
were you able to make it
The reason that you may have to reboot when done is that when installing shared libraries by hand you would usually issue: sudo ld_config command...this refreshes the shared library cache. Rebooting does the same thing!
This is a great project , thank u Edje. I need a small help sir. I am doing a project for the visually impaired. And i need to send a audio output of whatever object is being detected. Pls can someone suggest me a way out of it. Thank u :)
Hi, I am working on something similar and need to get an audio output aswell. Did you ever figure out how to get an audio response of what is being detected?
Any help would be appreciated!
@@sikhallday HI GUYS MEE TOO WORKING ON SAME PRODUCT, LET'S MAKE THE PRODUCT SUCCESSFUL BY COMBINING TOGETHER, IF YOU GUYS ARE INTERESTED, DON'T HESITATE TO CONTACT ME. SEND YOUR WHAT'S APP NUMBER TO MY MAIL akashyanakandla29@gmail.com
Hi Nitesh,me too working on same product. SEND YOUR WHAT'S APP CONTACT NUMBER to my mail akashyanakandla29@gmail.com
I was able to get this to work using your instructions. Thank you!
Hi can you help, I can't get it right, did you use the same steps than in the video ? Or maybe do something else
Juan David Barrero L hi, yes I used the same steps in the video. I also used a 64 Gb SD card. What step do you get stuck at?
@@dogdrone5186 I can finish the all process the problem is when I run the example it just crashed, and I am using a microSD of 32Gb
Juan David Barrero L when you run the program and camera it crashes? Are you using USB camera? Mine would crash but I tried a different usb camera I had and it worked.
@@dogdrone5186 I am using a raspberry'camera, did you use the same library's or you actualize some of it ? For a newest version
This is a great tutorial, well-explained and detailed. I built it and made it work, and -- to give you a hint to my noobie-ness -- I don't think I have ever successfully built ANYTHING from source before! Thanks. I have had in mind a "bus detector" project to get my kids involved (and to keep from driving them to school most days),; and this may put me over the really hard part of that project. If you have advice on tuning to get the software to pick up a bus (as opposed to a mug, ball, person... which it does extremely effectively!), please share. I haven't tried, yet. Cheers, Drew
Awesome, glad you were able to get it to work! I like the bus detector idea. You're right, the detection accuracy is pretty finicky. I think the easiest way would be to add code to make it so the program ignores other objects and only reacts if a bus is detected, see my other video ( ua-cam.com/video/gGqVNuYol6o/v-deo.html ) for an example of how to do that. You can also try training your own neural network that ONLY detects school buses, but that takes a lot more work (like at least a weekend or two if you're starting from scratch). I have a different video that shows how that works. It's getting a little outdated, so you might run in to errors. ua-cam.com/video/Rgpfk6eYxJA/v-deo.html
witch python version for tensorflow-1.8.0-cp35-none-linux_armv7l.whl because 3.5.1 is not compatable?
Just use "pip3 install tensorflow" 😀
@@EdjeElectronics Hi, everything work great. All problems were gone. Great tutorial. All best for you😊
Hi, thanks a lot for this! Any chance you could make this work with a google coral and an RTSP camera, running in Home Assistant?
hi I have a question
how can I make the code to say in voice the detected objects
thanks sir
Hello, take a look at this website, it shows several options for doing text-to-speech. pythonprogramminglanguage.com/text-to-speech/ . My Pet Detector video can show you how to get the names of the detected objects: ua-cam.com/video/gGqVNuYol6o/v-deo.html
@@EdjeElectronics Yes sir I have seen the website but the codes did not work because of that text file format in picamera code is pbtxt
the file name is "mscoco_label_map.pbtxt"
I need your help please
@@mohammedalshatri9728 Look at the 'gTTS text to speech portion' of the page I linked. It shows you the code for getting it to say "Good morning". You just need to substitute "Good morning" with the name of the detected object. You can see how to find the name of detected objects in my Pet Detector video: ua-cam.com/video/gGqVNuYol6o/v-deo.html
Excellent tutorial my friend! I did it to read snapshots from an IP camera. It works fine but after a while a red thermometer shows and after a while again the raspberry freezes... Do you know why?
I ran in to the same problem too! It means the Raspberry Pi’s processor is overheating and shutting down. Running the neural network takes lots of processing power, so the CPU generates heat more quickly than it can dissipate it. If you put a heat sink on the processor and set up a fan to blow air over the heat sink, that should keep it cool enough.
Hey, thank you for your tutorial!
I got this after i wan to start some object-detecting
tensorflow.python.framework.errors_impl.NotFoundError: /home/pi/tensorflowl/models/research/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb; No such file or directory
How do I fix this?
Thank you in advance for any help given!
See my response on Twitter! 😃
@@EdjeElectronics can you please link the response for this Error, I've the same error
@@ivantodorov267 It looks like the frozen_inference_graph.pb file is missing. Did you download and unpack the SSDLite-MobileNet model as instructed at the end of Step 5?
Hey Edje, Thanks again I have successfully done the installation of "SSDLite-MobileNet model" but wants to do my own model of objectdection so I follow the step 6 17:26 but when it opens the script is empty nada no code do u know the problem? I use VNC view to code
You could share the image of your SDcard!
thanks for a good tutorial!
however, did you consider using something like NCS Movidius to improve the FPS?
Hi, can you give me a python script that just detect objects in a image and also that working on your system?
You can use the Object_detection_image.py script from my other GitHub repository ( github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10/blob/master/Object_detection_image.py ) with the Raspberry Pi. You just need to change the MODEL_NAME, PATH_TO_LABELS, and NUM_CLASSES variables to match what you are using on the Pi.
Thank you. Can I ask based on your experience, do tensorflow can detect objects base on its color? (I was worried it because I see many articles about opencv detect object's color but none of them tell about tensorflow).
Hey great tutorial. Once I have the program running that detects things, how do I get it to output useful information about what it detects so I can control motors or something similar? In other words, I want to detect an object, then based on what it detects make a motor perform an operation. Any help would be appreciated. Thanks in advance.
Thanks, sounds like a fun project! I think my Pet Detector code will help you get going in the right direction. It shows how to trigger an action if a certain object is detected. Check out the video at: ua-cam.com/video/gGqVNuYol6o/v-deo.html
@@EdjeElectronics Thanks for the tip. I'm trying to work on a Lego classifier at the moment. I am trying to figure out how to make the program output the name of the class that it detects. I have 20 Lego types, so once it detects a Lego I want it to output the class. I can't figure out how the program spits out a class for you to see. I just want it as a simple string output.
Hey, did you get the solution? Please help. Thanks
Somehow I get an error I dont understand by typing: "python3 Object_detection_picamera.py --usbcam" in "~/tensorflow1/models/research/object_detection".
The error is:
Traceback (most recent call last):
File "Object_detection_picamera.py", line 23, in
import cv2
File "/home/pi/.local/lib/python3.7/site-packages/cv2/__init__.py", line 3, in
from .cv2 import *
ImportError: /home/pi/.local/lib/python3.7/site-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8
I am using a clean installation of Raspbian Buster 10 on a pi3 b+.
Had someone a simmilar issue and know where I made an error?
As the next step im going to redo everything linke in the written gitub tutorial.
Thanks
That's just because the newer versions of OpenCV still have errors. Try installing an older version using: pip3 install opencv-python==3.4.6.27 . Then, re-run the script. Hope that helps!
@@EdjeElectronics Thanks for your answer and excuse my very late reply. I will test it soon.
@@EdjeElectronics after doing this the initial error subsided and was replace with--> AttributeError: module 'tensorflow' has no attribute 'GraphDef'
Great video. I am trying to send webcam video over my local network from the pi to a PC which should perform object detection and send the output stream back to the pi over the local network. I hope this will increase fps even after networking overhead. Do you have any tips or experience on how to implement that?
Hi everyone, I fulfilled each and every step successfully except the compiling one : "python3 Object_detection_picamera.py "
Once I had tried to compile it, raspi warn me about exceeding memory requirements, it says kinda : "Allocation of 5031936 exceeds 10% of system memory"
I have raspberry pi 3 model b+ and have 32 gb class 10 sandisk sd card to operate.
I dont know what is wrong and how can I fix it, pls suggest smt to improve my allocation strategy :(
You can ignore that warning, the object detection program should still run. Make sure to close all other applications when you run Object_detection_picamera.py so you can free up as much memory (RAM) as possible.
@@EdjeElectronics thank you soo much for your fast response, but furter more, it is rebooting raspberry pi automatically. so can't move on with it :(
The exact error before it reboots itself :
W0723 02:39:30.401308 1996098256 deprecation_wrapper.py:118] From /home/pi/tensorflow1/models/research/object_detection/utils/label_map_util.py:137: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.
tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 5660928 exceeds 10% of system memory.
EDIT: solved. I replaced my power source (I used to use apple charger before) with samsung fast charger. I don't know how but it affects the performance. In my project I was using raspberry on an unmanned aerial vehicle, once I feed it directly from Lipo battary of the drone using BEC voltage regulator, It worked fine too interestingly. I hope It helps.
I have fast the same problem, only there are some other deprecated functions from tf and after that i can't allocate also....
WARNING: Logging before flag parsing goes to stderr.
W0731 12:40:06.483510 1995642432 deprecation_wrapper.py:119] From /home/pi/tensorflow1/models/research/object_detection/utils/label_map_util.py:137: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.
W0731 12:40:06.636931 1995642432 deprecation_wrapper.py:119] From Object_detection_picamera.py:82: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.
W0731 12:40:25.970835 1995642432 deprecation_wrapper.py:119] From Object_detection_picamera.py:88: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2019-07-31 12:40:41.656910: W tensorflow/core/framework/allocator.cc:107] Allocation of 11059200 exceeds 10% of system memory.
2019-07-31 12:40:41.672656: W tensorflow/core/framework/allocator.cc:107] Allocation of 11059200 exceeds 10% of system memory.
2019-07-31 12:40:41.779372: W tensorflow/core/framework/allocator.cc:107] Allocation of 8640000 exceeds 10% of system memory.
: cannot connect to X server
Hi, just wonder if you are able to solve the allocation exceeds system memory issue? i am facing this issue and I closed all irrelevant applications before running the Object_detection_picamera.py command but still cannot. Apparently it is just a warning, but picamera still couldn't on for object detection:(
KUA MEI SAN Hi, after I replaced my power source (I used to use apple charger before) with samsung fast charger, it affects the performance in an interesting way. In my project I was using raspberry on an unmanned aerial vehicle, once I feed it directly from Lipo battary of the drone using BEC voltage regulator, It worked fine too.
All related suggestions on the internet is about changing power supply or its cable. So, please try to feed your device with better alternative.
I hope It helps. Let me know the result.
Hi, Edje. Thank you so much for your awesome tutorial. Can I ask a question? I am a Ph.D. student in biology. I plan to use object detection to count the dead or living insects under some treatments. I want to monitor the survival rate of these insects under some treatments. So, do you have any recommended tutorials with Raspberry Pi? Thank you
Hey all, while executing the statement 'python3 Object_detection_picamera.py' in the last step (detecting objects) I got the following messages. Does anyone know what went wrong? Thanks a lot:
pi@raspberrypi:~/tensorflow1/models/research/object_detection $ python3 Object_detection_picamera.py
Traceback (most recent call last):
File "Object_detection_picamera.py", line 23, in
import cv2
File "/home/pi/.local/lib/python3.7/site-packages/cv2/__init__.py", line 3, in
from .cv2 import *
ImportError: /home/pi/.local/lib/python3.7/site-packages/cv2/cv2.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8
same problem
you have to install opencv version 3.4.6.27 and uninstall the one you installed
pip3 install opencv-python==3.4.6.27
@@habibabassi9876 Hi Habib, thanks . I uninstalled opencv and installed the version 3.4.6.27. But I still get the same error.
@Beautiful Greece Thanks to you and @Habib Abassi ,it works. It's amazing :-)
@Beautiful Greece thx...i did this and installed openCV 3.4.6.27,,,it works fine
This is awesome. Worked like a charm. The best guide ever. Thank you and Keep contributing.
pls help me!!!
everytime i run this code my raspberrypi 3b rebooting all the time! i have no idea why?? pls help guys
I can't event think about the number of errors that I would run into by trying this.
Check the updated written guide, it's much less error-prone 😄 github.com/EdjeElectronics/TensorFlow-Object-Detection-on-the-Raspberry-Pi
If I follow this video would it work with the raspberry pi car through the camera once I complete all these steps?
I have a question. At 11:50 he said that you have to save and exit the menu but there are different options to save it (Dos and Mac). Which one do I have to use?
Is it possible to photograph objects according to the direction of movement and to save the images differently depending on the direction?
Thanks for this, really helpful. Keep up the great content!
Hello! Great tutorial. Do you have a proyect that recognized when a person use a mask covid19 or no? I use raspnerry pi 3b model. Thank you
Hello , thank's for the tuto , it seems that the object detection script will be running on its own otherwise the raspi will stop , could you specify cpu and memory consumption , thank you
Got an error at very end saying there's no matplotlib after using updated step 2. Fixed with "pip3 install matplotlib" which at first did not work but in error it mentions 2 other things to install which can be done using "sudo apt-get" followed by their respective names. Do "pip3 install matplotlib" again and it should work
Awesome tutorial! Do you think this can work on a flying drone? My drone has a Picamera and I wanted to install this software, could you give me any tips/advice on making this software run smoothly during flight?
I have to detect and sort three color based objects my question is how can i connect a servomotor with this in order to detect and sort in accordance with the relevant color match?
That's definitely do-able! However, it might be easier to use OpenCV to determine the color of the object. This link might help you: www.pyimagesearch.com/2014/08/04/opencv-python-color-detection/ . Also, check out my pet detector camera video, which shows an example of how to trigger actions if certain objects are detected: ua-cam.com/video/gGqVNuYol6o/v-deo.html
please help me out im stuck at
Traceback (most recent call last):
File "Object_detection_picamera.py.2", line 23, in
import cv2
File "/usr/local/lib/python3.5/dist-packages/cv2/__init__.py", line 3, in
from .cv2 import *
ImportError: libQtGui.so.4: cannot open shared object file: No such file or directory
I had the same problem, Edje helped by saying the solution to the problem is to try use the command "sudo pip3 install opencv-python". Hope it was the solution:)
Worked for me: sudo apt-get install libopencv-dev python-opencv
i have a problem with my screen....I got this to work and massiveee thankss to your tutorial here!! but im using a 3.5" screen and the tab is too big and i cant change the size. is there any way to solve this? ive tried changing the framebuffer_width and length and still nothing (ps thats 320 and 280)
Ty!
Hello thanks for the tutorial, i have a question. I'm able to use this project on raspberry 2 with raspberry pi cam v2. However, i can only get 0.8 fps. How can i make my fps 60? Do i need to buy other versions of raspberry such as 3,4?
Hi, you're welcome! Please check out this video to get an idea of what FPS you'll get with various configurations of the Pi 3 and 4. You won't be able to get 60FPS, but you can get 30FPS with a Pi 4 and Coral USB Accelerator! ua-cam.com/video/TiOKvOrYNII/v-deo.html
Is there an endpoint that you can query to know what the camera is detecting? This would be extremely useful for creating an alarm system in home assistant.
You did a great job on this video.
Hi. During I execute the "make" command in the protobuf directory i get this error of
"libtool: error: 'google/protobuf/compiler/csharp/csharp_generator.lo' is not a valid libtool object"
I need your consultancy since I couldnt figure out what to do.
Hi Evan,
I have managed to install all the steps up until point 15:51 in the video . After typing python3 Object_detection_picamera.py I get the messages listed below:
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
File "Object_detection_picamera.py", line 23, in
import cv2
File "/home/pi/.local/lib/python3.5/site-packages/cv2/__init__.py",line 3, in
from .cv2 import *
ImportError: numpy.core.multiarray failed to import
In truth, I have no idea what any of this means. Any assistance with this would be greatly appreciated.
I am using a Pi NoIR Camera V2 on my Raspberry pi 3 Model B+.
Thanks in advance, hoping to hear from you
Hi
i want to know how to send signal digital to pin on rassberry pi when it find objects
Question
If I want to read a dog. How do I send a digital signal in the GPIO?
For example: If I see a dog that lights up a led
Hi Nicolas, I wrote a script that will play a video if a certain object is detected in the image. You can use it as an example for how to perform some kind of action (like turning on a GPIO) when an object is detected. Here's a link to the script: www.dropbox.com/s/xlpiup19vu8qqla/Object_detection_webcam_notification.py?dl=0 . Look at lines 132 - 176.
Great work, I'm really grateful for what you share with us
Hi Edje, what aspect of this project do you think can be improved on? For example, could the framerate be improved by changing the code (other than the camera resolution)?
You could slightly increase the framerate by reading the camera in a threaded way. At the moment, the camera is read and the detection is performed in the same loop. Not sure how much fps you'd gain though!
@Michail Vanis I did try creating a separate thread for the camera as described here: www.pyimagesearch.com/2015/12/28/increasing-raspberry-pi-fps-with-python-and-opencv/ . However, it didn't play nicely with TensorFlow! I got lots of errors. I think TensorFlow just requires too much processing bandwidth for doing multithreading.
@Immanuel Ng There are lots of ways to improve this project. You could train your own model that has more accurate detection (the pre-trained SSDLite-MobileNet I'm using here isn't very accurate at identifying objects correctly). You can add more functionality to it, like causing a robotic hand to wave when a person is detected in the image.
As far as improving framerate goes, there may be some small improvements that can be made to the code, but I don't think they will make a significant difference. You can always keep an eye on the TensorFlow model zoo ( github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md ) to see if they release any models that are faster than SSDLite-MobileNet.
@@EdjeElectronics Interesting! I've got a handy Python class for doing the camera stuff here: github.com/mikevanis/CameraController I might have time to give it a go in a few weeks.
hi sir edje thank you so much i use this tutorial and it is working any recomendation or suggestion on how to improve the frame rate?
Hey Allanrey! Unfortunately, there aren't really any good ways to improve the frame rate. You can try looking into an Intel Modivius NCS (a USB device that increases processing power: ua-cam.com/video/p8o284gBSqo/v-deo.html ), but they don't work with TensorFlow models.
Hi, everything look good but when I run the Object_detection_picamera.py raspberry is freezy out.
I waited half of houre.
Wow this is some crazy timing, I'm sitting here trying to get this set up right now and what do I find? Thanks!
Hello! Your guide is really helpful. May I know if it is possible to run the detection on 2 cameras i.e. Pi camera and USB webcam at the same time?
I wanted to share an issue that I had in case someone else has the same issue. When you install pip3 install tensorflow, no matter the version, it does not compile protobuf automatically, you still have to do that part yourself. I thought it did and I kept getting bash protoc command not found. It was straight twisting my brain around.
I saw the same thing
Hi, thanks for this set up, is there any way too add intel Movidius to accelerate the FPS?
Hi Jassiel, I haven't tried using the Movidius NCS. It may be possible, but it takes some work. Take a look at Mark Jay's video series on using the NCS with the Pi (ua-cam.com/video/p8o284gBSqo/v-deo.html). Note that he shows how to use YOLO rather than TensorFlow. This page shows you how to compile TensorFlow models for the NCS: movidius.github.io/ncsdk/tf_modelzoo.html . Please let me know if you get it to work!
Edje Electronics thanks!! I’ll let you know
Grat tutorial thank you. Hopwever when using wget the coco models i get the follow regardless of which model i use
2019-02-16 19:03:27 (14.7 MB/s) - Read error at byte 5655576/51025348 (Connection reset by peer). Retrying.
thanks. it works fine. although i don't like the barely 2 fps. Is there any other small hardware that can give better performance as i am trying to work on an image processing project? thanks
Hey!
What do you think is best to detect playing cards? Your Open CV algorithm or your tensorflow model?
Thanks in advance!
Hello, thanks for the good question! That's a tough one. The OpenCV algorithm is much more reliable and accurate than the TensorFlow model. However, it won't work if the cards or overlapped or if the cards are against a non-uniform background. Ultimately, I think the TensorFlow model is better because it provides much more flexibility in the application, and its accuracy can be improved by giving it more training data.
Actually, I just finished training a YOLO model that works very well at identifying cards. I'll be posting a video on it this month! I based the work off this guy's excellent video, which shows just how good the machine learning model can be: ua-cam.com/video/pnntrewH0xg/v-deo.html . This is the approach I'll be using for my blackjack robot!
Thank you very much! Looking forward for the next video then ;)
And thanks for your amazing content
is there any chance of getting the text as a voice feedback after the object is detected?? because i would like to use this as a project for the blind with detecting objects and hence alerting them with a voice feedback..
Nitesh Bharadwaj try integrating a text to speech library like so: pythonprogramminglanguage.com/text-to-speech/
Can I implement my custom object detection using Raspberry? Because you said that your previous video about object recognition is for Windows.
Yes you can! Please watch this video starting at 16:20 , it shows how to set up your custom object detection model on the Pi.
Edje Electronics Thank you very much for your answer!
@@vladcalin4008 hey, were you able to do it?
Hi, thanks for great tutorial i've manged to install evereything. And it is working! But at first I faced the problem of overheating (small termometer icon appeared on desktop and raspberry pi just hanged) when i run python3 Object_detection_picamera.py --usbcam (I use cheap usb camera). I've placed a cooler on top of pi and the problem with temperature was solved (no termometer icon anymore). But now Object_detection_picamera.py just hangs my Pi, after a minute or two of work. Can you advice me how can I debug the issue? I'm using Raspberry Pi 3B (without plus) and lateset Raspbian.
How much memory is left on your sd card? Try issuing df -h in the terminal , maybe that's causing problems although unlikely... And maybe you need to download fwusb ( not v sure of the name) library so that usb camera runs well , tge library was mentioned in a previous comment it's for supporting external usb cameras .
For me , it made the code ran perfectly after that , I only had problems when I tried to turn it off though .. I'm also using raspberry pi 3 B+ model
fswebcam is the correct name
Google it 👍
Thanks for the video. I woke up this morning, and thought to myself "I wonder if my Pi can run tensorflow?" And here I am. One question, why didn't you use make -j 4 to speed up compile time?
Thanks! It makes for a good weekend project 😄. It seemed to fail more frequently if I used the make -j4 option, so I decided not to use it for the video. Fortunately we don't have to build it anymore and can just use "sudo apt-get install protobuf-compiler" to install it now!
I am trying now to follow the guide. When installing the .WHL file, the PI reports error: .WHL is not a supported wheel on this platform. Any idea why? I am using PI 4, and tried latest version, as well as 1.8.0
Really Good video. I can’t wait to try this out on my own Rpi. Thanks for sharing!
Hey can you please tell me is it possible to convert the detected object in to speech. That will help as i am trying to this as a final year project.
At the last step when trying to run the code to activate the camera, I get an error saying ImportError: No module named 'matplotlib'
Any ideas what to do here?
For step 2 I used the shortcut Edje Electronics mentioned in the comments by issuing "sudo apt-get install libatlas-base-dev" and then "pip3 install tensorflow" , I wonder if that has anything to do with it?
Use "pip3 install matplotlib" which at first did may not work but in error it mentions 2 other things to install which can be done using "sudo apt-get" followed by their respective names. Do "pip3 install matplotlib" again once you've installed the other parts and it should work
@@acrippledmango Thank you man, that fixed the issue! I appreciate the help
@@sikhallday hey man it would be kind if you can please tell me in which directory the matplotlib is needed to be installed? and the other two things.
Can we convert the.. same output into speech using tts
Thaks for the video it works perfectly, i have one question, the fps in my raspberry is 0.7 sometimes it's 0.8, is there a way that I can rise like yours maybe 1 or 1.2 ?
Hmm, what Raspberry Pi model are you using? Mine is a Pi 3B+, which is the fastest one available. That might be what causes the difference. Also, make sure all your other applications are closed while you run the script.
@@EdjeElectronics I have already fix it
Juan David Barrero L may i know how you improve the fps sir?
@@strangedean1 sorry to answer so late, I just change the resolution of the camera, and I try with the latest version of protobuf and tensorflow
I have some problems with line51
With the word utils,
there is no module named that.
Did I miss a step or something?
This is awesome! Many thanks for this brilliant job.
Thanks a lot . I set up the system on my Raspberry PI and it works after some little problems.
all the best regards
Thanks for the video!
So could I train my own model on a certain detection model from the model zoo (ssd_resnet) and then run it with a different detection model (ssd_mobilenet)?
No, the models aren't interchangeable like that. When you train a model, it generates a frozen_inference_graph.pb file which contains the detection graph for the particular model you trained for. When you run a program (like my Object_detection_picamera.py) script, it loads that frozen_inference_graph.pb and runs it. So in other words, whichever model you trained is also the model you run. I hope that makes sense!
@@EdjeElectronics great, I think now I understand how the models work, thanks for the quick answer! :)
Hi everyone. Excellent tutorial. I desperately need help. I am following all the steps of the tutorial but when I try to train my own model I always get an error related to the tf.contrib library regardless of the version of tensorflow. Do I need to make any additional changes? I would greatly appreciate a help. (I have been using TF 1.14, TF 2.0 and TF 2.1 +Python 3.7.4 with the same result)
Hi. Do you know how feed a pre recorded video instead of the live feed from the camera. I wanted to test the speed first before going live. Thanks :))
Hey Dominic, check out the "object_detection_video.py" script I created for my TensorFlow on Windows tutorial. Here's a link to the code. You might have to tweak some things to make it work on the Raspberry Pi. github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10/blob/master/Object_detection_video.py
Hey Edje Electronics, thank you so much for your video. We followed all the steps and are able to get the object detection working successfully on our raspberry pi 4. We are actually trying to get an audio output for the object our pi camera detects. For example, if a chair is detected by the pi it should give an audio like "there is a chair in front of you". We are trying to do this for a project and it would be really great if you could give us some tip for it. Thanks.
Hello sir could you please help me to run it on my raspberry pi 4 too??
Hey did you get the solution..please help
@@AliKhorshidi Hello Ali. You follow the steps?
Can you post some errors?
Hello I need audio feedback too or the detection coordinates. Have you some advances? Thank you
@@dembogurski I can help if didn't figure it out
is there any ways to convert the text obtained in the box into speech ??
how can I intergrate your Object_detection_picamera with .hi5 model?
Edje Electronics can i use tensorflow as computer vision?
Hi thanks for this tutorial, I am getting a error "ImportError: libImath-2_2.so.12: cannot open shared object file: No such file or directory". What could be the issue??
Same here getting stuck here