Tello drone and computer vision: selfie air stick
Вставка
- Опубліковано 9 лют 2025
- The Tello drone is a lot of fun by itself. But it is also the opportunity to apply some well-known computer vision techniques to expand its capabilities !
Github: github.com/gea...
I want to thank all the people who wrote and shared the great libraries/programs I used here :
github.com/han... : DJI Tello drone controller python package,
github.com/CMU... : Real-time multi-person keypoint detection library for body, face, hands, and foot estimation. This is an amazing library !
github.com/Ubo... : Ubotica wrote a code for the Tello to follow a color ball. Instead of starting from scratch, I used his code. It makes me saved a lot of time for UI !
Music credits:
The Place Inside - Silent Partner • The Place Inside - Sil...
Cello Suite #1 in G & Your Call : Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons...
How does someone dare to hit dislike button in such video? This project is absolutely amazing! 😲
you probably dont care but if you guys are bored like me during the covid times you can stream pretty much all of the latest movies and series on InstaFlixxer. I've been binge watching with my brother these days xD
@Ian Benicio Definitely, I have been using instaflixxer for years myself =)
I've watched your other videos too, and as a Vision professional myself i'll say this....you sir are on a whole different level...these combination of ideas of yours are Classic
Thank you sir ! I really appreciate your kind comment.
Using simple tello drone to acheive that is amazing!!
as someone who learn machine vision, watching your video feels like when a painter discover bob ross for the first time. beautiful!
Thank you for your nice comment !
@@geaxgx1 How do you get the video so fast? I am on windows, and have only been able to get the video at very slow fps with lots of delay
@@ShredderJT The fps you can get depends on your GPU. I was using a gtx 1080ti.
Amazing video! I unfortunately am not good at programming, but its awesome to see the capabilities of tello extended without any extra apps
Really amazing!! Love it
You are genius, incredible utilization of the tello
I'm absolutely amazed. This opens totally new horizons. Thanks.
Really love this video and the codes! You demonstrated perfectly what can be done even by a toy drone.
Impressive job!
Thanks for sharing your work and a bit of behind the scenes! 👍🏻
Excellent project! Thank you for sharing.
Really impressed by your project!! Keep up the good work, subscribed to your channel as well.
this video is so well explained I finally can picture now how to create an object tracking algorithm!! thank you so much !!
Man, this is so cool! Thank you for sharing!
One ML engineer to another, dude, I am not worthy, great job!
Just Amazing! Congratulations, impresive project!
Fantastic work! Thanks for sharing. This is definitely going on my list of projects to try out
Congratulations! It's so cool man, I've loved it.
Definitely inspiring.
Awesome! Very good idea & video. Impressiv!
Congratulations ! great job, very impressive
genius. really great ideas and thanks for the inspiration and the code!
So inspiring! Thank you, Sir!
This is Amazing, well done and thanks for sharing!
Amazing project... Simply Fantastic!!
Very slick!! Nice work.
This is so cool 😁😁😁 nice work!!
Nice to see you here, Edje !
That is huge! Thank you!
thanks a lot for code Sir! I will teach kids on your examples!!
Wonderful job sir 👍👏
smart project sir...i was seriously looking for such,thanks for sharing.
this is a great project! great job! open pose looks really impressive. ive been wanting to try this in conjunction with a vr helmet to have a realtime 3rd person ar camera with a mix of computer vision and human input to control it. imagine adding cinematic angles, pans, slides, and zooms that could be triggered and strung together!
Amazing work, building on it
I love you. :)) You have some of the coolest programming projects out here.
Good job, regards from Mexico
love all of this, love you sir
Wow!! Excellent!!
Great work!
That was seriously impressive. Thanks. :)
This is just brilliant! I have the drone already, not used too much, but this would be perfect. If this is able to run on RPi zero W or something like that - that would be amazing...
Thx! Unfortunately, to run Openpose, you need something more powerful than a Pi.
What about a cellphone? Like snapdragon 855?
Well, you can run a pose estimator on a phone (if it is not OpenPose, you can use another one like Posenet maybe lighter?). But I am not sure it can run fast enough to use it to "comfortably" control the drone. Imagine you can only decode 2 frames/sec, you will need to limit the max speed of the drone to a low value otherwise the chance to hit a wall is too big. In any case, you will need to tune the settings of the PID controller accordingly to the decoding rate.
this is solid! good job!
Great work
When you search for a project with the drone and openCV and a guy did all the hard work :D
Wow! excelent job :)
Subscribed, you've inspired me to do something similar, thanks!
Thx, it is rewarding to read that !
This is amazing!
Thank you for sharing... nice music ;)
Nice! one thing you might try, using the tello to take images for photogrammetry. I might buy one and give it a go myself.
Thx! I have already tried photogrammetry with the tello. It was outside and using frames from the video stream, for a poor result, probably because of the low resolution of the frames. I will try again but this time by taking 5M pictures.
@@geaxgx1 Best advice for photogrammetry is to take as sharp focus pictures as you can. I live in the UK so there's a lot of overcast weather so its almost perfect for photogrammetry (avoid hard shadows). Having a programmable drone that can take images around an object seems like a perfect use-case though.
Incroyable!
What you did is amazing
Thank you for sharing
Really cool, i want to make my own now
Well done !!!!😄
You have done an excellent job it's amazing ✨😍🙌
WOW, Amazing!!
Woah!!! 😍 Amazing 🤩🤩
Amazing project!
great stuff! subscribed!
Excellent!
Epic!!
i love computers, tech, and quadcopters...
do you think you can make a video or explain somehow how to do all that? i mean how to install all what is needed on the tello to do these hands tricks? thank you :)
You don't need to install anything on the tello itself. But you need a PC with a powerful nvidia GPU and install a few packages/libraries as explained in the github : github.com/geaxgx/tello-openpose
cool, gonna try it.
Absolutely awesome!!
Fabulous!
This is the most awesome project I have seem!!!
Hi there, saw your video on Reddit and it prompted some questions that I'd like answered if you'd be so kind...
1. Can it be programmed to focus on one target?
2. What happens with multiple targets in the room?
3. Can more expensive drones be programmed this way?
If there are multiple bodies in a frame, the program currently selects the one which appears to have the bigger size. The size is evaluated by measuring the length of some segments of the skeleton. Often it will be the body which is the closest to the tello. If 2 bodies appear to have close size, the tello may "oscillate" between them. To make the tello focus on one particular target, we could use face recognition, so that the tello can distinguish between several faces. Face recognition alone wouldn't be enough because it works well only when the face is facing the camera. By using tracking in complement, we should be able to keep the focus when the face is not facing the camera anymore.
I am not very familiar with drones but I know that some more expensive drones can already do that kind of things (body tracking and gesture recognition), and in a more efficient way than mine (because their program is embedded in the drones and they certainly don't have the processing power of a gtx 1080ti on board :-)
@@geaxgx1 thanks for taking the time to respond. It was a great video with an interesting concept
You're excellentl!
impressive!
What an awesome project. I wonder how it will behave when it sees more than one person?
Thx! If there are multiple bodies in a frame, the program currently selects the one which appears to have the bigger size and ignore the others. The size is evaluated by measuring the length of some segments of the skeleton. Often it is the body which is the closest to the tello.
Great work! With RTX 2070 Super GPU I am only able to achieve 30 fps before running openpose and 10 fps after. The frame rate seems to cap at 30fps and 10 fps respectively and does not change even if i downscale/upscale the images.
have you tried this project ?? i am having issue in the last stage, after running the code , window appears that shows video ,fps ,etc but with in a minute it stucks. no errors it just stucks
Awesome
Hello, can I please know how to make openpose work, everything is just fine except that..
Very cool.
Amazing. Truly. This could replace cameramen. Lol.
Everything is perfect, but i want to know which hardware inside drobe is being used for the coding, or for the motion controller... (Means which hardware is used for communication of drone with openCv or OpenPose )
Thanks! I don't know what is precisely the hardware inside the drone, but actually I don't really need since I rely on the Tellopy python package that manages the communications between my program and the drone (via wifi).
@@geaxgx1 okay, i know , python or languages made communication between operator and drone, but the thing is, in which hardware has been this code is being implemented ? ... But i am expecting u don't know, but can u prefer from where i will get my answer ? Because i really need to know the hardware for my project that contains this coding ..
@@Kkkkkkkkkkkkkkkkk-g8p If you ask what hardware is inside the drone, this page says the tello uses a Intel VPU Movidius Myriad 2: gobot.io/blog/2018/04/20/hello-tello-hacking-drones-with-go/ Not sure if it answers your question :-)
@@geaxgx1 i will read it, Thanks a lot for u time and u told what u know..
Sir can you tell how to connect the tello drone to the project and sir your project is wonderful I loved it.
Thanks ! On startup the Tello configures itself as a Wireless Access Point. By using the Tellopy package, the program connects to this access point.
@@geaxgx1 Sir can you tell me it a little more briefly please.
Sir it is also showing a problem file:///C:/Users/rishi/OneDrive/Desktop/Screenshot%202021-01-07%20094741.jpg
My tello drone keeps moving to the left even though I'm not commanding it to do anything. How can I stop this?
amazing!
Amazing
Respect sir.
Sir can I get the source code i am working on this project in which drone detect the face with pose estimation and follow and using gestures it does some tasks it will really help me in FYP if you share the source code
Wow, great project! I am thinking of buying a tello drone as well in order to learn programming that way. I already have a basic knowledge of coding with python and would like to start an ambitious project for myself. Therefore I would appreciate, it if you could answer me following question: Would it be possible to go outdoors and make the drone follow you while running for lets say 50 or 100 meters recording you from the side? I'm thinking of wearing a colored cap for example in order to make the drone follow me by flying sideways and recording me running.
Thx! From my experience, detecting a color to follow an object in a image is not reliable when you don't control the lighting conditions. Depending on the sun, the clouds or the shadows from the tree, the original color can look very differently in the image. Using a person detector is probably a more robust option.
Hey, man! It's a great job. Thanks. I'm trying to figure out, can i program drone to fly every 15 minutes from the base, check for small trash on the table, pick it up if there is a trash and throw it to the garbage, and then return to base for charging?
Hi. This is super amazing and inspiring. I am trying to build openPose for python on mac and really struggling. Also when compiled openPose and ran using CPU_ONLY mode, FPS is very poor (0.4) on the latest MAC. The FPS in your demo is 25+. Was there anything different than the specified configuration in the openPose github to get it to 25+?
Thx ! I was using a powerful GPU (Gtx 1080 ti) in order to get such FPS. If I had to make this project today, I would not used Openpose, but a recent model like Blazepose or Movenet that can run much faster on a CPU.
@@geaxgx1 thanks. Just tried blazepose mediapipe sample and it is way better. Getting around 15 FPS. BTW this video is really good. Have shared it with my friends too. Especially the Morse code touch was brilliant.
Dear sir i installed all the libraries and openpose is in running condition i also run the examples but how will i open your project file of tello_openpose_master in openpose
Congratulations and nice work!
I'm wondering about:
- Two people are present in the same frame, what's the behaviour of the drone?
- How can you avoid collisions with walls eg behind the drone looking at you?
Thanks!
Thx !
- Two people are present in the same frame, what's the behaviour of the drone? The drone only follows the one which appears bigger in the image. It means that if the people are moving, the drone can switch from one guy to the other one, which is probably not the behaviour we would like.
- How can you avoid collisions with walls eg behind the drone looking at you? I can't ! The Tello has no sensor to "see" behind or on the sides. So the guy who is "piloting" the drone has to be careful :-)
@geaxg1 Hi. Is there a more powerful drone that we can program it with this SDK like you did? For example a drone that has sensors, and much bigger than this ? Great job!
can you guide me to how you did the pose recognition? did you use ai for that or jsut check which points are connected to each other and in want locaion? plesae someone answer if you know
I am not using AI for pose recognition but just do computations on the landmark locations. For instance, to determine if hands are crossed, I can check if the distance between hand landmarks are below a threshold.
I want to try when I have the Tello.. 😁
Wonderful project! You earned a new subscriber! :-)
Btw - Do you think this project can be done through Google Colaboratory in case the GPU poses a problem?
Thanks ! It is a good question but unfortunately I don't know if it is possible. The tello drone needs to communicate with a process that is on the local tello wifi network, and this process would also communicate with google colab.
Is there any drones that more powerful (big, has sensors etc) than this with SDK support? How do you overcome the stabilization ? Great job!
Thx ! I am not an drone specialist, but yes there are bigger drones that have their own SDK support, like the DJI Mavic (developer.dji.com/) or the Parrot drones (developer.parrot.com/). The Tello is more like a toy but it has its own stabilization mechanism and it is working great ( I don't have to care about it).
Hello sir,
Thank you for sharing, I want to ask you which graphic card are you using for training the models of Deep learning ???
can u share the process of how to do the face detection with tello drone...
In this project, I don't directly do face detection with the drone. Instead I rely on pose estimation to localize the different parts of the human body, including the head.
Would I be able to measure your distance without using OpenPose? I have a project that is to follow any object that has been classified on a CNN. The only idea I have is whether the detection box is large or small, which would show if it is approaching or moving away from the drone.
You have probably observed that the size of the bounding box can vary a lot from one frame to the other even if the object is almost not moving. So you should probably use a moving average of the box size to decide if you are approaching or moving away.
unbelievable! a
I can only repeat: absolutely amazing - Very creative and impressive solution
Do you have any experience in Xcode/Swift as development environment in this area ?
(And thank you very much for the additional info at Video)
Thanks ! Ah no, I have zero experience on Swift.
i see for tracking is very good realtime and fast, but how if in front drone stand 2 person, its will lock both person or first person find?
In the current version, if it detects several persons, it will consider only the "biggest" person in the image, the one which have the largest shoulder width for instance. It is not a very satisfying solution because if 2 persons are close to each other, the focus may oscillate from one person to the other depending on their movement and their position relative to the drone.
Great work!! How are you classifying the pose ?
Very easily, by using the coordinates of the keypoints given by Openpose. For instance, to take a picture, I cross my hands under the neck. To recognize this pose, I just need to check that the keypoints for the right wrist, the left wrist and the neck are close to each other. To ask for an immediate landing, the left wrist keypoint needs to be close to the right ear keypoint. Just a bit more complex to make the drone move (forward, backward, left, right), because I need to look at the angle between pairs of keypoints.
I am lucky here because I just have 6 poses in this project. If I had more poses to recognize, I would probably need to train a classifier.
@@geaxgx1 correct me if i am wrong here , you are calculating the euclidean distances and checking if they are are below or above a threshold ,you are not using a classifier.
Yes, exactly ! Euclidean distances and that kind of stuff. No need for a classifier here :-)
@@geaxgx1 thanks a lot!!
Amazing project ! Is it possible to move the drone camera from 0° to 90°?
Thx! No, on the Tello, the camera is fixed.
Hi there , very good indeed. I am
Trying to install everything on windows . Python 372. PyAv gives a bunch of syntax errors
Though . Struggling with ffmpg . Is it necessary and how do I go about it ?
I am sorry I can't directly help you on windows. But pyav is just used to decode video frames sent by the tello. There may have alternatives. Maybe you can take inspiration from these examples : github.com/hanyazou/TelloPy/blob/develop-0.7.0/tellopy/examples/keyboard_and_video.py (which relies on mencoder), or github.com/dji-sdk/Tello-Python/tree/master/Tello_Video (which relies on h264decoder). Good luck !
excuse my temporary ignorance. where would i start in order to do this to my tello? i purchased it for really this reason but had found no one but you successfully completing this awesome task. i am a fast learner but just not sure where to start on this. i figured instead of combing the internet endlessly, why not ask the person with the most experience :) ...you
thanks
-mike
First, you can begin to play with Openpose. Install it from here : github.com/geaxgx/tello-openpose You need to build it with the python API. At this point, you will have an idea on how fast your GPU can run Openpose. If it is too slow, you probably will have hard time to pilot the Tello like in my video. Secondly, you can play with the TelloPy python package (see the link on my github repo: github.com/geaxgx/tello-openpose). With this package, there are some examples you can try : for instance, with keyboard_and_video.py , you can pilot the Tello with the keyboard and display the video stream from the Tello camera. Third step, you can download my repo (same link as above), install the needed python packages, and try to run my scripts. If your GPU has not the same power as mine (gtx 1080Ti), you may need to tune the parameters of the PID controllers. It is a trial-and-error process. Store the glassware in a safe place before your first try :-))
Can you please make a video explaining about how to incorporate these in a tello drone. Because im not very familiar with OpenCV and it is my final project
Actually you don't need to incoporate anything in the drone itself. You run a program on a computer which interacts with the drone.
@@geaxgx1 thank for the reply, but, there should some interaction, so is it via Wi-Fi? Can you please make a video of the full process. It will really help me or anything else which could guide me through the processes necessary to implement it.
i think you need to make an app for us ? i need this so it can track myself on my bike rides in the summer and i don't feel like messing with coding
Currently, it would not run fast enough on a smartphone. But if you carry a powerful PC on your bike, it could work :-)
@@geaxgx1 Which CPU/GPU do you use for the calculation?
I got good news for you , it is possible and coming soon in beta stage now ua-cam.com/video/S8WNGTORqJg/v-deo.html
@@xquadrat1234 i7 7700K and GTX 1080 Ti. This is the GPU that may have the bigger impact on the behavior/usability of the program as it conditions Openpose speed and consequently the frames/second you can get.
Hi. Your work looks cool and it's fantastic and myself working on something like this that requires streaming video from Tello drone and I am able to do it and then after two to three attempts of streaming it stopped. I am not able to get the video stream and it gives me errors like corrupted data log and timeout errors. It's troubling a lot can you tell me what kind of error it is? and how can it be solved? It would be so great
Thank you
Sorry, I don't have enough information to have an idea of what can be wrong. Are you also using the TelloPy library (github.com/hanyazou/TelloPy) ?
Hi sir, how do you connect the tello to the laptop so that it can be use to code with phyton, hope you can answer it asap
The tello itself is a wifi access point. You simply connect the laptop to the tello wifi network. Once connected, you can use the TelloPy python package to manage the communications with the tello.
geaxgx1 thank you sir, i have another problem, i cant get access for the video streaming through the tello,so,may i ask.. how to get it?