Raspberry Pi Robot Arm With Computer Vision + Image Processing Pics
Вставка
- Опубліковано 12 вер 2024
- The robot arm controller is a Raspberry Pi 2 Model B. The Servomotors are Dynamixel AX-12A. There is a Raspberry Pi camera module mounted on the top for image processing.
The Computer Vision algorithms applied here are Edge Detection, Binarization, Pixel Expansion, Labeling and Object Extraction. In this Video I tried to show how the robot see’s the world by adding pictures directly out of the Image Processing algorithms (I just added the coloring in the Labeling process). I also tried to sync the pictures to the superb music of the great artist “broke for free”.
Here's some further info on the thing:
I didn’t use OpenCV. The image processing algorithms applied here are all very simple. I wanted to write them by my own. Two important libraries which I used are pythons "picamera" and a library called "ax12". "picamera" provides an easy way to get greyscale pixeldata from the Raspberry Pi camera module. "ax12" is used for the communication with the Dynamixel AX-12A servos. I did write some code to make the servomotors move smoother (starting and stopping in a smooth sinusoidal manner). And then there was a bit of code to actually get the junctions into positions, which would allow the electromagnet to pick up the metallic things. In other words, this was about getting the thing to move correctly given some x and y values which were extracted from the image earlier.
Music:
"Night Owl" by Broke For Free
Love the Dynamixel servos just not the price tag!
Excellent calibration of servos according to the frame of the vision
Wow thats a very cool project and a very expensive way to manage spare parts around the table but why not. I totally loved the idea to have some sort of electromagnetig finger instead of a clasic mechanic claw. Congrats!
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
Excellent. Most satisfying video on Robotic ARM With RPI. Keep up the good work.
Thanks! :)
I see what you did there.
Congrats !! brilliant engineering ! very smart and simple solution for object classification
hi, this is excelent. Thank you for the showing that job. Could you please share or prepare a full system video for this? Or could you share the references which you got information to build this system
Dude awesome project .
congrats man , and please continue in your work .
Thank you! :)
I'm getting Short Circuit nostalgia from seeing this.
Awesome - loving the electromagnet arm
Thank you! :)
Extremely interesting and well made content. Subscribed.
Nice, Beside everything really like the way you expand the Pi Camera!!!
Thanks Israel Goytom!
WE DEMAND MORE!!! MORE SIMILAR PROJECTS!!!
wow. Thanks! I will try to upload some similar stuff. Though I can't promise anything. ^^
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
Nice man, I had doubts about using rapsberry pi to control servos but your robot gave me some ideas about how to make it. Now i need to control a 5 DOF arm robot with NEMA motors
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
what the hell, man. this project is the most awesome project of robotics in youtube. i have been tailing all robotics projects in youtube, and this is the BEST. it offers the simplest ever way, to learn CV robotics. why do u stop doing this project, man. u should make similar projects to this one!
wow. Thanks!
I'll try to make new stuff with this robot arm.
nice to hear that, man! i have been tailing all CV robotics projects in youtube for 2 years, but i didnt have courage to start, because they were all so complicated. this one is the simplest ever. and it even has many real life applications, further encouraging people to learn, coz this is useful. these dats, i am starting to learn robotics, by learning raspberry and CV because of you, man!
Thank you so much for giving this much of information
NIce! I sugget you to program a continously motion with acceleration and dec.
Hey Walter, That would be cool! Thanks for the comment!
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
@@JaydenLOL this project with code offusced is a fucking shet and then not has the robot arm.
I love it. Great work!
Thanks!
amazing! i will try to replicate someday
This video is perfect! good job
Excelent, Thanks for sharing
super awesome bro...
that's cool~
good project.
Awesome!
Well done
very indeed nice project!
awesome!
Excellent!
Excellente work.
Thanks Emir!
Excellent
Is this type of image processing possible with Arduino?
Pretty cool! May I ask why not integrate a cognex vision system?
Wow, just wow
Excellent work !! It will be more helpful , if you explain more details about this :)
Hey Vijay!
Thanks for the comment! Here's some explanations about the thing:
robotshop.com/letsmakerobots/raspberry-pi-robot-arm-with-simple-computer-vision
It's not much. But maybe you're interested.
Thank you, I will check :)
very nice ......
Pretty awesome!, Hi Tobias, I am making as well a robotic arm with 4DOF, I have a question. Its available using your library Json?, if you could supply it to me.
Also if the other library Ax12 is able for using with a TowerPro MG995?
Hi Andres,
Thanks! I am afraid the Ax12 library and the code I wrote can not directly be used with TowerPro MG995 servos. The Dynamixel AX12 servos are smart servos. They recieve instructions via serial bus. On the other hand, as I am sure you are aware of, the TowerPro MG995 servos use a PWM signal as input signal.
I a small doubt if you place any object it will pick or only the specific object
Could you share with me the technical design and software of this robot please ???
sweet man :)
Wow! Hi, can you tell me how the processing pics working? Based on ML? Thank you
Great project. I can see from the video that you assigned ID 3 to the wrist, ID 2 to the elbow, and ID 1 to the shoulder. What ID did you assign to the base please. Since the number is hidden, I can't tell. Was it assigned ID zero?
It was ID 4.
@@ElectronDust Thanks very much. Did you use the Pi GPIO 3.3v pin to power the electromagnet (on and off) or a separate power source (relay)?
Yeah, there's not enough current flow provided by the GPIO to power the electromagnet directly. I used a power transistor, but a relay would of course also work.
did you use python for this or c++?
plz make full video on this project please .....
Good project. I will soon be working on a simular project to pick and sort resistors. There may be problems here;-)
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
thats a weird and cool design
hey man, maybe you should take a look at SimpleCV. it will make your coding much easier, and will expand your projects possibilities. looking forward for your similar projects!
Oh, thanks for the tip!
This is most awesome!!! I would really like to build one of these do you have a repo/code/part list?
Did you do it?
I am working on a trash bot,which recognises trash using video processing,which processor will be compatible for video and image processing,iam trying to keep my budget as low as possible,so can u recommend me any less expensive processor,thanks
Hello Friend, how are you? I am a Brazilian research student and I am doing a similar project. Can you explain me how you made the connection between the Raspberry and the robotic arm?
nice arm! this one inspired me to do myself on my own
can u guide me while doing this project !!
how long it takes u to complete this project ?
Hey! Here's a blog post about this project (if you haven't read it already):
electrondust.com/2017/10/28/raspberry-pi-robot-arm-with-simple-computer-vision/
I can answer your questions if you want.
It took me about 3 months in total I think. I was working on weekends on it.
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
Hello, nice video, started looking into this for my raspberry pi as i am tired of programming with them. Do you think it is possible to just get a robotic arm and integrate some coding into it instead of building the arm from scratch?
Thanks
Hey, thanks!
Sure! Just be careful when choosing the arm. If the thing isn't providing any feedback on motor positions it will be hard to get it to move the way you want it to move.
Hey is there a way can I communicate with? Badly need teacher for my Project. Hope I get your response.
Do you need to boot into raspbian for this? I've done lots with arudino but not an electronics project with Raspberry Pi.
Yes, the Raspberry Pi is running on Raspbian in this project.
What would happen to mean coordinates in case the pixel expansion is not performed?
It should be using load cell, so the weight of item can be detected
Do you have any code or some sheet for teach how i can do this
i also interresting
Did you use a regular computer to make this thing work? Can I use a MacBook Pro 2013 to program this thing?
I used a regular PC to program it. Your MacBook Pro 2013 will work just fine. I programmed it by opening an ssh connection to the raspberry pi. But you also could edit the python files on your main machine and then just transfer them via WinSCP (or a similar software for mac.)
Electron Dust I see that your using 4 servos but i was wondering what type of servos are those?
@@clarencenikel_6420 Those are Dynamixel AX-12As. Very powerful, solid servos. Their a bit expensive, but it's worth it in my opinion.
How did you find the angle at which servo motor is rotated??
thank you！
hm ... no Problem! ^^
Would a cluster of pi help speed this up processing wise?
The image processing is the bottle neck when it comes to performance. Assuming that the processing load could be distributed evenly among the pis I see no reason not to think that it would speed it up. I never worked with a pi-cluster, so I am not clear on what conditions need to be met for the processing load to be distributed successfully among the pis. Image processing might not be as suitable (because not as easily attributable) as other forms of data processing.
How did you manage the communication?
please make full video on this topic ......
Need improvement but it is so cool .
How how do you get the arms to lock/brake after a rotation?
Hello, I hope that you askme my question, i have some dynamixel, but 2 have the led in red color and when I try moved this dont work, the others had the led in green color and work, my question is, why the led changed to red and dont work?, sorry for my english, I hope you understandme 😔
Hi,
Is your LED blinking red? If so, that means there was an alarm. You can figure out what kind of alarm it is by reading the 1-byte size error register at address 18.
the LED remains red, it is connected to a ROBOTIS controller card and the motor does not recognize me, the other motors do recognize me.
Hey bro how did you get x,y and z coordinates for the target position. And then u applied inverse kinematics?
this is better project ua-cam.com/video/vGCEwSspKgA/v-deo.html
How did you create it. Parts used please.
Will you plz share codding u have done in this robot making
where there are sold servo Dynamixel AX-12A
Hi Tobias,
Change capital "A" to:
from ax12 import Ax12
Only if u r using python 3.4.2 or later
Hi supra,
Should this be done because it is a programming convention?
+Tobias Kuhn Sorry. I used python 3.4.2. I had to modified. Ur is okay.
You could of done this with high torque servos. No need to buy expensive servos.
di you use kinematic for robotics arm
But nothing sophisticated though. Just the absolute minimum to make it move the way it moves in the video.
What is the cheeper price for motors "Dynamixel AX-12A"? It is absent at AliEpress and Bangood.
Dynamixel AX-12A are expensive. But they are really good and worth their money IMO.
Amazing !! dosnt look a resource hungry as open cv. Do you have a tutorial on this project by any chance >.>
I am sorry to tell you that there is no tutorial. I should write one.
There is some more info here:
robotshop.com/letsmakerobots/raspberry-pi-robot-arm-with-simple-computer-vision
But it's not a tutorial.
thank you so much, im working on a dual armed arduino/pi robot i think this would be great for.
What programming language did you use and what libraries?
Hello AUSSIE LAW,
I used Python. I wrote my own libraries for image processing and smooth moving.
But I did however use several other libraries including:
- picamera
- ax12
- spidev
Here's the source code:
github.com/T-Kuhn/ScrewPicker
hi i went to ur site github but i didnt understand wich code should i use
Help me with the code love your work
How did you convert pixel coordinates of objects to real world coordinates of objects ?
You must use a homography matrix to map the pixel coordinates to real world coordinates
could i see your publication
Can I use another servo motor using the library that you wrote?
It has to be a dynamixel servo. They use a data bus to communicate.
what can I do to use my own servos?
sir help me source of source to make the robot I am interested in your robot. and the stages of the process. thank you
We can do this with normal servos by any chance
You can. But you probably won't be able to get it to move as smoothly as in the video with cheap servos. But depending on the servo, you should be able to get fairly close.
@@ElectronDust libraries remains same or it will changes or code will change according to that
@@agoogleuser5128 You will need to change some code
How did you found the real world coordinates from pixel coordinates?
I guess the system knows about the x and y of the camera, so the vision system identify the x, y relative to the camera. Know transforming these coordenates relative to the base frame the system would be able to reach that point. But I'm just guessing.
this is excellent
phenomenon
would you like give me structure about this project
and reference file and ROS file
Here's some more infos:
robotshop.com/letsmakerobots/raspberry-pi-robot-arm-with-simple-computer-vision
and the source code:
github.com/T-Kuhn/ScrewPicker
Good job bro! Can you share source code of your project.
Sure, it's all here:
github.com/T-Kuhn/ScrewPicker
Hello, how did you run the Computer Vision Algorithms on Raspberry Pie ? do we need to convert the matlab code into python ?
Hey!
I wrote all the Algorithms in Python.
Sourcecode:
github.com/T-Kuhn/ScrewPicker
Thanks
+Tobias Kuhn i never learned python, i use matlab... can we convert matlab code into python and then apply it to opencv ? if yes, how so ?
Hello Sakibul Azam,
I actually don't know. I only used very simple Image Processing algorithms. That's why I wrote them by my own instead of using something like OpenCV. If you want to make something with a Raspberry Pi I highly recommend learning Python. Since it goes very well with the Raspberry Pi camera module.
please can you share the code?im doing a final year project related to this
Hey bro, really nice work with the whole project, I was wondering if you could help me I'm thinking of integrating this whole project into an autonomous moving robotic arm which could pick stuff can u tell me what problems I can ran into while doing that cause I will edit and use Ur python code for pick and place and one more thing if I some how make this work do u think image back ground will be a problem for object detection cause the surface will change as robot moves.
Pls looking forward for your reply thank you !
How does the Pi control the servos?
Via Serial bus. Read this:
electrondust.com/2017/10/28/raspberry-pi-robot-arm-with-simple-computer-vision/#more-11
how does it gets the x and y data of the placed objects that are sitting in the 2-d plane ?
It takes a photo with the camera and then applies some algorithms to the image data. In the end it gets the center of the object, it then just counts the pixels to that center (x and y) to see how far the arm needs to go.
Wow, neat :)
Dear Tobias Kuhn,
In those videos only can see, how it is working. if you can give any link address(video) having all steps that how to make this from starting to end, will be helpful for me.
I am sorry to tell you that there is no tutorial. I should write one.
There is some more info here:
robotshop.com/letsmakerobots/raspberry-pi-robot-arm-with-simple-computer-vision
But it's not a tutorial.
Model / SKU : SV-MG995/Servo Motor MG995 (360 degree)/Servo Motor LDX-218/
can i use any one from those. Because it(ax-12a) is very hard to get here.
You might be able to do something similar with servos other than the Dynamixel AX-12A, but if you want to so the same, you need those servos.
How would I adjust this project to do face tracking instead?
Wowguy233, use haar cascade for face detection, then use to track the direction the object moves arctan(y/x). github.com/danielskraus
Are you a software engineer?
jep
What is jep?
Make how to make it video
Do u used mcp3008?
Hello supra!
Yes, I used it to test some of the movements while programming the thing. I posted a link to the sourcecode in the description of this video. There's still no schematic. But maybe I could draw one if you would find it helpful.
Can you share the code sir ?
DO YOU USE in ros gazebo simmulation
Hey,
I don't use ros. Just python
Hi can I know you are using matlab image processing?
+. Arduino Hello!
No. Because the Image processing algorithms I used are all very simple, I programmed them by my own.
wow what program do you use ?C++?
I used Python. You could also use C++. But Python seemed to be the better choice because there's a really good library for the Raspberry Pi Camera module. With that library, getting pixel data from the camera is really easy.
I wonder if people at delfy look at this
wow...it's too gr8 project sir..i am also doing same project but with tower pro mg995 servos.does it need extra library?...i also want to kno what are those pixel algorithms that u used..i really need your help.can u plz guide me doing this??i can give u my email address ..i m just waiting for your response sir..plz
may i know how much it costs
about $300. I used Dynamixel AX-12A servos. They are quite expensive.
how do you implemented computer vision?. Amazing Robot!!
I did it by implementing some very simple pixel based algorithms. It isn't that hard as long as the background has a uniform color!
Tobias Kuhn wow. I start to see artificial vision with matlab. so, I'm watching any vidios for make me one idea that how i can do one project. Artificial vision sounds good. thanks a lot for your videos.