Really Useful Robot #3 - Head & Intel Realsense Camera
Вставка
- Опубліковано 19 сер 2024
- It's time to make the head and plan the robot arm mounting for the Really Useful Robot. The upper section of the robot can travel up and down the utility stick mounted on the robot's base. The head contains an Intel Realsense D435i depth camera that we can use to identify objects using NVIDIA's deep learning machine vision models, and then get the depth from the camera. This allows us to calculate kinematics for the robot arm so we can grasp the object.
The robot can already map and navigate it's environment using an RPLIDAR A2 and the ROS navigation stack, which is detailed in the previous videos, as well as the mechanical build.
Open source CAD and code: github.com/XRo...
You can support me on Patreon or buy my Merchandise:
***************************
Patreon: / xrobots
Merchandise: teespring.com/...
***************************
Affiliate links - I will get some money of you use them to sign up or buy something:
***************************
Matterhackers 3D printing supplies: www.matterhacke...?aff=7500
Music for your UA-cam videos: share.epidemics...
***************************
Other socials:
***************************
Instagram: / xrobotsuk
Facebook: / xrobotsuk
Twitter: / xrobotsuk
***************************
CAD and Code for my projects: github.com/XRo...
Huge thanks to my Patrons, without whom my standard of living would drastically decline. Like, inside out-Farm Foods bag decline. Plus a very special shoutout to Lulzbot, Inc who keep me in LulzBot 3D printers and support me via Patreon.
HARDWARE/SOFTWARE
Below you can also find a lot of the typical tools, equipment and supplies used in my projects:
Filament from: www.3dfuel.com/
Lulzbot 3D Printers: bit.ly/2Sj6nil
Lincoln Electric Welder: bit.ly/2Rqhqos
CNC Router: bit.ly/2QdsNjt
Ryobi Tools: bit.ly/2RhArcD
Axminster Micro Lathe: bit.ly/2Sj6eeN
3D Printer Filament: bit.ly/2PdcdUu
Soldering Iron: bit.ly/2DrNWDR
Vectric CNC Software: bit.ly/2zxpZqv
Why not join my community, who are mostly made up of actual geniuses. There’s a Facebook group and everything: / 287089964833488
XROBOTS
Former toy designer, current UA-cam maker and general robotics, electrical and mechanical engineer, I’m a fan of doing it yourself and innovation by trial and error. My channel is where I share some of my useful and not-so-useful inventions, designs and maker advice. Iron Man is my go-to cosplay, and 3D printing can solve most issues - broken bolts, missing parts, world hunger, you name it.
XRobots is the community around my content where you can get in touch, share tips and advice, and more build FAQs, schematics and designs are also available.
Man the quality of your stuff is just insane these days. The fact that you do so much work every week and distribute so much of your stuff for free (in video form and the open source stuff) is insane.
I remember discovering you building the Hulkbuster suit years ago and have been watching ever since. Heck I'd even say you were one of my inspirations for going to school for Mechatronics Engineering.
Thanks for the awesome work James.
thanks!
@@jamesbruton the code is free but the hardware costing the price of a small tesla isn't hey. Lots of kids wonder why they can't do this at the robotics class in their local makerspace. A jetson Xavier NX is as fleeting to them as the existence of Santa Clause.
13:35 "I'm still a human" , quite presumptuous, James.
This is absolutely the most amazing series I’m following on UA-cam right now. Your projects never fail to amaze, keep it up!
"I'm obviously a human. No problems there."
Exactly what our robotic overlords will tell us as they prepare to enslave us.
This was inspiring. Please continue working on such projects
You can add the cameras point cloud to the costmap even with a 2d costmap, we use if for navigating around tables, even though we only have a 2D map.
Also your basically recreating our semester project, which is kinda awesome :D
Thanks for that
These projects are progressing similar to child development. The dogs are learning to walk. Now you are training it to recognize objects. Soon it will respond to your commands and complete tasks. Eventually you'll have it doing chores around your house!
I have a degree in electronics and work in IT and am humbled amazed by James' achievements! great work!!
Nothing is difficult for you Sir. You seem to be able to churn out projects like a machine... Respect!!!. My projects take forever to finish.
Mate, it would be awesome if you could setup a 3d scanner with the Intel Realsense, it would help everyone out there.
A solid, exact hand scanner for cad. At a price that doesn't require a kidney
Isn't there a phone application that does this as well?
@@ks-yl9ed if you need to buy a $1000 phone to use it it defeats the whole purpose doesn't it?
Love seeing the Raspberry Pi 400 in the background!
I think that's probably just the official Rasp Pi keyboard + mouse set
keep it up james, always looking forward to a new video
Please continue. This is what I've been waiting for.
Very impressive. This is sleeker than some commercial home robot platforms. Well done.
"I'm obviously a human. No problems there."
That sounds rehearsed
No way I can do this sort of project myself, but I've subscribed to keep tabs on how you're doing with it! Amazing work, and I think it's great that you make everything you're doing free for others to follow along! This is how tech progress will become everyday technology!
thanks!
Super awesome, super jealous of having the time to do all of this. You can participate in RoboCup@Home with this once the arm is set up 👍
you are going from strength to strength. looking forward to seeing what you can get the d435i to do !
I like getting home at the end of a Tuesday and watching your videos. Been doing it for year. Keep it up.
With such progress, at the end of the series the robot will be self aware :)
Impressive! I can't imagine 3D printing at that scale...I'm too used to my little 120mm cubed 3D printing space!
So much going in to this, that I might have to watch it again just to take it in 😂
It's been a month getting anxious for the next one
Really curious to see what the arms will look like. I worry a bit that they might make it too top heavy.
He might use lessons learned from Open Dog V2 and keep the big motors close to center of mass
yeah i was worried that he might distort the belt or have issues with belt stretching and/or slip down the line. especial when it picks something up.
The models seem really well thought out.
Your videos are always amazing. Can’t wait to see the finished product. Can you explain the reason for moving the head with the arm carriage instead of it being static ?
So it can see things lower down
Phenomenal work James!
"get me the pink ball..." No, it should be "pass the butter".
But Sir the words, "Pass the Butter!", causes robots to have an existential crisis! lol...
For that LCD with SPI touchscreen you can try an Arduino Pro Micro with an SPI to HID mouse sketch. You can figure out how to do that because you're a smart guy! Good luck!
Very smart thought out masterpiece!!! Having the arm and ability to change heights of the head should make it capable of opening doors or using an elevator- that's MIT robotics level !!!!
I am very excited about this project
Hey robot, pass the butter
Your work and results are just phenomenal !
I don't like smooth idler gears on toothed belts. Getting the belt tight enough to eliminate slop causes the teeth on the idler gear to distort which can cause the very slop you're trying to eliminate. When you make the belt tighter to eliminate the slop, it doesn't help. It's caused me some problems in the past. It's a small thing, but I suggest in general that you use a toothed idler gears.
@16:42 This is the first time I heard someone used words pink-ball for a bear. :)
I am really enjoying this series!
Amaizing work!!! Please add navigazion using Slam on depht camera!!
Maybe you can get it to follow you around while carrying your tools using the depth sensor human identification feature~
Great video James! Great progress, thank you for sharing! You make us feel unproductive when you achieve this much great work in a short time.
I would like to add a few points that might not be touched in the video for who would like to integrate Intel RealSense cameras to their projects.
1. For D435, although a USB 2.0 connection would work, it would not give full set of features like depth and higher resolution, so USB 3.0 is almost mandatory. Otherwise it would be a very expensive webcam :)
2. Finding a single board computer with USB 3.0 is a consideration to be done early on in the project. Nvidia Jetson series (in the video) and LattePanda Alpha/Delta are good candidates. Raspberry PI3+ and below are not usable. I have tested with PI3 and can report that it is unusable. Raspberry PI 4 might work, but still could be underpowered for a large scale robot project.
3. Intel RealSense cameras are power-hungry and sensitive about USB supplied power. They do not have external power inputs, so usually they require a powered USB hub. A small factor USB 3.0 hub with good amount of ports is almost a must for a substantial robot project. Plugable and Anker have good and compact ones. Some hubs are more finicky than the others in Linux. Plugable needed few power cycles every now and then, Anker worked perfectly.
4. Besides the D4XX series (D435 and D415), Intel also has a T265 camera that can do visual odometry, tracking and localization. It is an amazing camera that compliments the D4XX series and makes the wheel odometry unnecessary and let's you build a complete vision based solution. This is particularly helpful in scenarios where wheel odometry is not feasible or not accurate (outdoor off-road, flying, human/animal mounted systems etc.).
5. The T265 camera is a pair of stereo fisheye cameras with onboard SLAM processing. Besides the very accurate localization output of the camera, the raw camera images can be used for other vision processing purposes, including a crude fisheye stereo depth estimation, but don't expect a detailed depth view like D435 provides.
6. Both D4XX series and T265 are mainly using near-infrared cameras. DXX series have an active near-IR illuminator (laser dot pattern that is not visible to the eye) just like Kinect, but it is also a passive stereo camera on the near-IR band (unlike Kinect). So it works both indoors in dark or light places with the help of illuminator and outdoors more relying on passive stereo. T265 does not have illuminator, but is pretty robust to lighting and I can report that it works accurately both indoors and outdoors, but most probably won't work in pitch black like D4XX series would.
Please feel free to reply to this comment if you have any questions or issues about Intel RealSense integrations.
Keep tinkering!
Rossum's Universal Robot. RUR. I see what you did there...
Amazing project 👏
waiting for this episode...if you have time can you make a tutorial on ros for navigation.
he should probably add ultrasonic drop sensors, an ultrasonic low ledge sensor(ultrasonic sensor on the base looking forward), an ultrasonic backup sensor on the base, and a front facing floor camera on the base. he should also replace the y axis belt with a lead screw to prevent problems down the line like stretching and slipping. though for the arms it might be best and go the asimo route and use hydraulics in the base or head to control the griper and/or the arms.
awesome stuff. Design is quite logic and simple, that's a sign a genius ! however, a bit top heavy and robot will have to manage the weight of object carried at the bottom of his arm... maybe he would need a kind of mobile weight at his back to avoid falling ?
It would be cool to program it so it would keep track of objects such as keys, phones, and wallets. Then when you can't find them just ask your robot
Wow very interesting project!!!
I think that you could add a rail protector (as cnc use) to make it looks more professional, without the sight of the belt, and to make it safer (it should be used in the future with kids in house). I'm talking about bellows. You can buy cheap but still professional ones from aliexpress (intended for cnc as I said) and make your very useful robot even cooler!!!
Thanks for your idea-sharing of every video, hope the best for this project!!!
something he really needs to do is make it speak coz it has eyes, and later on, give it emotions
Its like robocup home. I hope we go for two arms
7:29 I made a UMPC using that exact same touchscreen!
Very good job. Is amazing.
So excited for this series! I have the real sense camera and a nano and have been tinkering with it for the past few months. I also have a platform for it made from an electric wheelchair base. My hope all along has been to build my own version of this robot. Now I can follow along with your project for tips and ideas! I'd like to use the mapping from the camera though since I don't have the laser scanner. Do you have any interest in showing that feature in a future video?
Slight blind spot under the screen - might wonder where its willy is..
maybe a smaller PTZ wide angle camera under the screen for allowing dual vision?
Masterpieces !!!!!
I wish the iPhone LiDar was available as a module for DIY projects.
my dude really making droids a thing
Very good
I see a physical version of SHRLDU coming up
Woo superbly
Fantastic!
Amazing
This seems like it'd be really useful in the medical feild
Great robot
In todays episode of "how many times can James say ROS in the first minute"....
Really Cool!!!
you should tell us about the new TAZ
Do you have PEI on the bed?
yes
The music in this video really reminds me of Apex Legends
If this guy is not humanity's real Iron Man I don't know who is
If it's getting a bit wobbly, you could maybe just add some weights into the bottom? Casting lead would be fun but perhaps a bit much, big bag of rice would do the same! I think as well if the base were slightly longer since it's wobbling back & forth, side-to-side seems less of an issue.
another thought - perhaps some vibration damping on the post itself? if you still have some flexible filament you could maybe make some bushings that isolate the stalk from the vibrations in the base
Amazing!
I feel as if "open source" allows you to sleep at night....good Ole git hub
Hi
Can you please share the link for the joysticks of your Smart Robot Remote?
Thanks and congratulations for this project.
Great job as always. One day I'll get back to building a new robot that can use ROS navigation, yours seem to work just fine for that purpose.
I'm a bit intrigued about the current base version. You said you'll consider 2 caster wheels to make it more like a square (I'm guessing it will be closer to turtlebot waffle version). Could you please elaborate a little more on that?
It was just a thought really to make it more stable, but I won't be making any major changes for now. I guess the wheel layout would look more like an electric wheelchair.
Nice
would adding a second camera give the robot better depth persepstion like in animals?
Some guylines might help with the bouncing head. Like a mast on a ship.
you *are* amazing
Hi can this robot use as temperature sensing and kick in data to storage maybe it useful at supermarket
Damn that's awesome =D How do you feel about binary vision in robotics? there has to be a reason its not more widely used, but it may be a little better at getting a 3D perspective on an object.
why not mount the camera on the very top? seems like having it move will only complicate things, especially if you want to do any machine learning using a video feed.
So that it can move down to the ground
Would be better if it has edge detection and has ability to vectorize image
Did you get another Lutzbot printer? The Workhorse? Will your code work on the Jetson Nano Developer?
Exciting stuff, was wondering what 3d printers you are using. Mine print rubbish..
All Lulzbot printers, they support my channel.
@@jamesbruton Let me check them out..thanx
Can you sell one to us. We are in India. We need some functions custom made.
How does the Xavier handle running everything? Are you running it near 100% capacity or do you think you could use a Xavier NX instead?
YES!!!
this dude is building the allay from minecraft
Sir can you make iron man like personal ai assissent
hi! is it possible to use RGB camera for the object detection and stereo for depth estimation on ROS or RealSense software at the same time? I'm gonna get one of these cameras but they cost a lot and haven't seen that info anywhere.
What is the estimation of cost for all hardware?
So the name of the robot is R.U.R. Maybe you can put that on the side of it one day haha
Have a look at RTABmap_ros for 3D mapping. Happy to assist sorting out the config, i've used it extensively.
How can i test the imu in my 435i? In realsense-viewer, the view frame does not move.
nvm I thought it was like the t265.
Looks like a Fetch 👀
I chalange you to make a fighting robot
likeness of a llama, alpaca
or a camel
partial item guess recognition
subset detection
that mapping system needs to improve in resolution, when you move the head and the bot around
now its only generates a snapshot from one point, not an integrated construction
Just make the head a Dome James, you know you want to :P
.
#R2D2
I came here cause of mark rober
I have one doubt. by creating robot what could you gain and after creating new robot what will you do with that.
He's gaining invaluable systems integration experience in a public setting. He will apply the sophisticated sensor fusion, environment mapping, and navigation resolution solutions that he implements into his open source OpenDog robots.
James makes entertaining and educational content, the robots are cool byproducts!
#TouchScreenROSWorkstation
Wish I could be your intern