Robot Dances so You Don't Have To
Вставка
- Опубліковано 26 вер 2021
- AD $2 for 5 PCBs, SMT Assembly fee $0. Order from jlcpcb.com/RHS
I really wanted to experiment with Posenet running on NVIDIA Jetson Inference, so I can use it to make a gesture or pose recognition interface for events and live shows - using my Performance Robots which I built a while ago. I'm using Python-OSC to send OSC messages to QLC+ Lighting Controller which is a DMX Controller. My Performance Robots are DMX controlled but they are in storage, so I built another smaller DMX controlled robot to test it out - it dances so you don't have to!
CAD & Code: github.com/XRobots/Posenet_OSC
You can support me on Patreon or buy my Merchandise:
***************************
Patreon: / xrobots
Merchandise: teespring.com/stores/james-br...
***************************
Affiliate links - I will get some money of you use them to sign up or buy something:
***************************
Matterhackers 3D printing supplies: www.matterhackers.com?aff=7500
Music for your UA-cam videos: share.epidemicsound.com/xrobots
***************************
Other socials:
***************************
Instagram: / xrobotsuk
Facebook: / xrobotsuk
Twitter: / xrobotsuk
***************************
CAD and Code for my projects: github.com/XRobots
Huge thanks to my Patrons, without whom my standard of living would drastically decline. Like, inside out-Farm Foods bag decline. Plus a very special shoutout to Lulzbot, Inc who keep me in LulzBot 3D printers and support me via Patreon.
HARDWARE/SOFTWARE
Below you can also find a lot of the typical tools, equipment and supplies used in my projects:
Filament from: www.3dfuel.com/
Lulzbot 3D Printers: bit.ly/2Sj6nil
Lincoln Electric Welder: bit.ly/2Rqhqos
CNC Router: bit.ly/2QdsNjt
Ryobi Tools: bit.ly/2RhArcD
Axminster Micro Lathe: bit.ly/2Sj6eeN
3D Printer Filament: bit.ly/2PdcdUu
Soldering Iron: bit.ly/2DrNWDR
Vectric CNC Software: bit.ly/2zxpZqv
Why not join my community, who are mostly made up of actual geniuses. There’s a Facebook group and everything: / 287089964833488
XROBOTS
Former toy designer, current UA-cam maker and general robotics, electrical and mechanical engineer, I’m a fan of doing it yourself and innovation by trial and error. My channel is where I share some of my useful and not-so-useful inventions, designs and maker advice. Iron Man is my go-to cosplay, and 3D printing can solve most issues - broken bolts, missing parts, world hunger, you name it.
XRobots is the community around my content where you can get in touch, share tips and advice, and more build FAQs, schematics and designs are also available. - Наука та технологія
![Evolution of Boston Dynamic’s Robots [1992-2023]](http://i.ytimg.com/vi/Rdm2ggtFvmQ/mqdefault.jpg)
![[柴犬ASMR]曼玉Manyu&小白Bai 毛发护理Spa asmr](http://i.ytimg.com/vi/0TsXQ7z2Dh4/mqdefault.jpg)
There's a security vulnerability to your priority code. If someone comes in the field of view with a giant paper mâché head, they can steal control of the robot from anyone at much greater distances
XD ah yes the new weapon for hackers the paper mache head
Ah, the famous "Frank Sidebottom" attack. :-)
It will become an arms race to see who can emulate the largest head.
Nerds be taking their security seriously these days.
Or make a regular sized paper mache head with the eyes super far apart
The project is awesome but Ive seen you dancing and you're pretty good at it
Well integza is interested so WATCH OUT!!!!!
You could make an infuriating piece of art that shows an image that draws your attention when seen in the peripheral vision, but blanks out when you look directly at it.
How about an automatic "boss key" for your computer?
Fucking genius man
I've been thinking there should be a horror game like this. Monsters appear exactly where you can't see them.
This would be awesome!
Absolutely Demonic!
There's a book I read once about a necklace that makes it very difficult to look directly at the wearer, would be cool to see those two used together
Humans have all this technology but still can't find me in the forest smh
We can we’re just to lazy
It would be quite funny to see your expressive animatronic doing "Baka mitai"
Dame...
dame na no yo..
Cringy af
@@papalevies no u
Random idea and I know it’s just a test bot, but I reckon you should add random blinks to the eyes, just to make it even more immersive.
They gotta be fast too. To look realistic.
+1
Agreed, but his does have an endearingly drunk look.
Yeah a quick delete on the servo smoothing only on the eyelids and randomized actuation would be neat
Smoothed out eyes makes it look tired. Eyeballs don't decelerate like that.
Yeah, look into saccades. 💜🤓👍
This is really awesome! You make it look so simple :) Using the DMX/OSC/MIDI stuff to control the robot is brilliant! Would've never thought of that myself
Great stuff. Your generosity with your hard work is brilliant.
I know it's such a minor thing but I love how your protoboard almost perfectly matches the color of your filament! really ties the whole project together :)
GREAT use of FOS and great use of different aspects coming together and working
This guy did more work in a week than a team of engineers could do in a month. And I couldn't understand the monologue.
Just because you don't make videos of you work doesn't mean that you don't do good work. A lot goes into being a UA-camr. A lot goes into being whatever type of engineer you're referring to. I feel that you're going to need to be very specific with the engineering team that can't do what James does in what you think takes a week. There's different types of engineering. Therefore, one size does not fit all. Also, speaking from experience, many people can't and/or won't record their work InProgress.
Nice use of pre-existing technology!! That saved tons of dev work. I may have to look into using that!
0:45 my man busting a move over here!
You are either crazy efficient or really put in a lot of work to be able to make all this in just one week! Really astonishing!
I wonder if this can be combined with the head tracking apps that use VR API?
Then you could have full body tracking VR without the helmet.
This is awesome, James! You never cease to amaze me!
This is the answer to why it's important to take your math classes seriously.
why?
@@ernestasgrimas4367 Programming requires everything from binary math to algebra, statistics, calculus, and discrete math. This is why someone with a computer science degree, like my wife, was one class off from also having a mathematics degree. The other class would have been analysis mathematics.
I'm self-taught because back in the 70s and 80s we had to know how to program computers to do anything with them. And for some incredibly awesome reason, computer programming was a graduation requirement at my high school in 1988. I was also in a gifted program so I got to be excused from my classes for part of the day to use computers. I mostly just played videogames or made videogames.
Yes, math is unavoidable but if you work on projects that you're passionate about then it won't seem like a big deal.
And that's one to grow on - *TING!
@@maxxsteele9396 I disagree, I’m a developer, I suck at math. I think the fact that I suck made me a better developer because I generally want to write things that did the math (and all other tedious work) for me/others.
@@Mrcaffinebean I don't understand what exactly you're disagreeing with me on. I gave a very long winded response to
Ernis Grimas' question to my comment.
That’s a really nice addition - seems almost straight forward to implement. I should definitely get a jetson for my robot
Thanks for sharing this :)
Awesome to hear your 3D printer running... I know it may not be entertaining for everyone, but I first subscribed because of your shots of the 3D printers without crazy music.
😳😳😳 wow the smoothness in movements of this robot is very epic 🤝🤝🤝 this by far the coolest thing I have ever seen looks straight out like an animation
That's Brilliant using DMX/OSC/MIDI to control a robot. We use QLC+ at out church it works great for controlling a dmx universe and pixel mapping from a single simple interface. I even built a custom midi controller for it.
Teasing us with a non opendog vid!!! Love your content.
You may want to experiment with decreasing your smoothing IMO. The current lerp parameters makes it take a far longer to reach the target pose than is probably necessary
Bruton's got moves! 0:58
glad you got to take the performance bots somewhere during the beforetimes. Was thinking how typical it was that the moment you have these ready, a pandemic comes along and stops all the fun happenning!
This is truly Mechatronic art!
I love this project and it’s simplicity. I took it as an opportunity to learn fusion 360 better :) - I’m adding a 8x8 LED matrix to the front for a mouth with a small microphone that will be right next to a docking station for an Amazon Alexa. That way it can either use the voice of the person purporting it or pick up Alexa’s voice while it’s running idle animations. I can’t wait to see how it turns out :) I thought my addition might be an interesting way to give the bot a little personality- I guess we’ll see!
Nice work, just used RS495 with profibus protocol, interesting to hear about the audio protocols.
Excellent 👏👏👏
I love your little robot, it's so cute 💚
bro was getting sturdy 0:45
Great video
thanks! learned new things. Time to buy a Jetson :D
as someone who had to make a servo work with DMX, thank you for using OSC... lol
I see that robot and i think of Tobor, the robot from Shakboy and Lavagirl. Very cool video.
Man's one bad day away from becoming William Afton. Very cool project!
the eyes and eyelids are too smooth
I know this is old, but if you use two cameras you could do the 3d trig a bit easier for the arms. You could get the two dancing robots to each have a camera placed on them and combine the data to find the relitive position in 3d. Then use some projection mapping to find the angles of the arms, and then place that onto the servos.
At the end of the video i noticed the Bender head, how awesome would it be if you would implement those robot eyes in the Benders head so it would be rolling its eyes around while you talk in the background .
I would guess that using a kinect camera somehow would help with mapping 3d depth as well which would probably work with the other robots that aren't just on a 2d plane with the arms
Blown away!
Hey James, as another non-touching control, maybe you could use on of those step on piano mats. If you could find one that has midi outputs, you could do that. Or if not, tap into the wiring of it, and use each key as a button press. That way people would just be able to tap the control with their foot.
I think I featured that in the clip of my Performance Robots
I've commented on this at this channel in the past. I am a graduate student researching new intended motion systems for exoskeletons. My team has created a table-mounted wrist prototype to test our system, but I have created a backpack frame and shoulder that will one day hold the arm. For now it is a geometry test to make sure the links have the proper range of movement, so it is not actuated, but I'm creating the design so that actuators can be easily added later. So since all the joints are passive for the moment, and since I need it to be able to hold the weight of the succeeding links, I need to make the passive joints of the shoulder resistive, so they can move freely but hold their position when not moving, a lot like a laptop lid. Right now I'm using rings of magnets around the bearings but there has to be a better way. Anyone with any knowledge or ideas, please let me know! Thank you!
brilliant! ❤
Awesome!
That is straight up bonbon
Skynet is coming along just fine.
idk if it was set up this way, but if you are facing the robot, the robot should raise it's right arm when you raise your left arm. proper mirror like. imo.
He said he flip the video of him in editing this video
Could you use a second camera to make 3d movements?
crikey this is next level...
I hope this project get upgrade
Awesome
Switches camera on and........ starts robot dabbing!!
Is it possible to use this type of tec for a animatronic type mask that moves its eyebrows when you move yours for a little bit of expression?
U can try some 3d pose recognition models from rgb input. But getting them to work optimised on jetson inference is hard
Thank you for teaching ai in such a digestible manner. 🦾
Now that’s cool 😎
Hi James,
I have developed similar program for 7dof arm in 3d space.
This is epic
3D motion capture just requires more cameras to triangulate positions or use a Kinect.
Nice
Vibing robot :D
Can you mirror the video before it does the tracking, to solve the mirror issue?
Great video as always James 👏👏👏
THIS COMMENT IS **NOT** AIMED AT JAMES.
The very fact James has needed to adapt to think about covid and “not touching anything” is sad. Ok, you could argue that adversity drives innovation. Such as wars. And this video could be viewed in this light (that’s how I choose to judge James’ video, positively)
But other large companies are using covid to introduce technological changes that are bad. Such as a cashless society.
Anyway, loved this video. That nvidia thing looks awesome.
Great !!!
put the cameras in the easy gesture characters eyes and upgrade to usb webcams that can zoom in and out
"It dances, so I don't have to!" That's certainty one way to go about it...
I'm agree
Hi, do you know if it would be easy to recognize facial expressions? happy, sad, surprised...
To sort the camera mirroring point the camera at a mirror?? I am quite inspired by this. I work with children with special needs this would be an interesting use of ict some of the pupils would love.
There's a similar webcam software but it's for VR to give you full body tracking using webcams only
Uh oh, James is making cute robots now. Time to work with look mum no computer.
There's a collab with LMNC in my channel from a while ago
you know those keyboard on the ground you can play music with ? would be perfect for you show aint it ?
Is this possible to do on a raspberry pi as I don't own a jetson nano
If you end up using buttons, make them self sanitizing with cloths that wipe them every few minutes if no one is present
genius
What kind of servos are they
Anyone else getting the Hugh Jackman Real Steel vibes?? Obviously this is very early prototype. But could imagine if this robot was larger, servers moved faster, and the camera was tracking a fighter in a control cage to fight another robot????
Robot Fighting Begin!
this lookss funny
Could you combine the data from two sets of cameras at 90 degree to get a 3D representation?
Almost definitely
Sir, my team and I are working on making an autonomous rover, we have an Intel Real Sense, an RP Lidar, and few other sensors. Do you think it is worth using a Jetson Nano for the processing as we are planning to acquire one?
Thanks 👍
We are using ROS Noetic as the middleware.
You should still keep the buttons for accessibility reasons
This looks like the top half of one of those Endoskeletons from Five Nights At Freddy's, and we're nearly into October... Are you building an actual functioning Freddy Fazbear animatronic for Halloween?? O=
I'd love to see this robot mirroring Ricky Gervais' "monkey dance" from The Office, tracking David Brent character on the TV screen.
Awesome stuff. How do you pay for all of this??
I'm guessing from the ad revenue from his youtube videos? Am I wrong?
Could have swarn that your robot dog was at the NEC today lol
Yep, I was at the NEC today!
@@jamesbruton Haha no way must have missed you, I just had a quick pass though as i was doing Interplas today, going to have a better look at the printers tomorrow :-)
I like how no one controls it. 🙂🙂🙂🙂🙂
how about that bench, the fun ride seat, with total freedom with control sticks, to control robots, remotely, or inside one, both are fine
also leg position/movement, seat on a stick, climb pole
you can do all demonstrations, no need to touch, or let anyone come too close
just have it look and point at the closest person, should be spooky
Would you please show us any replica of your projects done with your viewers.
Why don't you use a xbox kinect as a camera? The tool provided for Windows contains different demos, 3D skeleton included.
Can you make that to ooguye, his smooth motions seems like that
You, sir, are a true hacker.
"Basically it dances so I don't have to"
RIP DMX.
I wonder what your neighbours were thinking when you were posing and casualy fist pumping mannequin heads to test the code.🤣🤣
Make a 6 or 7 dof robot arm
shadow boxing!
💗💗💗💗
Is it possible to put the camera in the eyes