Arduino to Blender Tutorial: Control Virtual Worlds with the Real One

[@desmios444]

Bro, you're a lifesaver bro, i've been doing HOURS of research on how to make an affordable mocap suit, so please, don't stop making these.

[@checkeredbug8015]

I am working on a tutorial on how to build a suit, but I am swamped with other work atm. Hint, I just posted a new github repo with all the suits docs if you want to jump ahead

[@desmios444]

@@checkeredbug8015 bro, yeah, I understand, but thank you bro, that really means a lot

[@weihangli1491]

@@checkeredbug8015 hey, bro. i'm so glad to see this kind of video. Could you tell me how can i get valve out of blender? Much thanks

[@AKKJ420]

I should have known about this when I did my PhD it would have made my life so easier. Speaking of PhD. I got confirmation that I have passed it today. Thanks for your video mate.

[@NauberSV]

I've been looking for such video for days! Excellent!
I'm a maxillofacial surgeon, and an Arduino hobbyist too.
I've been using an addon (OrtogOnBlender) to import medical computed tomography scans into Blender and then perform virtual surgical planning. Your tutorial will help me to start a new project: to be able to capture the functional movements from the patient's mandible and virtually reproduce it.
Thank you so much!

[@nickkeeline4840]

Jaw on floor.. most reactions I get are "too complex for me".. Ive had a theory that there needs to be more advanced tutorials and people will rise to the occasion. If you you were the only viewer I had it all be worth it. Thank you sooo much for your encouragement..I am humbled and honored to be any part, no matter how small, in what you're doing.

[@NauberSV]

@@nickkeeline4840 Hi Nick!
You said "small part"? Man, you just unlocked the first door that I have to pass through to get into this!!
I reproduced this tutorial and I can say "I got into it".
Next step is ordering the BNO055 IMU, as you suggested... and try to open the following doors.
If you don´t mind, I would like to ask for your help in the future :)
By the way, if you want to see three examples of how I´ve been using Blender on my professional pratice, please check the posts tagged as #ortogonblender on my instagram (@nauberbucomaxilo)

[@checkeredbug8015]

I took a look and it's really cool. yeah, feel free to ask anytime. you can steal my code from the other tutorial I did on the real time mocap suit, it will work fine with one sensor :github.com/nkeeline/KeeCap-Open-Motion-Capture-Suit

[@NauberSV]

@@checkeredbug8015 Thank you, Sir!
I watched the mocap suit tutorial too, and now I´m looking forward to receive the sensor and get started.
Meanwhile, I´ll study the codes and instructions that you´ve shared.
Let´s keep in touch. Thank you again.
Regards,

[@NauberSV]

@@checkeredbug8015 Hi Nick!
Instead of using a FOR loop with a predetermined number of executions, I discovered a template called modal timer. It allows the execution of the script until you press ESC (or other command you change). Blender functions still available meanwhile.
Since you are the specialist, maybe you are aware of this... but my obligation is to share... just in case :)

[@domain3973]

You are great
I can say there is no one who can beat your intelligency ane teaching ....
Very very thanks for this masterpiece....
Please dont stop making these type long tutorials
Love you sir ❤

[@jorgeguberte]

I started messing around with Blender recently just for fun and i thought about doing something with websockets, and then i thought "hey what if i can control whatever i make with an Arduino?" And here I am.
It amazes me that we have access to all this kind of technology very cheaply or even for free.
I'll be experimenting.

[@quincy3d69]

Yes , this tutorial is exactly what i needed for digital puppetry !!!!! i thought of it yesterday and today i found this video , very nice !!!!!!!!

[@cleisonarmandomanriqueagui9176]

This is amazing and how opensource software can interact with the real world ... I am from civil engineering and here you can prototype tools for the BIM world . Its superb

[@MrStumpi1]

hidden gem of a video

[@toapyandfriends]

Just a note if you're doing this tutorial when he goes to blender you have to close out your Arduino IDE window otherwise blender won't detect the com port

[@emanuelsiu1232]

thank you!

[@yossepbinyoum2181]

Just beautiful. Perfect bridge between two worlds.......................

[@dr.vijayanbalasubramanian818]

Great! I have been longing for it. You have shown the way. I am grateful. Thanks a lot🙏. Also I am thankful to Blender🙏.

[@toapyandfriends]

Tysm... I'm checking now I hope you have a bunch more tutorials that'd be awesome (on this)...😆

[@DolfVeenvliet]

This is excellent dude... just copy the serial folder... I totally didn't think of that!

[@toapyandfriends]

This is really cool...love to see mocap suit tut

[@toapyandfriends]

Just bought one of a board and excited to learn this...subscribed!😎(😆)

[@toapyandfriends]

😆Yayyyy 😆Python...is in the Building 🏫🤠

[@hallersamuel]

Excellent tutorial! Still trying to figure out how to use a MPU6050 and use its rotationvalues instead

[@stefanguiton]

Phoenominal, looking forward to more tutorials like these! Great work

[@nickkeeline4840]

I use the BNO055 IMU in the suit.. thanks for the comments!

[@VinayKumar-ge1dl]

Hello sir m having a bit trouble with above code the thing is when i try the code in blender it stops responding i tried solving the problem i short listed to line 6 that part with readline code pls help me when i found this video it felt like i found a gem pls make more

[@nickkeeline4840]

@@VinayKumar-ge1dl the code freezes sometimes in windows. I tried to fix it but it's in blender somewhere, works great on my linux box every time.

[@VinayKumar-ge1dl]

@@nickkeeline4840 oh ohk 🥲😨🥺🥺

[@SamFX]

Great video, thanks a lot. Hope you can manage to do a video tutorial about the body tracking!

[@algls8529]

You are really great!!!
Thaks you a lot for this share

[@checkeredbug8015]

You are welcome, someday when censorship reduces I may start making videos again.

[@GauravGupta-by1ml]

Hey! At 15:06 You told that Strings in Python should be written in the format b"string". But actually the string you used is the byte form of string and in python also string is written within normal quotes. But when we need to use byte string we need to put b in front of the string. Otherwise, everything is awesome. Thank you for the awesome tutorial.

[@checkeredbug8015]

Thanks for the clarification!

[@peterkrauspe9217]

This is a great idea and a very nice tutorial !!

[@checkeredbug8015]

Glad you like it!

[@elisakrippner]

Good shit man, great explanation! Thank you

[@amralmatni6586]

amazing man!
Thanks you helped me a lot

[@checkeredbug8015]

Glad it helped, makes it all worth doing:)

[@acess091]

That's awesome man. Good idea

[@checkeredbug8015]

Thanks 👍

[@marcd2011]

Super cool thanks !

[@Zaladquiel]

Hi. Thanks for your video. I suggest you to develop a group of nodes in Sverchock, with your encapsulated code. This way, we could use the potential of parametric and generative technology of sverchock with the information from Arduino technology. Something similar to Firefly addon for Grasshopper3D.

[@checkeredbug8015]

At first I had NO idea what you're talking about. Just watched a few tutorials on Sverchock. Very interesting idea indeed. Thanks for the suggestion!

[@Ikxi]

I will definitely try all of this next year!
(not enough money rn haha)

[@weihangli1491]

Hey,Man!!! It's happy than i could see your work in communication between Blender with arduino. And I also want to work in export real-time rotation data form blender to arduino to control a servo move in real-time. And I am a little confused in how to send real-time rotation data to arduino board when I rotate a bone in blender. Please give me some help in this filed. Thanks a lot!!!!!!!!!!! Hope you could have a good day!!!!!!!

[@upbrohd]

POTENTIAL FIX:
Hello Checkered Bug and everyone else,
when I tried this it didn't work and told me that "Python script failed, check the message in the system console" but I couldn't understand the message.
However, after trying a few things, this made it work:
Change
ser = serial.Serial('COM3', '9600')
to
ser = serial.Serial("COM3", 9600)
Good luck :)

[@toapyandfriends]

Is it better to use my Ubuntu Partition for this or can I do this just as easily on Windows 11...🙂

[@checkeredbug8015]

Ubuntu, windows has freeze up issues with real time scene update

[@rapideggcooker6945]

Thanks for this fantastic video, it could be great if there are any possibility to put this data on a 3d character, so it can create animations inside of blender the easiest way.

[@checkeredbug8015]

That tutorial is coming....

[@rapideggcooker6945]

​@@checkeredbug8015 That's great thank you, i'll subscribe so i don't miss any of your next tutorials

[@ajayjoseph8722]

What a legend! I was initially looking to interface with RPi (GPIOs) but I can only find "installing/running blender on PI" which I don't think is exciting as this, is it? lol

[@ztdeng]

Can I use MPU6050 to control human movement? What parameters should be entered

[@mikealbert728]

Very cool. What type of sensors are you using on the mocap suit? Seems to me like some of this stuff might be used for real time camera tracking and virtual sets.

[@checkeredbug8015]

BNO055 Bosch Inertial Measurement Unit is the suit sensor.

[@Hosses4X]

Where can i learn the sintax

[@Unpersona47]

This is brilliant man! I’m currently working on a project using audio to animate objects in blender. I’m using teensy as an alternative to arduino and was wondering if you have any tutorials on this:)

[@sandhanakrishnanravichandr9400]

I want to animate a cube in realtime using serial data from Arduino. I am able to read the data but using while loop does not show any realtime changes. The screen gets stuck. Any idea on how to solve this?

[@checkeredbug8015]

Sorry, but there is a bug in the windows version of blender that makes it freeze randomly. Linux works fine.

[@Mrisometric]

My blender is crashing please someone help ,its crashing at ser.readline()

[@swikarpoudel6704]

in arudino write serial.readStringuntil(
) . You must have forgetten that

[@justinbao7331]

Amazing Video!!!
But I'm using a MacBook Pro and I found that force update code seems not working on my blender, my blender just froze and nothing happens... I'm using the blender 2.83. Do you have any idea?

[@checkeredbug8015]

Sorry I don't own any macs. You should google pyserial for macs. If you delete the python dir in blender it will use python installed on your pc instead if you install the correct version, so if you can get pyserial working at all on a mac in python, it will work in blender

[@justinbao7331]

​@@checkeredbug8015 Hey! I think the pyserial is working on my MBP and I followed your instructions to add the pyserial folder in my Blender(Mac Verison) lib and I successfully send a text command in Blender's console and received a feedback from Arduino.
So I think there is another problem with my blender.

[@checkeredbug8015]

Sweet, glad to know macs can do it.

[@sasastudio4183]

nice work a fan from egypt .. keep it up .. two question can i use MPU9250 instead for the motion capture suit ... and how to turn servo motors as the bone rotate .. thanks in advance

[@checkeredbug8015]

That part seems doable at first glance. Controlling motors shouldn't be hard, especially if you just use the parts euler mode for each axis motor.

[@atharvvadehra3460]

Can somebody help please. I did all the code properly but when I run the blender code then it show Access denied on the COM port i use

[@checkeredbug8015]

Linux right?

[@Gaunza_]

Sir can you please give time and bpl library link from where we can download.

[@MrSinoRex]

when i type hello = ser.readline() blender stop working
why ?

[@swikarpoudel6704]

in arudino write serial.readStringuntil(
) . You must have forgetten that

[@tahahelwani843]

My blender gives Not Responding after a little of running the script? How can I fix this issue? Thanks a lot for this tutorial it's really helpful!

[@checkeredbug8015]

Its a windows problem with the screen update, my linux machine doesn't do it, I really spent some time trying, but no luck, sorry

[@tahahelwani843]

@@checkeredbug8015 Alright thank you very much!

[@eledikohabib3369]

Can you make a tutorial for using Bluetooth like HC05 with blender to get data from arduino

[@checkeredbug8015]

For bluetooth all you need to do is hook up the bluetooth device to the tx rx pins for the serial port to a bluetooth module. Then get a bluetooth usb adapter for your PC or download the arduino bluetooth app for your phone, connect to the module with the PC or the phone and use it like a terminal. It shows up in the pc as a serial port just like you are direct wired. create.arduino.cc/projecthub/mayooghgirish/arduino-bluetooth-basic-tutorial-d8b737

[@eledikohabib3369]

@@checkeredbug8015 and by the way...I tried to run the script in from GitHub but it's showing errors...

[@gustavosuarez7945]

Did you set any special option inside Blender to be able to update the screen in the middle of the Python script? I coudn't get the screen updated. It's always updated at the end of the script execution.

[@NauberSV]

Take the template called modal timer, native from Blender script templates. This script is executed while Blender functions still available. If you change any property of a given object (rotation, location, etc..), it is updated to the screen in real time. Esc to stop the script (or other command you put instead).

[@nickkeeline4840]

So blender uses a 'hack' to do this.
bpy.ops.wm.redraw_timer(type='DRAW', iterations=1) i have found this line is flaky in windows . I never had problems when I went to linux

[@gustavosuarez7945]

@@NauberSV Thanks a lot. That made the trick. I've also noticed that readline() doesn't work inside Blender, so I replaced it reading chars one by one.

[@NauberSV]

@@gustavosuarez7945 a couple of weeks ago, I started testing MQTT protocol to communicate the microprocessor with Blender, replacing serial communication. There is a Python library called paho MQTT, that allows Blender to subscribe Topics that are published by the microprocessor (an Esp8266, instead of Arduino). It made the communication wireless, and worked fine with the modal timer operator.
I suggest that you and Nick Keeline take a look at this possibility...

[@gustavosuarez7945]

@@NauberSV OMG, that's my next step to not depend on a wired connection. For now I'm playing with 3 sensors to see if can move an arm -at least-. Thanks a lot for the info.

[@izzaazzurri]

hi ,from the other comments, it looks like this doesn't work on windows, but your entire video are on windows, how do you do that?

[@nickkeeline4840]

It does work on windows, but the refresh or screen update freezes after random amounts of time, the script is still running, just no feedback on screen.

[@izzaazzurri]

@@nickkeeline4840 so how does blender still running in your video? is it depends on the version?

[@nickkeeline4840]

It runs for a random amount of time before the screen freezes, i just stopped recording before it froze. I have seen it run for several minutes without freezing, but that was on older versions of blender seems to have gotten worse in recent versions. Again it doesn't do it in linux.

[@izzaazzurri]

@@nickkeeline4840 okay thanks

[@JesTheii]

omg

[@jorgeguberte]

I started messing around with Blender recently just for fun and i thought about doing something with websockets, and then i thought "hey what if i can control whatever i make with an Arduino?" And here I am.
It amazes me that we have access to all this kind of technology very cheaply or even for free.
I'll be experimenting.