blendartrack 2.0 - ar motion tracking for blender
Вставка
- Опубліковано 1 сер 2024
- Facial animation transfer tutorial:
• blendartrack - ar face...
Camera motion tracking tutorial:
• blendartrack - camera ...
If you don't have blendartrack installed already, search for blendartrack in the app store to get the app or use the following links.
iOS:
apps.apple.com/us/app/blendar...
Android App:
play.google.com/store/apps/de...
The Add-On you grab at GitHub (requires Blender 2.9+):
github.com/cgtinker/blendartrack
Want to chat or need help? Join me on Discord! :)
/ discord
Want to support?
/ cgtinker
Timecodes
0:00 - introduction
0:47 - import camera motion trackings
1:06 - compositing feature
2:03 - facial animation transfer feature
Thanks a lot to Afonso Rodenbusch @Sketchfab for the stylised face: skfb.ly/o7Xpx - Фільми й анімація
the king!
Glad you like it!
Love your stuff, been watching you a lot when I started with b3d :)
holy shit dude, you are literally changing the industry with this rn, no exageration, you're opening doors that haven't ever been open to certain people
To make the technology available for everyone was my main goal, never was about the money. Glad you like it :)
no, it was my idea proof: ua-cam.com/video/LLFhKaqnWwk/v-deo.html
I guess he is one of the best developer who love open source concept.
CGMatter brought me here. Thank you for making this! I have been searching for automatic AR motion tracking for so long.
Great features, easiest workflow. Thank you so much for this. We use in Vfx Class...
Amazing, I am very happy to hear that!
Thank you for making Face Mocap possible on Android
really really appreciate it, thanks a bunch!!
wow you've gone a long way since the first release. well done
Wow. Such a great addon. Keep up your great work Buddy 😘
I am highly impressed with the amazing work you are doing to help lazy animators like myself is incredible, you deserve to hit a million subscribers on UA-cam.
Already been using your app and now it's gonna be more helpful in my workflow🙌
Wow! Great improvements! Thank you for sharing!
You are welcome!
Now I head back to the mobile app to fix some bugs :D
absolutely amazing work, well done.
You are a goddamn genius man!!! So talented and so helpful to the Blender community!! Greetings from Colombia! YOU ROCK!!!
Thanks :)
You are doing a great job!
Justo me agarras a mitad de un proyecto y estuve buscando soluciones para lograr algo así sin necesidad de un iPhone, En LATAM es un gasto increíble un dispositivo con esas características.
De veras muchas gracias por este gran aporte!
Thank you for your work.
You are a legend man seriously hat's off 👏👏👏💯
Thanks for this.this is amazing
You kind sir, just made my project posssible
You are a legend! Thank you
Thank you a lot man👍🏻
amazing work man
Very fantastic...Thanks!
Wow thanks so much
Yooooo..thanks again bless u
This app is amazing!
You are grate. A real Hero
Everyone should buy him a cup of coffee, this man is awesome!
amazing !!!
Thank you so much for this solution, when I already have income out of blender I will donate it to you sir
Thanks! Keep on creating :)
Thanks
You are the best brooooooo
This is the best app saver than iPhone... So that we are no longer jealous of the iPhone.
I'm an android user myself and really missed tracking applications. iPhones still have an edge due to LiDar sensors, but I was focusing on creating a solution that's (also) available on android.
@@cgtinker Yes... That's save my money... :) So I don't have to bother buying an iPhone because there are features like this. :)
@@cgtinker Thanks a lot!!! 🙏🙏🙏
This is freakin' awesome! As someone who worked on VFX and is teaching VFX this tool is invaluable! Sent you 10€!
Thank you, glad you can use it with your students!
Where are you teaching? :)
@@cgtinker I teach at the Institute of Technical Education College Central and also part time at Singapore University of Social Sciences. I used to work at Industrial Light and Magic Singapore. :) Been using Blender since 2008.
Great! In the settings, you may enable point cloud that may help you add obstacles in Blender, as is tree. Imagine scene where alien is there, but you do not want to see you. So, make cylinder instead tree, use cryptomatte node to 'move' the tree in the foreground. Not easy to explain...
Nice option:)
tinker great job the zips are killing me witch to open witch to use cant seem to choose right one again good work man
Great work my friend, is it possible to use a pre-recorded video?
T H A N K Y O U !!! :)
This is incredible! I played around with using the tracking data to set up a handheld shot of sorts, using an HDRI background to replace the background of the original video- and it turned out really great 👏
If I could ask, is it possible to change the camera's origin/position after the tracking has been added? I actually just downloaded it a couple days ago, so I'm a Blender newbie (the reason really being to play around with tracking software like this xD)
Big props for creating this program! It's just really cool what can be done with it 👏
Thanks 🙏
Yeah sure you can:
1. Create an empty object
2. Select the camera
3. Shift select the empty
4. Press 'ctrl p' and choose parent to object
5. Transform the empty to your likings, the cam will follow as child
@@cgtinker Thank you for replying! I have tried the parenting function, (actually the first time I did it xD) and the camera unfortunately doesn't follow the empty object. It worked on a basic mesh, though. Do you think there's something I might have done wrong in the process? I do think setting "Camera_Motion_" as another child of the empty object works.
@@vzm4663 yea my bad, that's due to the constraints - you got to do the parenting on the "camera_motion" empty, not the actual camera x)
I splitted it up in the past so the motion can be used by other objects (by just parenting objects to the camera motion..)
@@cgtinker Ohh, thanks for the explanation- I think I got it now, thanks a lot for your help! =)
Just wanted to note some things:
- The second github link in the description is broken
- Not animating the floating head with your tool seems like a miss
Thank you for keeping this proyect free
Fixed the link, thanks.
Floating head? It isn't hard to apply the parent animation to the target face, but it's way easier to separate the steps (less prone to bugs and easier).
@@cgtinker I meant your human talking head overlaid on the video on the bottom right corner. I thought the term was floating heads, or maybe talking heads? anyway, sorry for the confusion.
AHH now I got it 😂
Yeah didn't thought about that, good call
Amazing my friend can u used mediapipepose to make it full body mocap
Seems interesting, I'll look into it :)
Super App und cooler Workflow!🤘 Kann man das Audio in Zukunft integrieren ? Vg
This is great! Is there a way so that I can change the framerate that the iPhone records so I can match it with footage from cine cam? I'm attaching the iPhone to my main camera and mostly shooting 23.976/24 fps and then correcting the camera placement in Blender.
This is amazing! I have a quick question, do you know if problems arise if i delete certain bones in my rigify amature? My model doesn't have a nose or ears, do you think this will cause problems when I use blendartrack to animate my face?
If I match the frame and focal length of my iPhone to my Mirrorless camera and attach them together, would I be able to use the footage from my mirrorless camera as the background plate with the tracking data from the app? That would be so stellar!
I wonder if you could implement an additional feature in Blendartrack Facial Mocap, ARKit and ARCore works with Shape Keys internally I think, could be possible to get the output mesh animated with those shape keys instead of an animated vertex one?
Right now Rigify has changed its facial rig, also Cloud Rig and Auto Rig Pro are widely used, so I'm not sure how to transfer this mobcap to any of those systems, having the system working with Shape Keys could help to this I think, and could give us a bit more control over intensity of the deformations :)
Great Addon and app!
Thanks for it!
ArCore sadly doesn't - and I don't want to separate the application.
I won't support paid add-ons while not having a pricetag myself.
Cloud rig is widely spread? Interesting. Just seen it on some BlenderStudio characters. I'll look into it.
This is amazing. Are there any plans to allow changing the frame rate and shutter speed in the camera? Would like to get 24FPS and 48 shutter.
thanks!
this feature is not planned for blendartrack. I'm planning to develop a tool in the future for tracking with videos files on desktop.
One question, can you use this app for get only the camera movement? Not to add models, I want to get only the movement of the camera.
you take XRP?
you deserve it bro! TY!
❤️
Very nice! Whats the license for the addon and phone app? Commercial use allowed?
Ye allowed
ATM it's a CC lincese but I'll change it up soon. Guess will make it GPL.
can i use rigify quadruped, or is it only work with human rigify?
Hi. Would like to ask if Samsung a13 is compatible? Using android 12. But could not find the apps.
Really an helpful add-on, I'm using it for camera motion in my scene and it works really well, the only thing is I found a bug while using motion blur during the animation rendering, there are some frames in which the motion blur make some kind of glitches or artifacts, both in cycles and in eevee, they disappear only if I disable it; I also noticed that these glitches have a duration of 2 frames and if I set in the motion blur panel the position to "end on frame" for the first frame and "start on frame" for the second, the glitch disappear, still I don't know why, for the moment I'm just rendering in cycles with vector blur node in compositing as a work around.
yep it currently has an issue with motion blur, planning to fix this soon. Will take some time however because I have to change the apps source to do a proper fix.
the issue occurs because of discontinued euler rotations - you can select the camera, navigate to the graph editor and run the "discontinuity euler filter" on the rotations. I think it should do the trick for now.
@@cgtinker Thank you, I just tried and it works, I couldn't find a solution online, only other users with the same problem.
Not working for me the iOS app in iOS 16.0 (iPhone 11) - App crashing after loading the unity splashcreen
Can I use this addon with web cam
Your app is amazing it works so well on my 12 pro max, can wait for future updates like audio recording and more robust camera features and potentially a solution for mounting it on a mirrorless camera i would pay good money for that! thank you again for the app its great
Hello not compatible with Xiaomi redmi note 11 on Android 11 nor with Huawei P10 lite ? why ?
This is a great program, but I have a question.- Why is it that after the tracking ends, the recording seems to be sped up several times? Thank you
Hello!!
I have a XIAOMI REDMI 7 with android version 10, but the AppStore says that my device is not able to support Blendartrack!!
Can you tell me if this is true? Which requisites do i need to use Blendartrack??
Thanks in advanced
There's a generate rig face animation???!!! Waw!!!
I can't download the app because
It says it's not compatible....😭
Event tho I am using a fairly new mobile... It's a realm Narzo 30 pro.
Trying to figure out how to get this to work with the Human Generator models...
ua-cam.com/video/KD2xsO160xk/v-deo.html
Prolly this will help u :)
Is this app no longer available in USA ios store?
This seems like the ONLY facial mocap software on android! is there a way to use this with UE5 metahumans?
I don't have money to buy this
But it's a great aap
I will buy it in future♥️
Thanks! U can download it for free on gumroad if U enter a zero. Or download it from GitHub :)
@@cgtinker tq man love u❤️
@@cgtinker how can I install dependencies..I require external packges
@@cgtinker hey it's running in version 2.11
Tq
♥️♥️❤️❤️♥️♥️😁😁😁
great job.
Can you tell me the name of the software used to display every time you operate the computer keyboard, and the computer mouse.
That's a great tool.
Screen cast keys
@@cgtinker thanks you
Eagerly waiting for an update so that it works with Blender 3.1.
It's up on gumroad/GitHub
My bad! Thank u so much for the wonderful work 💖
hey, so when i try to move the camera to a different position, it doesnt work. im new to animation in blender so haha. also great add-on!
simply parent the "camera motion"-empty to an empty object and move it around ;)
- create an empty
- select the camera motion empty, then shift select the created empty
- press "ctrl+p" choose parent to obj
-> transform the empty
what about pre recorded video..?
Hey man you are amezing. But I can't download blendartrack on my phone when I want download said not compatible with version i really want to download. My phone redmi 7a
Thanks, sadly your phone doesn't support ar core and therefore not the app . Is there actually an option to download it in the play store? Because there shouldn't be.
I'll try to make ar tracking with webcam possible soon. Will be different but fun ;)
Video is shown as pink in blender. Please tell how to solve it.
Uhm thats blenders behaviour if the video length and the frame start / end in blender are not matching.
I agree with lots of the positive comments :D Just one question : what about audio ? Do we have to record it separately ?
yes, the current version doesn't record audio.
@@cgtinker So ! What's the point of such an App if I need to record sound separately and have to sync after ! I can do it, and I have solutions because I'm an 57 Y old man with 35 years of experience ! I'm sure you will figure out a solution because your are good at what you do ;)
Thanks for your answer
@@leconodium you are welcome. will be addressed once I find a better video backend. I tested a bunch but there are always issues :D
@@cgtinker I knew it ;) Thanks ! I'm a very patient man ;)
What is best record time in Blendartrack? 10s? 10m?
Usually I'm going for something in between 0-2min
this isn't for the motion tracking, but how do I get the video that was recorded for the face tracking? I want to composite it with the face track
That sadly isn't supported. Tried it in the past, took to many ressources and therefore crashed a lot :/
My camera track is imported below the floor. Is there any way to fix that??
same issue. Found the fix yet, fella?
can you use the app for camera movement
sure, when importing the footage there is an empty "Camera motion" - just delete all besides it and use just the motion (b.e. with constraints or parenting)
Can i merge dslr footage with this ar data
Found any answers ?
Anyone notice for the blendartrack app, after 1 min, the animation starts to go bad? You don't notice it in the app, it's only after you open the zip file in blender you notice either the file is empty, or there are keyframes missing or the keyframes are superimposed on other keyfreames.
on some devices ar core isn't that stable :/
but yea, tracks can go bad after a while, usually a minute is no issue..
@@cgtinker thanks for the reply. May I know when the app is actually being updated? Because I also notice the frame reate fps keeps fluctuating. It is only 60fps on the iphone for the first take, after that the fps just randomly goes down and never gets back to 60 fps again and is unstable.
Maybe I'm stupid but I can't figure out how to get the files from the app onto my pc. Any help?
select the folder button > select a file (so the circle icon is filled) > press the share button -> you'll be prompted with several option to transfer the data
Great Work Bro But the App is not working For samsung A12 Hope you Fix it 👍
Thanks! So far the galaxy A12 doesn't support ArCore which is required to run the application
For anyone that is having trouble with finding BlendARtrack and BlendArMocap here are the links
BlendAR Track 2.0: cgtinker.gumroad.com/l/tLEbZ
Blend AR Mocap: cgtinker.gumroad.com/l/rotom
Edit : FIXED IT
nice addon but everytime I render a video with the cameramovement from it there is a weird flickering bug from the lights. I tried making a new project file with new models and materials but nothing seems to work. I postet the video on reddit if you search :
Blender Render Problem: Flickering Lights
Need Help!
Dous anyone know how to fix this bug? When I render my vid there is a weird flickering from the lights. In the viewport there is no flickering.
(just copy paste in google) you should find my post. It doesnt happend when i render with a normal camera. I even tried replacing the camera from the addon with a normal camera because i only need realistic movement but it doesnt work. Would love if anyone help me because I wanted to post some yt vids. Or does anyone know an other way to make realistic camera movement like this? I know there is another app that can do that but its only for IOS and i have android
Edit: I found out when i made the keyframes from the camera movement smoother in the graph editor with alt+o that at some points the camera makes a quick full rotation which should not be there. at exactly these moments there is the flickering when i render my vid. So there is some bug that causes this. I just dont know if its the fault of my phone or maybe the addon
und übrigens bin auch deutsch
Great app for the face cap. Can it track the eyeballs ??
no, it cannot
@@cgtinker ohh ok, can we animated the eyes later with the mocap datat on it??
@@bhushankandalkar3104 yea the rigify functionality is still there, so it's pretty easy to keyframe :)
@@cgtinker that's cool !!
How do I move the Camara where I want?
Parent it to an empty obj and move it around ;)
Oh thank you!
Link to Android APK ?
this does not work in 3.1, 2.93 though works great! this helped resolved my issues, i might try to update it myself in the future but im use to 2.93 anyways so i shall stick with it for now
what doesn't work in 3.1? are you using blendartrack v2.1.1?
@@cgtinker there were errors in python, I was using the latest download I could find aswell yeh. I did try to fix it but it didn’t want to work, so I just imported it in 2.93 then went back to 3.1
@@Volt64bolt whats ur os system? (wondering as I've tested blendartrack v2.1.1 in 2.9, 3.0, 3.1, 3.2)
@@cgtinker windows, it kept giving some form of file not found error even though the code it was pointing too had nothing to do with files, so I think it might just be an issue with the fact that I have 5 drives on my pc and blender is on one, and the save file I was in another, and the tracked file was on another. So it most likely is my end with something like file permissions but it works now so all is well. There is a little bit of jitter in the track but that’s not too difficult to clean up manually and it has turned out great so far! The trickiest part was the lighting but since I have an android phone I used he photo sphere mode in camera to make a hdri of sorts, then I just took the brightest points and made hem emission shaders, and used the rest as a reflection sphere thingy at its great
The android and iOS links are broken in your description
fixed, thanks for info!
can someone tell me, if that is for free or not? cause Im a bit confused with the website and I am unsure if I have to pay for it.
you can use everything for free, even commercially.
Thank you soooooo much for this beautiful app and for helping us poor artists! I'm creating a show and would love to use this for my characters. I will definitely have you in my credits(if I should get paid, I shall send some your way)If possible, can u make a body motion capture for us?
Very cool, hope you will be successful! :)
Will try to make pose capturing possible in the future
Hello! Could I ask you for some help?
I'm getting strange artifacts after importing and rendering my animation from the camera. Any idea how I could fix this?
What kind of artifacts?
@@cgtinkerAfter importing the camera motion and checking the animation through the viewport, it's fine. But when I start to render, it starts out normally, but some frames look almost as if I've spun the camera around really quickly. I'm unsure if this is a motion blur glitch or just a problem with Eevee.
@@Razer_Dash there could be motion blur, also you might have to record in lower dimension (usually it defaults to 720p, recording in 1080p can be hard for some devices). Usually, the video background is not affected by eevee / cycles, but you can test this by just rendering some frames in cycles and compare. If there are differences might just render with alpha and combine the two clips later?
@@cgtinker Thank you for your answer. The motion blur bug is in both Eevee and Cycles, I checked to see if there were any keyframes offset and there are no problems there. I recorded the motion outside in the sun, the floor was exactly where I wanted it to be and it still glitched. I'm also recording in the lowest resolution for good performance and in Blender, my resolution is set to 640x480.
I'm on android , is there any way to fix the frame rate sir ? depending upon the scene and movement the framerate varies from 26fps to 29fps
So far sadly not..
@@cgtinker Thank you for the reply.
I'm facing another issue. The recorded video is appearing as low quality, when I'm rendering using the "internal compositing" . Even the .mov video saved in the zip is of 640x480 (my phone supports 4K) .
@@muscle__rr in the app settings you can set the video recording quality up to full hd (4k aint supported) - however that drastically increases the required performance. So try hd first and check the fps before going fhd =)
hi. thank you so much for creating this app . can you makr it available for redmi note 11 pro. it says its not compatible. thanks
Sadly not the app requires ar core support on android and arkit support on iOS. The app depends on these subsystem.
@@cgtinker hello thanks for the reply. but i have a xiaomi pad 5 and i saw on google ar supported devices. it has been added to arcore. but i still.cant install your app. i hope you can help me thank you
@@kimsjoyfamily yea according to google ar core should work on xiaomi pad 5.
can you give me some more details via mail? hello@cgtinker.com
maybe we can figure it out.
do you get an install option in the app store? does it crash while installing? does the app install and when executing crash?
@@cgtinker Hello. Seny you an email, but basically,
1. I can't find the app ni playstore
2. When I click the app link in description, it goes to the app in playstore, but it says not compatible and I can't download it.
3. I tried pureapk, to download the apk, and I can install the apk, but it crashes when I try to open it.
thank you so much
Hello @cgtinker, still not compatible with my device even though it is on google ar. Thank yoy
is there a way to import it as a cube ?
The motion? Kinda. Unzip the zip, select a cube and import the .._cam.json. It will throw an error but works in that case
@@cgtinker alright thanks that way i can import the animation into a game the camera is attachtopart so yeah
PLEASE Im new and dont know how to do this PLEASE show how to import the file from the app because it doesent work for me.
When I press the folder icon you'll see your recordings. Select a recording to export it by pressing the circle next to it (,so it's filled) then press the share icon - you should be prompted with some export options.
@@cgtinker hi yeah so Im not sure what export option to use. do you have a recommended one? Thanks for responding so quickly tho
wait actuslly turns out Im a complete idiot. dont woryy i found a way to do it. Thanks for the amazing app!
I have to ask. The video goes silent after I record the video. Why is that?
goes silent?
@@cgtinker Yes. The video doesn't include the audio with it.
@@brandonneumann4043 yea there currently doesn't get any sound recorded - not a bug, just as it is atm
@@cgtinker Oh. Okay, thank you.
can we use cycles render for this?
Yea sure
@@cgtinker thanks for answering, but when I'm use cycles render, it seems there is a bug in my blender, when i try to render, the object looks like it has no texture
@@RizkiAchs uhm in this video I didn't apply any textures nor I set any lightning. :>
@@cgtinker yes I see but when i try to create my own tracker, then insert objects and add lighting, when i try to render it, the object is blank like it has no texture
YOu can charge for it bro. there should be a minimal price for such a useful addon.
Sir I think you use python language for programming....?
Well for blender add-ons that makes sense :p
@@cgtinker Sir now I am start learning python...I am very motivated from you.....It is possible to build this type of android apps in python ...or I learn other language..I confuse. ...please answer me
@@akshaysaundane9520 for android development you shouldn't use python.
Python is a great and fun programming language to get started or to run scripts directly in Blender.
For Android development checkout "AndroidStudio" by google and watch some tutorials (it's the way to go to build native apps). The language is Java or Kotlin in this case.
I've developed the mobile app in Unity which is a GameEngine with great Augmented Reality support - love the engine, however while it's great for AR / VR / Game Development etc, I wouldn't develop standard utility apps with it. In this case the language is C#.
It really depends what you want to do, I think it's a great start with python in blender or C# in Unity - but you can also head straight into Java and build native apps.
project > engine > programming language
@@cgtinker Thank you Sir