blendartrack - ar motion tracking for blender - release!
Вставка
- Опубліковано 1 сер 2024
- Check the links below! :)
With blendartrack you can make use of camera motion and face tracking technology and export the tracking results to Blender. For that purpose, the mobile device requires ArCore support on Android or ArKit support on iOS.
The tracking data can be imported to Blender by selecting the created .zip and pressing 'Import'. Here some more informations if you run into issues:
• blendartrack - beta ad...
Search for blendartrack in the app store to get the app or use the following links.
iOS App:
apps.apple.com/us/app/blendar...
Android App:
play.google.com/store/apps/de...
The Add-On you grab at GitHub (requires Blender 2.9+):
github.com/cgtinker/blendartrack
Want to chat or need help? Join me on Discord! :)
/ discord
Want to support the project?
/ cgtinker - Фільми й анімація
A fantastic contribution to the blender community! Thank you!
For real!! \m/ Big Ups!
I love this thing. Makes the job 10000 times easier than messing around in blender and loosing my mind cuz of shaky results. Much love 😄
love you buddy! this is absolute priceless und an outstanding example for this awesome community around blender
Please dont ever stop making and updating this!!!!!!!!!!
you can't believe how long i've been looking for this app for android
Works really well! Good job!
hi .. could you help me. How do you import the facial data from blendartrack, what addon do you use for blender?
This is amazing, i have subscribed!
That was so amazing ❤️
No doubt you are legendary developer. AMAZING 🤩
Hi , Trying this out today , if it works well , will definitely support the project ! thank you for making this !
You are amazing!!!! Your work is incredible!
Amazing Video! Thank you so much.
Wow! Thanks a lot! Instantly supported/subbed! -RT
Thank you , short and easy understand
This is so cool thanks alot for the contribution
👌 THANK YOU FOR THIS ADDON
This will be so helpful thanks a lot
something cool you can add is to have the facial tracking be linked automatically to blender and not have to export it and send it to your computer so basiclly have it live.
That's massive 👏
Thank you so much :)
Amazing 👏👏👏
Amazing work, thanks for make this free!
Thanks ;)
@@cgtinker Well, actually I have paid £1.20 pounds that means in dollars $0.20
@@Mr_Eduard108 u didn’t have to pay though.
YOU ARE A GENIUS
thank you! you are my hero!
I wanted to write soft for this task, but you did it first!
Sorry, I can not support you with money, cause of law restrictions in our country
Great App!!! I love it
You are the best!
finally for android!! thanks to you.
Yes... I used to be jealous because it was only on the iPhone. Now available on Android...
Your are amazing bro ....
GENIUSS!
i love you man!!!!!!!!!!!
u are a life saver
wow wow, awesome
Amazing
thank you...awsome app
Amazing! Now I can animate camera movement - perfect for my Backrooms series!
How to do camera motion with this app? I still confuse
thats why im here too lol
For android users. If you can automate the setting up of a rig/bones from empties it would be easy to then create a pose library and save poses out as shape keys.
the face mesh vertices don't share the same positions, as the geometry is not accessible on android. I'm basically working with vertices in space. Therefore the geometry depends on the users face and a generic approach is fairly hard. It could work for users which set it up for themselves though...
I hope in the future shape keys will be accessible on android, this would make it much easier :)
@@cgtinker Hi, if you can get the Blender plugin to add animated bones it would be possible for users to save out some of the poses as shape keys. This would be a manual workaround but may just work.
@@ian2593 I'm planning to add a feature in the near future which improves working with rigs or create a rig - still testing around, will take some time I guess :)
Amazing app
This is awesome! Can you mabe tell me why my backround video wont show in the render? I see it in the live render, but not in the final result. I did the same like in your tutorial, I also added an alpha channel. But now the background is transparent instead of colored..
thanks!!!
Bro...This is so clutch I can't even! Is there any way you could enable it to record pro-res as the video plate on the iPhone 13 Pro? If you could record Pro-res, plus motion tracking data, you could literally use this for cutting-edge visual effects.
woooooow what how is it possible that not 10000000000 mio people know that??? soooo goooood
Very good app for tracking. .. . But if we want live action vfx with a person or subject we need atleast to shoot the video in manual mode. Kindly add manual camera settings on the app. . So that we can control camera parameters from within the app. It will be of immense help on adding realistic vfx with professional video quality. Thanks.. . .
Hello! how to move the start of the animation to another location in blender? I can't move the camera from coordinates x=0, y=0; when playing, everything returns to the beginning :(
this is amazing!
Do you think it could be possible to calculate realtime tracking data on a video stream that comes from an external video sources?
For example, I'm imagining a camera delivering video stream inside the smarphone.
Thanks!
I think about trying to make it possible with prerecorded videos in the future.
incredible contribution to the community, you are a legend. can you show how you got the face to snap so well to an animated character? i have tried with shrinkwrap and I can't get it to work well. are you just using basic parenting?
in this case, it was just parented.
atm I'm working on a motion tracking data to face rig transfer - will release it soon
@@cgtinker That's awesome! I love the way this exports btw, with regular XYZ lanes instead of shape keys, makes editing (particularly smoothing) a breeze!
미쳤어!
Awesome app! Just wished there would be a body motion capture as well! :)
ua-cam.com/video/pji6IHNCnAk/v-deo.html
Check out his last video :)
I have an issue when importing, the track is incredibly jittery ... is there a reason for this ?
Good 👍😊
Very cool! What does the "video bitrate" setting control? Doesn't seem to have an affect on resulting video bitrate. Also, I second the request for more camera controls. Thank you so much.
You are welcome :)
Not sure why it doesn't work. Sadly the developer of the recorder I've been used stopped supporting the video recorder. So I'll be using a new system for recording videos in the future which will change quiet a bit... Even consider to write my own video recorder but that's quiet a job.. :D
Do you think it is more realistic than the pftrack for matchmove ?
Is it possible for this to work on bluestacks like at all with a webcam?
Can I mount the Phone on a better camera and retarget the Track data to that footage even If Sensor size / focal length differ a bit?
thanks
i was just thinking of something like this the other day, after trying myself by using ARCores .record() function and trying to extract the motion data from the .mp4 file/injecting camera pose into it to extract i found many issues. then i found this whilst doing research and although i would rather use my own as i am just stuborn, this works well enough that there is no need for me to finish mine!
TLDR: this is easier than making my own
So U found a proper way to inject mp4 files? Seems cool, are you working with Android studio or unity?
Wow, I just realised the reply I wrote yesterday didn’t post. Sorry about that, I will re write it shortly and hopefully UA-cam doesn’t break this time
@@Volt64bolt interesting, thanks for the insights :)
By now my project is getting a bit dusty and I will have to update quiet a bunch. At the time of development, scene form wasn't available for unity, glad it finally is. It's not likely that I'll implement this concept while it could yield better video quality.
Unity is a great and fun tool, keep it on!
@@cgtinker thanks, I might revisit this at some point in the future, if I get it done before you renew all of yours then feel free to take a look and use anything needed since it will be up on my github. Although I doubt my code will be of any use but just in case it is
@@Volt64bolt don't plan to do so. here the link to my repro, feel free to link yours!
github.com/cgtinker/blendartrack_app
I've loved using this, the tracking is accurate and the results amazing - any plans on making this compatible with Blender 3.x?
It should work with Blender 3.x. If you had any issues installing it let me know the version please ;)
Will there be an update that allows you to record the sound also?
omg ima make the backrooms with this thx I have been using camera shakify but that doesn’t work as well as this
is there a way to set the FPS/quality of recording and also to put an nodal offset ( e.g. mounting the iphone on a Cinecam rig)
Currently neither is supported. ATM I am focusing blendarmocap but I consider to add an option to lock FPS in the future.
Any chance we get an update? Currently I cannot get it because the play store says it was created for an older version of android. (Pixel 8 Pro here)
Hi! I was wondering if you knew how to move the camera once it is placed in the scene. I've tried moving it by the Camera_Motion_ axis, but when I try to play the animation it immediately snaps back.
Parent the camera motion to an empty and move the parent ;)
a fucking legend
Is it possible to sideload the android app? It is not available for my Pixel 6 Pro
How to share recorded file to pc ...?
I have a doubt sir. While capturing the video, I found that the final video being recorded seems to be in time lapse or what you could say is kind of of very low fps. Is this my problem? What could I do in rectifying this? Anything to do with bit rate or something? I didn't find any options to correct it in the settings. Please help
What is the minimum compatible on android? My phone can't download it, it says "Your device not compatible in this version"
Bro please add video stabilization option in app. Because the app doesn't using camera stabilizer.
How to get the face animation from blendertrack to a makeHuman model?
It's cool. But I don't get the original footage as background when I import the zip file. It only gives reference & tracking points.
did u record a video (has to be enabled in app), also, did u press "numpad 0"? to activate the camera
This is so cool. Does it capture facial animation on Android? In Blender ( the addon) What is the "file path" supposed to target? the captured file? Do you have a documentation page to read the features? Thanks!
thanks! yep it works on android, atm face mesh tracking is supported - I started working on transferring the facial animation data to rigs, but its harder than I thought :D
The file path should lead to a .zip. If you export multiple files then several zips are stored in a zip - if so, you got to unzip it. Otherwise, if you just export one (.zip) file, you can use that as path target.
I don't have a proper documentation yet, will try to improve the current state soon:
cgtinker.com
@@cgtinker what sensor does it need. whatever phone but what sensor.
Is there a way to render with the background video being included? It only rendered the object. I suppose I could use an editing program and layer the animation on top of the bacground plate, but I thought if Blender 2.9 is capable, I'm hoping to include the original video
Look up for videos on compositing 3d objects over movie clip blender
In render properties, under Flim...check if transparency is checked...try toggling it
Excuse me please, am I able to animate a rigged character with a various format after when the camera motion tracker has already been imported?
I don't understand. The camera tracking animates the camera. The face tracking results can be transferred in the same scene
I cant use blendar tracking because it is meant for an older version of my android phone, which sucks.
Hey! Thank you for making this, for facetracking I keep getting 'cannot preview data - it's probably corrupted. th .json may contains a closing parenthesis error' Any tips on how to get it working?
Can also be to large for the in app viewer - did U try importing it to blender?
@@cgtinker hey thanks for writing back, I could not get it to work so I am trying the other 'media pipe' method on our channel, but I keep getting this error when I click 'transfer animation'
BlendArMocap-main\src\cgt_blender\cgt_rig\abs_rigging.py", line 123, in edit_bone_head
position = self.armature.data.edit_bones[bone_name].head
KeyError: 'bpy_prop_collection[key]: key "lip.T" not found'
location:
i can only move the face it doesnt have any gesture, one of two empties does that i suppose?
Yeah, I parented the face to an empty. The empty can be transformed and the face will inherit the transforms.
Hey great app, you know how the background shows up from the original video you recorded when you upload it to blender?
I was just wondering if there is any way to remove it. Thanks heaps
Click the camera and disable the background images
Nice!! :O Do it combine/import face video to background so yo can build fx over face mesh/bg video?
Sadly I was not able to get video recording and face mesh tracking to run simultaneously. It's just to performance heavy :(
But I love the idea, hope I will get that going in the future without roasting phones ☺️
@@cgtinker That would be nice feature. So, it's heavy to merge camera data to scene / head mesh rotation and same time recording video? Maybe on faster phones :) Keep on rocking mate! It's already awesome program! \o/
@@donjoey4627 thanks!
yep I hope I'll get the running in the future, would like that swell. I nearly roasted my phone while trying the last time so I'm bit scared xD
but I think it's possible (heavy one the hardware for sure but possible!)
@@cgtinker yeah I was wondering if it could get the video and the track at the same time. Or alternately, if there's any way you can record video then AR track the saved video file? I assume probably not, since i'm guessing depth info would be lost when the phone encodes the video?
Either way, it's an amazing app!
@@frankbardelli7950 as mentioned it sadly takes up to many resources. maybe I find a way to improve that in the future.
I don't find a lot of resources about it, but it seems that ar core has an option to record videos with additional depth and acceleration data. the ar experience then probably could be added on the pc. guess that's a project for another app. it would allow to record a video and do the tracking after.
interesting but not sure if it's worth it.. I'll keep an eye on it.
Hi, thank you for this incredibile program.... There's any Android smartphone where you allowed 4k 30p recording?
you are welcome!
currently the max resolution is capped at full hd as higher resolutions aren't supported by ar core
What should I do if I have json instead of python??? I can't see the right format in the blender.
Use the add-on 'blendartrack' to import the zip containing the JSON files
is there a way to disable the video and only use the camera motion to like, make backrooms videos and that stuff?
you can just unlink the video, then you have just the motion. The motion of the camera is just transferred using constraints so u should be able to figure out to even throw away the entire camera
Is there a way I can motion track the shot with my front camera? (i want to film vlogs and then composite my virtual character into them :)
I don't fully understand - camera motion tracking uses the back camera, face tracking the front camera. Afaik that's not interchangeable atm.
Also the fps keeps changing while shooting. .. . At least kindly make it possible to control the exposure settings and fps while shooting
Is there any way to get record the video while face tracking and get the footage in Blender ? The same way it does it for camera tracking but for face mask basically :))
sadly not at the moment
Please make the app compatible with Android 6.0....Please
can you help me?? I made the video step by step, but when I install "blendertrack" and add it, it doesn't show up for me
is there any bug in the console? The name in the side panel should be "BlendAR". Did you press "n" in the 3D-Viewport?
video is not visible in render view any know why
Can you please make this for full body motion capture
Hi There. i took all the steps but when i try to import the cam track file it shows error. I don't understand why. can you please help?
can you copy the error for the debug console and paste it?
Do you just add the face mesh to the full body character? I don't know what rigs support that face rig
there is none - at least so far, working on it :)
on the character I added the face mesh. the rig shown in the video sadly isn't really functional enough to provide it for general use.
@@cgtinker OK, for now I'll just merge it with the mesh and weight paint it to the head/face
Great App. Thank you. Can you enable Mic/Audio on the Video file?
Thanks! Maybe in the future, currently it's not planned. Focusing on a face tracking update and maintenance for now 🙂
@@cgtinker good app, thanks!!! but the audio it's important for resynch tha face tracking. i view with the front camera don't record the video, why? am I making a mistake with some settings?
Why is the camera moving so slow? When I run in my recording my camera moving just slow
Are you using iOS? If so, it usually record at 60fps - just select all in the scene navigate to the timeline and scale till its as fast as you want
the background is not showing for me in the final render
Press the internal compositing button ;)
Check out the blendartrack 2.0 vid if you do not find it
i cant download this addon, how do i do that?
How to do make camera motion with this app?
its saying this device wont work for this app :(
Badly need this for Unreal Engine.
There are no apps for Android that can work with Unreal.
hey thank you so much for this app,
Have one question
Been really struggling with tracking since I started blender but this will give my channel new hope : )
Just I wanna ask you can I Create vfx shots for my Channel using this app?
Asking this because of this below line I didn't understand the last sentences....
( Resources and tools I provide are free for you to use, but running the project and content online is not. )
oh sure you can! it's totally free to use for you :)
Maybe I should cut that line... it's that I have expenses doing this (like getting gear, hosting etc..) so I dunno how long I can keep it updated
@@cgtinker Hey thanks again, I will definitely test it and mention you when I upload something using it on my channel for sure : ). This app is revolution !!!
@@cgtinker I got confused that's it Don't cut that line also...people should know how much efforts you took and still taking to make this app work : )
Best luck for that .
@@1neManShow oups - sorry I was just thinking about removing the confusing part from the description! :P
Thank you and good luck swell :)
@@cgtinker :)
Thanks for the amazing free app!!
When using the Android version, there is an error that the app freezes when you increase the camera quality. When I run the app again I get the same error. So I have to reinstall the app. (Even after reinstalling, the picture quality is lowered back to 640 480. Is there any reason why this picture quality is the default setting?)
Is there a way to normalize the picture quality to 1920 quality?
The phones I use are Galaxy Z Fold 3 and Note 20 Ultra.
Not sure what's causing this, this bug pops up way more frequently by now. Maybe there's been an ar core update? Or more low spec devices are getting used. Not sure what I can do about it as this bug goes hand in hand with the ArCore subsystem..
the downside to this app is you can't make a "found footage" arg cuz you can't shake which is required for authenticity to make it feel real
Hi! Very cool Addon. Love it! Unfortunately for me, is not supporting my phone... I have a Samsung SM 217 M... Any idea why this could be?
the app requires ar core to run. so sadly your phone isn't supported
@@cgtinker does it work con an ipad?