FreeMoCap Rough Cut Tutorial/Walkthrough/Brain Dump (Nov 2022)
Вставка
- Опубліковано 1 сер 2024
- NOTE - This is a work in progress kinda thing! This video was recorded in Nov 2022, if you're watching it significantly after that, it is likely out of date!
Website: freemocap.org
GitHub: github.com/freemocap/freemocap
Twitter: / freemocap
Mastodon: @freemocap@fosstodon.org
Twitch: / freemocap
Discord: / discord
Transcript for this video: share.descript.com/view/vsqpq...
This is a quick and dirty tutorial/brain dump about the use of the freemocap software.
It uses a pretty obnoxious workflow the involves recording synchronized videos in the `pre-alpha` v0.0.543 version of `freemocap` and then loading those videos into process in the `alpha` version of the GUI. This the current 'best' method of getting the best recording/mocap output possible, but hopefully that will change Soon™️as we develop new iterations/interfaces
The intention is to use this video as a reference point/brain dump that we can chop apart into viable documentation and knowledge base. We'll make a new 'how to get a freemocap recording' tutorial when the software hits a new stable point, which will be, like, ya know... whenever. As ASAP as possible. We'll get there 😅
0:00 Introduction
1:03 Cat 😺
1:09 FreeMoCap software installation
8:44 Camera Hardware (how to pick your webcam!)
13:08 Camera Setup (How to point your cameras at things!)
26:55 ChAruco Board
29:45 Recording Calibration Videos
32:25 Spot-checking Timestamp diagnostic plots (did the videos record good?)
33:01 Recovering from a crash lol 😅
36:31 (auto) synchronizing videos
37:00 (auto) processing calibration videos
39:54 Looking through a (pre-alpha) session folder
40:50 Recording motion capture videos! ✨
44:22 (auto-ish) processing motion capture videos
48:07 Cat + Cat = CatCat 😾😼
48:32 (auto) Exporting as Blender `.blend` file ( rigged armature for animators - filetype: `.blend`)
49:54 Examining Blender output file
51:31 Success! :D 🥳🎉🎊💀✨
53:09 Looking through an (alpha) session folder ( raw data for scientists - filetype: `npy` and `csv` )
55:29 Wrap up and goodbye!
56:47 ❤️ - Наука та технологія
cat appreciated
Absolutely brilliant work, Jon! Big thanks for your generosity! Keep up the amazing work! Subbed!
Wow! This is incredible. Thank you so much for developing this project as open source. I will definitely try that out and maybe contribute some code in the future.
Such tools could really become useful for indie game developers for example.
I really appreciate your work ❤
Btw, those sweet cats gave this great video a nice cozy touch :)
This is the most incredible thing ive seen! this is exxactly what ive been looking for! I hope you have more success on anything else you do because this is the best!
Keep up the amazing work, Jon!
Wow! Absolutely amazing work you've done here! Thank you so much! Really happy I found this :)
Thank you Jon, and please don't stop
Looking amazing so Far!
This is a very cool instrument. I can't wait to see the user-friendly interface of the program. I wish I could help with feedback about the program, but it's hard to use in the alpha version right now. Thanks for continuing to work on the product.
Thanks!
Yeah, I've been busy with other stuff recently but hopefully we'll get a viable software workflow together "Soon"
Do you know. It's 2 o'clock at night in the country where I live. But I didn't give up and I won. With 1 webcam and 1 smart phone. 1 solution seems to fail. But I got it working. Thanks to everyone who supported me. Special thanks to Jon Mattthis. The creator of this interesting method.
I will have a detailed report on this.
Sorry, but how did you get your phone to work with it? i cant get it to work.
You're awesome Jon keep up the work!
I'm looking for mocap softwares and yours looks the most promising to me.
Subbed btw ~
Incredible cinematography.
Detailed tutorial! Thank you very much!I can never detect my camera when I use the GUI, but it is possible to use traditional methods~ha ha 52:48😋
Keep up the great work!
Love your work and adore your kitty, it drew me in. 😊
Don't know if the gravity problem was solved but maybe you can use a specific calibration board for the floor. This way the software can recognize and define the workplane.
Awesome work, bro! I'm really looking forward to test it
Great Job !
amazing tutorial , thankyou very much
I really enjoyed the features from Poo, please bring him into more video, he strikes many alluring poses in which I am utterly captivated by his luminous shimmering curves. As for the technology? Eh.💩
great works. Is it possible for real-time MoCap?
seems great !!!!
I didn't find anythings on your website about the hardware setup needed for this.
Could you please detail it ?
Thanks !
EDIT: okay it is in this vid ;-) sorry for bugging you :P
will the result be improved? if I wear a special suit for motion capture? for example a black skinny suit with a white dot at the joint position
This is awesome,Jon! waiting for it.
Hi, great stuff. Is it still in active development?
It's possible to made recognition in realtime? I can connect to dev team and work on this feature. My last project is about Google-it(ORB SLAM 3) tracking.
Will single camera work . Given a 10 second single video file that contains one person walking can this software generate fbx files that contains the animation info?
Just here for the cat 😀
Same!!!
Poo is key dev 😼
This is VERY cool. Does it only work with webcams or can it handle multiple video files recorded from different angles say with DSLR?
It can handle multiple precorded videos from different devices like smartphones, DSLR, etc. They have to be synchronized though. The GUI has an import option that can sync using the audio track or you can sync them in a software like DaVinci Resolve
Hi, this is pretty wild! If I use a couple android phones and want to connect to my Mac as wifi webcams, seems I have to use software such as Camo. Will this sort of middle man software impact FreeMoCap? Thanks. BTW - maybe have the developers work on a constraints system so that joints/bones are never allowed to undergo impossible deformations. Also, will it work if you are holding an object? I am trying to find a way to do motion capture while playing my guitar, in which case I am seated, also at times walking with or without guitar. I don't need accurate capture of my hands playing guitar. I am using mocap to generate 3D models to then cast shadows and reflections in 3D scenes. When I film I don't show my whole body, but if I have android phones as webcams for the capture do you forsee any issues doing mocap in my use case? Thanks again!
Hi, there would not be a problem in using android phones as webcams. But you can also use the prerecorded videos option where you record in each device locally and then import the videos to the program. Regarding playing the guitar you could overcome occlusion by a good placement of the cameras. I think it should work fine if you don't mind a not ultra accurate finger capture.
@@fluxrenders great, thanks for the info!
38:30 - cmd can freeze execution when you select text 😢
Can FreeMoCap work from single camera/video ? lot of thanks
Where did i get toml? I have videos (downloaded), i install gui, i run gui and nothing, when i click Process All Step Below
best mocap #mocap
hey Jon ending up copying the installation notes into chat got and it helped name install it
That is so promising!
I have 14 ps3 eye cameras which has 60 fps option (lower resolution though). Do you think resolution of FPS will matter more?
Will it be possible to add DSLR footage afterwards? I assume it won't be able to go through calibration process with DSLR.
Can it capture 2+ persons?
i get dark images using -7 exposure on my C922 pro cameras.
1000 thanks for making multiple camera markerless mocap accessible to us! If I may ask: Is this tutorial 1:1 applicable to the latest version of freemocap?
No, the latest version has an improved GUI, workflow and Blender output.
And, will you make a new version of the video witht he new GUI workflow and blender output?@@fluxrenders
Well, a new tutorial is necessary indeed. Problem is lack of time and also that in this stage the tutorials get outdated pretty quickly.
blender :/... why not a FBX exporter? The one and only animation format to get the animations to EVERY software?!!!! However, I am totally unsatisfied with the usage... made it running but I don't see any production usage this way. Thanks anway! @@fluxrenders
I posted a detailed replied but it seems it was filtered out :( . At least I want to say that the FBX export is considered but not a priority as you can already export to FBX using a Blender addon. You can even export a FBX suitable for Unreal Engine. There is a video of that process.
Thank you for your wonderful work, it is truly useful to DIYers like me. I have a question: I used FreeMoCap on my system previously with a single kinect V1, no issues. Tried moving to multicam but now it won't detect any camera, even the previously detectable kinect V1. I've tried going back to single cam, but it doesnt work either. For info I have used 1-6 Ps3 Eyes, and/or 1-4 Kinect 360 sensors, and/or the aforementioned kinect V1. but not a single camera is detected. Used Ipisofts recorder to see if it was a bandwidth issue but it detects all cams, and runs them as well. Any advice? Reinstalled skellycam as well but no luck.
A workaround can be to use prerecorded videos (with OBS for example) instead of recording directly with FreeMoCap
@@fluxrenders thank you for your suggestion. I could record multicam using Ipisoft recorder, but no charuco board plus data means that wouldn't be very useful in FreeMoCap, right?
Yes, you need to record a calibration with the charuco board and then get a calibration file. This file is used to reconstruct the 3D skeleton when processing the capture videos.
@@fluxrenders I dont think I can do that out of the box with OBS or IPI... hmmm... thanks for the idea though.
@@fluxrenders do you think only one instance of type of camera is only being detected by skellycam because they have the same hardware IDs?
Hey Jon, great work really appreciate it 🙏 can you please help me I am trying to create a real-time mocap solution as a final year project for my university where I am trying to integrate freemocap with unity to get the tracked data and map it to a character in unity 3D
Is it possible? if so, then how
@@devpatel8276 auto-rig & skin your character when the animation dataset is calculated, then your character will follow the animation I guess. Better use simple models at start
I just did my yearly search for open source motion capture tools. Is it finally happening?
If so, this is big news, incredible stuff
is three the minimum amount of cameras that will produce moderately usable results? And proprietary options can, maybe with less accuracy, achieve this with just one camera and no calibration, is there hope of achieving that with these tools without drastic refactoring? Genuinely curious, and the current results look fantastic.
i cant use the gui,seems some packages cant be installed,😭😭i dont know python,learned it while trying,when can a ui come up
Hi! Thank you for trying to get this running! I know how challenging this kind of thing can be when you're new to programming.
Make sure you have done all of the steps and entered all the commands *exactly* as they are shown here- github.com/freemocap/freemocap#how-to-run-the-alpha-gui
If that doesn't work (or if you get confused about one of the steps) - please create a new Issue on the GitHub repository here - github.com/freemocap/freemocap/issues
When you make the Issue, please be sure to copy any `error` messages that come up in your terminal!
祝你好運！
Are you using the conda instructions? And missing PyQt6?
I switched to PyCharm and got it working that way.
I give up. :-( I can capture calibration data and record videos, but none of the other buttons in the GUI do anything. (Tried on my PC: Windows / Intel i9 64GB RAM / 4090 24GB VRAM. And tried on my laptop: Ubuntu / Intel i5 40GB RAM / no GPU.) No install errors, GUI running on both. Calibration toml generated successfully. Videos recorded and saved to disk. Then... nothing. The GUI buttons print statements to the console as if they meant to launch processes etc., but nothing happens.
Hi, you might want to ask in the server. Maybe we can find what the error is by checking the log files.
@@fluxrenders Thanks, I will probably try again when I have time. I got another solution working, but the quality was unusable. (And I found out how bad I am at moving my body for specific animations.)
Question, can i use one camera?
Yes! The latest versions of the software have a "single camera mode" that produces flat skeletons (compressed onto the Y Plane). Its very quick and easy and loads into Blender just like a multicamera system
@@jonmatthis Amazing, thank you, and, im really sorry but may i ask, Do you have a video on how to install the program? I got lost along the way with the text/website Tutorial. Thanks :D
If this gets refined, might help alot of solo dev's using unity or unreal engine. maybe xbox kinect would be better
You can already import the Freemocap Blender output in Unreal Engine. Not perfect but depends on the recording setup, the retargeting, etc. But it is improving. ua-cam.com/video/QcFmZ_eex0U/v-deo.html
Want to bring it to UnrealEngine, but its AGPL license.
The license does not apply to the output of a program so there won't be a problem
@@thegame3417 Python code must integrate into project. So, the programe should be open source that built on this project.....