thanks for the great introduction! Some colleagues and me are planning to develop a mocap system at university and this is a great start to get into the subject.
@@thomascarstens2729 yeah sure, i guess i could help you, by the way i'm not a pro at this, it took me so long that maybe it was better to just buy an already existing moCap suit
such a great video, but can we use other camera systems? (ones that could sync obviously, maybe even security cameras or something?) I also think you should release some documentation on this too :D
Hey, quick question what algorithms are you using for tracking the markers on the camera 2d projection (the image that comes from the PS3 eye cameras) is it blob extraction and a kalman filter for tracking (similar to openmocap) or are you using something different
Which models or S/N are the good cameras? I am looking on ebay to find which are the best. Also could someone also use a Kinect with these cameras, or would the sensors conflict with each other? Is it possible to split the synchronization in half?
oh ok, ipi recorder is only to record the cameras at the same time, i mean they are already synchornized and you export one video with special extension for ipi studio. It`s in ipi studio where you calibrate the cameras ad give you the xml file so when you record the action in ipi recorder again, and the open this action(performance)video in ipi studio, you import the xml file that has the cameras calibration
I don't think that ipi recorder implements any kind of shutter synchronization, simply because it cannot be done via software. However, ipi studio doesn't need it at all (or, at least, is not so critical), because the technique used is completely different. Ipi try to reconstruct the 3D volume of the moving actor (and so is markerless) and then try to fit the skeleton inside it, analyzing the 3D voxel space. However to do it properly, it has to remove the background and this is not an easy task to accomplish with a camera like the PS3 eye, that has a lot of noise. It works by far better, if you use a device like the Microsoft Kinect that, at the end of story, is just a realtime 3D-scanner.
hi, i was wondering how you did calculate the fundamental matrix what algorithm did you use to capture marker 2d positions in image ? did u do it manually ? and how did you convert from local coordinate system of the reference camera to the world coordinate system ? any alogirthm for that ?
I set up a basic Mocap system a couple years ago with an Xbox One Kinect, some adapters, and the Brekel software, but it (obviously) had some serious limitations since it only had the one camera. I'm looking to set up a newer, better one and stumbled on your video. I also own a Vive, but not extra trackers, so I don't know if I can use those Lighthouses in a helpful way or not. Are there any options for under 1,000USD today for full body Mocap? It doesn't need to be perfect by any means, as I am a 3D guy and can clean up any animation myself, and I am only using it for personal projects anyway. Project Neuron looks interesting, but it seems their prices have gone up and I can't find much documentation about it, and what I am finding is still dated in 2015 which makes me leery.
hey. I want to make a mini mocap studio and I've a question. 1 What gives better quality Kinect ? or 6 cameras ? 2 Is it possible to record fingers too ?
Both have their pros and cons. Kinect is probably cheaper since you don't have to look for the right one... Try iPiSoft.com for a pre-made solution that's guaranteed to work (in the right conditions) and won't break your bank.
Complimenti Gran bel video! Dalla lingua di Windows ho notato che sei italiano. Io ho iniziato da poco con blender, e devo dire che mi piace molto. In ogni caso tienici aggiornati su questo progetto, veramente interessantissimo!
Great Video ! Quick question tho..You mention a particular model of PS3 eye camera, can you be specific about which model to look for when buying online? Thanks
hey i didnt understand what software you used for this, in the video you said its a software called MoCap but there is no software like that so can you tell me what you used please
This is amazing I can't wait for whats next. Are you going to make a detailed tutorial on how to setup the raspberry pi and the modification of the ps3 eye cameras? did you wrote the mocap sofware or is it something anybody can get?
For the tutorials, I do experiments in my spare time and so I haven't much time, maybe in future. I wrote the mocap software by myself, using OpenCV as supporting library. Sorry, but for now I have no plan to release the source code.
Then why the hell you make a tutorial on how to build a mocap capture system that can be used only for YOUR software, which you will never release? Cant get it...
One camera is not enough. To reconstruct the 3D position of a marker, we need at least two cameras (this is the reason why we have 2 eyes :). But due to occlusion issues, it's better to have at least 6 cameras.
Thats the one Im going to buy next month. Clearly agpxnet is has no plans of finishing, sharing or selling. why have all the hardware when the Neuron system is clean and is compatible with other software systems.
Hi, really great work you have done, i am trying to do something like this in my students grad work. I am on Boundle Adjustment step now. Can you please help with code of Boundle Adjustment method? I’l be really appreciate your!)
The PS3 camera has been studied by the community and so is known how it works and how to synchronize them. In addition there are drivers available for PC (Windows cannot recognize it automatically). I don't know about the PS4 cameras, but I choose PS3 cameras also because they are cheap. Probably could be better build a camera using arduino, performing the detection of the markers in the device itself and send only the marker position through the network (this will enable a higher framerate).
by the way you can calibrate up to 8 cameras, i think more im not sure, with ipi recorder which is a free aplication and it gives you an xml file. another thing, in your video you mention a lot that photos and info abput topics you talk about are on the internet, can you please post the links? thanks in adnvance. i suggest you to check ipi recorder
About the synchronization stuff, you can found useful info here: nuigroup.com/forums/viewthread/9628/ nuigroup.com/?ACT=28&fid=34&aid=7360_7LpLFlSc5HWnVlvjHemf www.red.pe.org.pl/articles/2014/4/40.pdf codelaboratories.com/forums/viewthread/84/
Ivan Cueto ffdgiŕhhxģjb. nxxdhhtfoygghjjkjjjjjjjuuiùuuiiuìiijlkijkkkkkopyyffyuuuiiii990opoploooopocfhhjjijkkòo opioid until just jbjjjjjhhnh have just hhhhbjjnjğhhģğjhj just uucp
Great! because i'm working on two PS3 cameras connected to one PC, and i'm using openCV for calibration and reconstruction, but i found that openCV capture function can't detect more than one PS3 camera! i think i need to use something like libusb-win32 to manage that instead of using video Capture function of openCV .
Are you sure that usb controller is a bottleneck here? I have similar issues when max fps was not more than 32fps when using 6 cameras, but after i tweak some code and turn it into multithreaded madness now it can capture with stable 60fps and also save each image as jpg, this saves a lot of disk space. I reduced data rate by capturing only raw bayer data from camera, then I run fast opencv debayer to turn it into grayscale. If you interested in this please feel free to contact me at guitarsessionmusic@gmail dot com i’ll send you my build for you to test.
It's a very interesting video, but of course a bit redundant these days. Ikinema Orion is a very decent setup for not much money. Smartsuit Pro even better.
Personally, I think it is a bit exaggerated, I do not think there are so many people interested in the subject. However, let me clarify that the system is far from be a ready product, usable, robust and generally sellable. There are many aspects that need to be improved and completed. The topic is quite complex, the video does not enter into the details of certain issues. The cameras of the PS3 (those with removable IR cut filter) are hard to find (out of production) and make the changes described (especially for synchronization) is less simple than it seems. It is necessary to utilize a better hardware, with a frame rate much higher in order to make the tracking more robust. The algorithms need to be improved (or even replaced with others, more robust). Again, the project is far from be complete and currently has more an educational value.
agpxnet ignore the last comment it was for another video, anyway I can see you're reasoning and everything but if it's more for educational value then that's even more reason to get it out there so other people can tinker with it and find hardware available now that can work with it and further build upon what you've made.
May I ask a question? There are quite affordable motion capture system around, like the Perception Neuron (just to name one) that have no space requirements (this is a big deal!) because it uses no cameras, but active markers able to compute their position and orientation in space. Why make one by yourself and don't buy one ready?
low cost my ass, everything will cost you at least 300, for 40 you can get xbox kinnect, put it on a computer record video to blender and bam done, or even with your phone if you dont have kinnect, so BAM very low cost 0 cost if you have a phone
The kinect costs bout' 70 euros for a single piece of hardware, that works, but isn't the greatest. According to this video a ps3 camera is around 14 euros, so you buy 5 of them and get better results.
There is absolutely nothing wrong with using hot glue on circuit boards. "Hot glue" is just molten EVA plastic. The temperature of hot glue is an order of magnitude lower than typical thermals used to solder parts to the board. Hot glue has been a mainstay in the DIY electronics scene since forever. Manufacturers use similar industrial adhesives, colloquially called hot snot, in manufacturing electronic devices.
thanks for the great introduction! Some colleagues and me are planning to develop a mocap system at university and this is a great start to get into the subject.
Do you have any sample footage of what it looks like once your done recording and editing some of your own footage?
yes! finally i have my own MoCap after a year working on it, thank you so much, you and "Multiple View Geometry" book, both were completely helpful
Hi, can I dm you for a few questions about your process? I'm not sure where to start, what equipment to use
@@thomascarstens2729 yeah sure, i guess i could help you, by the way i'm not a pro at this, it took me so long that maybe it was better to just buy an already existing moCap suit
@@thomascarstens2729 you guys should start a discord server
@@RaigyoEcU Could you please tell me which software did you use for it?
such a great video, but can we use other camera systems? (ones that could sync obviously, maybe even security cameras or something?) I also think you should release some documentation on this too :D
awesome work bro... thanks for bringing this to people
Hey, quick question what algorithms are you using for tracking the markers on the camera 2d projection (the image that comes from the PS3 eye cameras) is it blob extraction and a kalman filter for tracking (similar to openmocap) or are you using something different
Hey, Could you point me in the right direction for a good resource for bundle adjustment algorithm? I am having trouble getting the BA camera poses
What's the total cost?
84 euro
94 usd
I liked the music. But it made your voice a little hard to understand. ;)
you made my day :)
Do you think it could track a Quadcopter ? Its response time is quite good for such a purpose?
Which models or S/N are the good cameras? I am looking on ebay to find which are the best. Also could someone also use a Kinect with these cameras, or would the sensors conflict with each other? Is it possible to split the synchronization in half?
oh ok, ipi recorder is only to record the cameras at the same time, i mean they are already synchornized and you export one video with special extension for ipi studio. It`s in ipi studio where you calibrate the cameras ad give you the xml file so when you record the action in ipi recorder again, and the open this action(performance)video in ipi studio, you import the xml file that has the cameras calibration
I don't think that ipi recorder implements any kind of shutter synchronization, simply because it cannot be done via software. However, ipi studio doesn't need it at all (or, at least, is not so critical), because the technique used is completely different. Ipi try to reconstruct the 3D volume of the moving actor (and so is markerless) and then try to fit the skeleton inside it, analyzing the 3D voxel space. However to do it properly, it has to remove the background and this is not an easy task to accomplish with a camera like the PS3 eye, that has a lot of noise. It works by far better, if you use a device like the Microsoft Kinect that, at the end of story, is just a realtime 3D-scanner.
hi, i was wondering how you did calculate the fundamental matrix what algorithm did you use to capture marker 2d positions in image ? did u do it manually ? and how did you convert from local coordinate system of the reference camera to the world coordinate system ? any alogirthm for that ?
I set up a basic Mocap system a couple years ago with an Xbox One Kinect, some adapters, and the Brekel software, but it (obviously) had some serious limitations since it only had the one camera. I'm looking to set up a newer, better one and stumbled on your video. I also own a Vive, but not extra trackers, so I don't know if I can use those Lighthouses in a helpful way or not. Are there any options for under 1,000USD today for full body Mocap? It doesn't need to be perfect by any means, as I am a 3D guy and can clean up any animation myself, and I am only using it for personal projects anyway. Project Neuron looks interesting, but it seems their prices have gone up and I can't find much documentation about it, and what I am finding is still dated in 2015 which makes me leery.
Enflux www.getenflux.com/pages/product
Can I use the cameras without ir filter or I have to choose with filter and remove manuy
Does this system work with VR and then play steam games?
Can you make a tutorial how to build the clock?
hey. I want to make a mini mocap studio and I've a question. 1 What gives better quality Kinect ? or 6 cameras ? 2 Is it possible to record fingers too ?
Both have their pros and cons. Kinect is probably cheaper since you don't have to look for the right one... Try iPiSoft.com for a pre-made solution that's guaranteed to work (in the right conditions) and won't break your bank.
Thanks, this was very helpful and professional, by the way perhaps You may know of there is any hybrid mocap? A mix between cameras and IMUs
Complimenti Gran bel video! Dalla lingua di Windows ho notato che sei italiano. Io ho iniziato da poco con blender, e devo dire che mi piace molto. In ogni caso tienici aggiornati su questo progetto, veramente interessantissimo!
Excelente Trabajo!
Can you post the sync program?
What is the name of the marker material?
Can you provide the source code you developed for this project? Thank you.
Which bundle adjustment library did you use?
I love this video, and I DIY my own Motion Capture suit!
Great Video ! Quick question tho..You mention a particular model of PS3 eye camera, can you be specific about which model to look for when buying online? Thanks
very amazing!!! thanks for this video!
Is the development of this project is finished?
Thanks. Not yet, I do my experiments in my spare time.
hey i didnt understand what software you used for this, in the video you said its a software called MoCap but there is no software like that so can you tell me what you used please
It's a software made by me.
Can i use other software?
@@agpxnet Can you share this program of yours? Or what are similar programs, if any?
Is it possible if i use clay as the actor?
This is amazing I can't wait for whats next. Are you going to make a detailed tutorial on how to setup the raspberry pi and the modification of the ps3 eye cameras? did you wrote the mocap sofware or is it something anybody can get?
For the tutorials, I do experiments in my spare time and so I haven't much time, maybe in future. I wrote the mocap software by myself, using OpenCV as supporting library. Sorry, but for now I have no plan to release the source code.
Then why the hell you make a tutorial on how to build a mocap capture system that can be used only for YOUR software, which you will never release? Cant get it...
Hmmm
Can I purchase your system? Can talk detail?
It looks like you have made enough progress to start a nice open source project. I think many would help you finish. I would.
I can not find software online
Can you sell this software?
man man i made full setup that you say,but which software i use
do we need only one camera for it
One camera is not enough. To reconstruct the 3D position of a marker, we need at least two cameras (this is the reason why we have 2 eyes :). But due to occlusion issues, it's better to have at least 6 cameras.
agpxnet but what about the new perception neuron set which is for about 1600$ is that a good one links below please give a advice neuronmocap.com/
Thats the one Im going to buy next month. Clearly agpxnet is has no plans of finishing, sharing or selling. why have all the hardware when the Neuron system is clean and is compatible with other software systems.
Yhea!! Great job stick it to the man.
Nice work! Can I contract you for some detail?
What kind of software are you use? Amazing video!
Thanks, it's a software written by me.
Mocap software?! Well... It's really cool.)
agpxnet make a shortened link to a download
Nice work.
Very informative video!
Hi, really great work you have done, i am trying to do something like this in my students grad work. I am on Boundle Adjustment step now. Can you please help with code of Boundle Adjustment method? I’l be really appreciate your!)
can u use cameras for ps4?
The PS3 camera has been studied by the community and so is known how it works and how to synchronize them. In addition there are drivers available for PC (Windows cannot recognize it automatically). I don't know about the PS4 cameras, but I choose PS3 cameras also because they are cheap. Probably could be better build a camera using arduino, performing the detection of the markers in the device itself and send only the marker position through the network (this will enable a higher framerate).
Wow, that looks pretty complicated.
hey can you make this video one more time so we understand it a little bit more.
by the way you can calibrate up to 8 cameras, i think more im not sure, with ipi recorder which is a free aplication and it gives you an xml file. another thing, in your video you mention a lot that photos and info abput topics you talk about are on the internet, can you please post the links? thanks in adnvance. i suggest you to check ipi recorder
About the synchronization stuff, you can found useful info here:
nuigroup.com/forums/viewthread/9628/
nuigroup.com/?ACT=28&fid=34&aid=7360_7LpLFlSc5HWnVlvjHemf
www.red.pe.org.pl/articles/2014/4/40.pdf
codelaboratories.com/forums/viewthread/84/
Ivan Cueto ffdgiŕhhxģjb. nxxdhhtfoygghjjkjjjjjjjuuiùuuiiuìiijlkijkkkkkopyyffyuuuiiii990opoploooopocfhhjjijkkòo
opioid
until
just
jbjjjjjhhnh
have
just
hhhhbjjnjğhhģğjhj
just
uucp
Quite impressive, well done.
How many PCs do you use?
A single notebook (core I7), but unfortunately the USB controller is unable to record at 60 FPS, so I have reduced the FPS to 30.
Great! because i'm working on two PS3 cameras connected to one PC, and i'm using openCV for calibration and reconstruction, but i found that openCV capture function can't detect more than one PS3 camera!
i think i need to use something like libusb-win32 to manage that instead of using video Capture function of openCV .
Are you sure that usb controller is a bottleneck here? I have similar issues when max fps was not more than 32fps when using 6 cameras, but after i tweak some code and turn it into multithreaded madness now it can capture with stable 60fps and also save each image as jpg, this saves a lot of disk space. I reduced data rate by capturing only raw bayer data from camera, then I run fast opencv debayer to turn it into grayscale. If you interested in this please feel free to contact me at guitarsessionmusic@gmail dot com i’ll send you my build for you to test.
It's a very interesting video, but of course a bit redundant these days. Ikinema Orion is a very decent setup for not much money. Smartsuit Pro even better.
i did simple motion capture program with two webcams , i have it on youtube
@John Aidan Logarta ua-cam.com/video/GMGWcvJ9f8E/v-deo.html
awesome!
dislike for not releasing the software, source or at least sell it dude you're sitting on a goldmine.
Personally, I think it is a bit exaggerated, I do not think there are so many people interested in the subject. However, let me clarify that the system is far from be a ready product, usable, robust and generally sellable. There are many aspects that need to be improved and completed. The topic is quite complex, the video does not enter into the details of certain issues. The cameras of the PS3 (those with removable IR cut filter) are hard to find (out of production) and make the changes described (especially for synchronization) is less simple than it seems. It is necessary to utilize a better hardware, with a frame rate much higher in order to make the tracking more robust. The algorithms need to be improved (or even replaced with others, more robust). Again, the project is far from be complete and currently has more an educational value.
agpxnet me personally I thought Star Wars Rogue One was an excellent movie.
agpxnet ignore the last comment it was for another video, anyway I can see you're reasoning and everything but if it's more for educational value then that's even more reason to get it out there so other people can tinker with it and find hardware available now that can work with it and further build upon what you've made.
May I ask a question? There are quite affordable motion capture system around, like the Perception Neuron (just to name one) that have no space requirements (this is a big deal!) because it uses no cameras, but active markers able to compute their position and orientation in space. Why make one by yourself and don't buy one ready?
agpxnet Sorry I didn't make it clear but I was asking because its something nice to tinker with and im sure other people would too.
excellent
Awesome!
Use camera kinect for windows is better. Like kinect xbox 360 or Xbox one.
thank you
I was able to get 4 pseye cameras for 20 bucks, they were selling them for 5 dollars each second-handed
NO WAY YOUR IDEA I CANT GET ..SORRY
low cost my ass, everything will cost you at least 300, for 40 you can get xbox kinnect, put it on a computer record video to blender and bam done, or even with your phone if you dont have kinnect, so BAM very low cost 0 cost if you have a phone
or just use a kinect
Kinect has bad quality
The kinect costs bout' 70 euros for a single piece of hardware, that works, but isn't the greatest. According to this video a ps3 camera is around 14 euros, so you buy 5 of them and get better results.
never ever use hot glue on circuit boards
There is absolutely nothing wrong with using hot glue on circuit boards. "Hot glue" is just molten EVA plastic. The temperature of hot glue is an order of magnitude lower than typical thermals used to solder parts to the board. Hot glue has been a mainstay in the DIY electronics scene since forever. Manufacturers use similar industrial adhesives, colloquially called hot snot, in manufacturing electronic devices.
And this is the result: ua-cam.com/video/cdPdoueaC-0/v-deo.html
all we can do with one device: Kinect its cheap and doesn't required any paid shit!!!