This is awesome, I got it working on my machine, but I need to get a bit more room so it can pick up my entire body properly like it does with you. Thanks a lot for the plugin! Was totally worth the effort of getting the drivers installed properly.
@Pirokoro You can't, the special thing with Kinect's camera is that it registers DEPTH. The camera can detect when something is closer or further away by bouncing infra-red beams off the area. Your regular webcam can only see 2D.
I am truly surprised by the endless posibilities with Kinect! No wonder it's so bloody expensive! Excellent demonstration as well! You've been very thorough with the possibilities on MMD!
@jamminjoemac Kinect does not have enough detectors to do both body detection and facial detection. It can do one or the other, but not both at the same time. That's why the Kinect video conference system doesn't allow you to move your avatar's arms despite being able to detect your facial expressions. It would have been too expensive to buy otherwise.
I just found out my mom is getting a Kinect for Christmas. OH YES OHHHHHHHH YES Oh my Gosh, you got recording too? :DDDDDDD CAN'T WAIT FOR THE NEW VERSION CAN'T. WAIT.
The kinect must be a godsend for low-budget developers wanting to do mocap for their games. Once there is enough good programs made for it I think it will become a staple for those kinds of devs.
@ieda18 The OpenNI he mentioned is a software that lets you use the Kinect on a PC... I think. I don't know if it still has to go through the XBox first.
@XxtsunamidevilxX Sadly no, The kinect camera works by using two infa-red cameras. Unless your webcam has two Infa-red cameras and uses the same drivers then sadly no your webcam will not work.
OMG! submit this to MS and make a game!!!!! it was hilarious how you couldn't dance with the others. but there should be collision detection like Kinect Sports (where if you reach out and try to molest the other runners, they'll scream at you to back away). but you made the character so responsive. great job!
@philnolan3d well both I guess. I was primarily thinking mocap for games, but given how cheap this is to get and use I could definately see this being used in machinima and independent 3D animation stuff too.
@moopMASTER2000Films : ) that's what I'm trying to do but the kinect doesn't detect the fast movement though. The shuffle I'm trying to do is of Pae & Sarah.
Thank you, Higuchi-san. (Your name is Higuchi, right? Or did I miss something?) This will help us MMD users to make more realistic and natural animations. You rock. :-)
@Inoryuki Yes, you can record motion data with Kinect, using MMD. I bought one for that purpose. I don't even own an X-Box. I posted a link; in the Bulletin section on my channel; for a tutorial if you're using Windows XP, or Vista. If you're using Windows 7, you can just Google, "Kinect SDK".
@RenediAmasawa Even Microsoft haven't managed to fit in both body detection and finger detection. I think we will have to wait for the next generation hardware for that. Thankfully it isn't at all impossible; we just need more infra red dots and better resolution detectors. And maybe have two cameras working in tandem.
@higuchuu you need to post all the install requirements in a list and process steps to follow. Many, many non programmers are going to try do this; they need hand holding.
@zimtower User Google translate to send personal message directly to Higuchuu sensi; You can also use Google translate to search the Japanese Nico Nico Duga web site; that is where most of this stuff is. At the Japanese CEATEC 2010 some important open source modules for MMD were also posted by one of the universities. Regarding real time rendering, lip sync, and agent chat capabilities. FYI: Higuchuu is the origional creator of MMD.
i used to think kinect was the worst invention ever. i dont own an xbox and dont plan to. but THIS!!!! this is what should be out commercially all over the world. to bad we had to use the equivalent of what we've had the technology to do for a few years now. thank you so freakin much to whomever programmed this work of art. now, how the hell can i afford a kinect?
Oh wow, this'll create some awesome MADs and music videos... Also, what if they used this to create something sort of like a video chat, except with the users' anime avatars mimicing their movements? I think that would really work!
@OokamiiKiba There's more to it than just connecting the Kinect to the computer to get to the motion data making part (Technical-ish computer-y things that my dad and I haven't figured out yet... XD) but you need a power cord thingamajig, because there's one that's online that has a USB extension, so that you can plug it into your computer! :D
Ok. I figured out my problem. It was a calibration issue. You have to give the kinect time to calibrate your body, and it can take up to a minute. Once stepped back and made sure there weren't other objects behind me, I was good to go. So, step back until you see your full body and stay still. You'll know when its calibrated when the image starts to move.
Thank you very much for sharing it.. I can't believe I am going to try it on my own PC right now :D I have one question : Do I have to remove the already installed OpenNI driver first before I install DxOpenNi ? Or will they work OK together ? And thanks alot for this awesome tool.
the difference between the two is that the multi model version lets you use/work with 2 or more models at the same time, while the 64bit version only lets you work with one model at a time...but I might be wrong so it's better to just check somewhere on google.. ^-^;
@fielrichard101 Infact, it said 24-26 on the last animation, wich is terrible, it said 206 on the single character, wich again, is nothing for such a simple thing.
@Deathblood2294 Kinect has an input lag of 150 ms (as measured in Kinect Sports). Killzone 2 has an input lag of on average 150 ms (as measured by Eurogamer). KZ2's lag can go up as high as 190 ms. Most shooters have a lag of 60 ms to 100 ms (CoD, Halo, etc.).
This is so fantastic - thank you! I have got all installed and working BUT I have a huge lag in the movement and the MMD freezes and stutters. I am running this is Windows 7 on a monster of a machine so don't know why this would be happening?
this is freaking awesome. I work atm with Kinect too for School Diplom work... it's kinda difficult... specially cuz there is barely and programmexamples in java >.< Which language did you use?
you know what you should startsing idk if its possable, you might have to work with mmd's programing as well. make it so multiple people can control multiple models. so that there can be two or more people acting out a scene or dance.
Awesome vid man just got one question we'er you using a xbox 360 kinect or the pc kinect? Wondering because I was th inking of buying a pc kinect 250$ but its for miku so I would.
@xRukiaxIchigofanx No I've never tried it before you need to download drivers for kinect so it could work on your pc. Then i believe you will need to download some sort of software for the kinect so you can use it with MMD. Try to google these stuff up like," PC kinect software or drivers." Something alone those lines.
can anyone tell me the difference between mmd 64bit vs the multi model edition? i currently have multi model but was wondering if i should get the outher one.
@Ampereox its a driver you can download it here openni.org/?q=node/2 you must choose which one best for your computer. plus you need a kinect device too.. even i am not buy it yet. coz its too expensive!! hope this will help you. (my english is really2 bad sorry)
Omg... I would so love a Kinect camera right now. This is the best thing EVER!
Thank you so much for doing this. You have no idea how much the existence of this means to me.
This is awesome, I got it working on my machine, but I need to get a bit more room so it can pick up my entire body properly like it does with you. Thanks a lot for the plugin! Was totally worth the effort of getting the drivers installed properly.
@Pirokoro
You can't, the special thing with Kinect's camera is that it registers DEPTH. The camera can detect when something is closer or further away by bouncing infra-red beams off the area. Your regular webcam can only see 2D.
I am truly surprised by the endless posibilities with Kinect! No wonder it's so bloody expensive! Excellent demonstration as well! You've been very thorough with the possibilities on MMD!
@jamminjoemac
Kinect does not have enough detectors to do both body detection and facial detection. It can do one or the other, but not both at the same time. That's why the Kinect video conference system doesn't allow you to move your avatar's arms despite being able to detect your facial expressions. It would have been too expensive to buy otherwise.
@higuchuu Woo it worked! I assumed we only need OpenNi, but a working NITE was also needed. Thanks a lot for all your work.
I just found out my mom is getting a Kinect for Christmas. OH YES
OHHHHHHHH YES
Oh my Gosh, you got recording too?
:DDDDDDD
CAN'T WAIT FOR THE NEW VERSION
CAN'T. WAIT.
New era for MMD had arrived, very awesome
きしめんの動きが素敵すぎますｗ
PC操作しに良く動作とか、息が上がってるのとか、かわいい！
I just got a Kinect, and just learned of this. I'm excited, I *have* to try this!!!
The kinect must be a godsend for low-budget developers wanting to do mocap for their games. Once there is enough good programs made for it I think it will become a staple for those kinds of devs.
@ieda18 The OpenNI he mentioned is a software that lets you use the Kinect on a PC... I think.
I don't know if it still has to go through the XBox first.
@XxtsunamidevilxX Sadly no, The kinect camera works by using two infa-red cameras. Unless your webcam has two Infa-red cameras and uses the same drivers then sadly no your webcam will not work.
@CodeGeassLelouchrocs 7.25 is the one that reads kinect plugin
MMD just keeps getting better
This seems so silly but so fun at the same time XD the cool thing is that the movements will be a bit more realistic now.
@PointsofData
Thank you for answering! : D
Yeah, I thought it might be but I just wanted to make sure, so thanks again ^_^
dude you just further broke the barrier of interaction with the virtual world!!!!
Sorry, I'm still laughing like a girl at the group dance part, but this has got to be the greatest Kinect project yet!
I don't know why, but when Miku started dancing with the rest of them (2:18-2:40) I was in tears.
i hope i'm not the only one ROFLING over hatsune miku dancing like this. also, i like the way you move.
OMG! submit this to MS and make a game!!!!! it was hilarious how you couldn't dance with the others. but there should be collision detection like Kinect Sports (where if you reach out and try to molest the other runners, they'll scream at you to back away). but you made the character so responsive. great job!
@philnolan3d well both I guess. I was primarily thinking mocap for games, but given how cheap this is to get and use I could definately see this being used in machinima and independent 3D animation stuff too.
Now someone must make "MikuMiku Brawl" and the circle will be complete.... great Kinect Hack pal... props to you
@moopMASTER2000Films
: ) that's what I'm trying to do but the kinect doesn't detect the fast movement though. The shuffle I'm trying to do is of Pae & Sarah.
Thank you, Higuchi-san. (Your name is Higuchi, right? Or did I miss something?) This will help us MMD users to make more realistic and natural animations. You rock. :-)
I do believe this will raise the Kinect's sales.
@Inoryuki Yes, you can record motion data with Kinect, using MMD. I bought one for that purpose. I don't even own an X-Box. I posted a link; in the Bulletin section on my channel; for a tutorial if you're using Windows XP, or Vista. If you're using Windows 7, you can just Google, "Kinect SDK".
This looks like it could be a great game! Good job.
@AbsentWhite It's mainly for the xbox, but you don't HAVE to have the xbox to get it. They're about $112 at bestbuy.
@RenediAmasawa
Even Microsoft haven't managed to fit in both body detection and finger detection. I think we will have to wait for the next generation hardware for that. Thankfully it isn't at all impossible; we just need more infra red dots and better resolution detectors. And maybe have two cameras working in tandem.
@higuchuu you need to post all the install requirements in a list and process steps to follow. Many, many non programmers are going to try do this; they need hand holding.
@zimtower User Google translate to send personal message directly to Higuchuu sensi; You can also use Google translate to search the Japanese Nico Nico Duga web site; that is where most of this stuff is. At the Japanese CEATEC 2010 some important open source modules for MMD were also posted by one of the universities. Regarding real time rendering, lip sync, and agent chat capabilities. FYI: Higuchuu is the origional creator of MMD.
i used to think kinect was the worst invention ever. i dont own an xbox and dont plan to. but THIS!!!! this is what should be out commercially all over the world. to bad we had to use the equivalent of what we've had the technology to do for a few years now. thank you so freakin much to whomever programmed this work of art. now, how the hell can i afford a kinect?
@TheXAnathema i'm pretty sure that was just not included in the video.
Oh wow, this'll create some awesome MADs and music videos...
Also, what if they used this to create something sort of like a video chat, except with the users' anime avatars mimicing their movements? I think that would really work!
@Deathblood2294 Kinect has no lag, some games are not well done. Your Shape for example has no lag at all.
@OokamiiKiba There's more to it than just connecting the Kinect to the computer to get to the motion data making part (Technical-ish computer-y things that my dad and I haven't figured out yet... XD) but you need a power cord thingamajig, because there's one that's online that has a USB extension, so that you can plug it into your computer! :D
これまでもKinect使った色々な物が発表されてるけど、誰でもが手に入れられる完成したアプリケーションということと、海外でも人気の初音ミクをアバターにして操れるという点で抜きん出てますね。
the future of creating motion data is going to become easyer
(as long if you know the dance xD)
@ProAbortionFetus This is what I was thinking. A new level of mods can come out. Maybe with two kinects it would look better.
@jenden44 From what I heard, yes, it's fine to have just a Kinnect and also something to hook it up to your computer. :3
Ok. I figured out my problem. It was a calibration issue. You have to give the kinect time to calibrate your body, and it can take up to a minute. Once stepped back and made sure there weren't other objects behind me, I was good to go. So, step back until you see your full body and stay still. You'll know when its calibrated when the image starts to move.
Congrats! this is awsome!
can't wait for having my kinect!
Very fast and smooth tracking. Nice software indeed.
Yeah, we go a Kinect for Christmas and I'm bored already. This can REALLY help me and my brother with his 3D CG things...
Thank you very much for sharing it.. I can't believe I am going to try it on my own PC right now :D
I have one question : Do I have to remove the already installed OpenNI driver first before I install DxOpenNi ? Or will they work OK together ?
And thanks alot for this awesome tool.
@AbsentWhite
No. Its just a camera that connects to it.
This is so many levels of awesome!
the difference between the two is that the multi model version lets you use/work with 2 or more models at the same time, while the 64bit version only lets you work with one model at a time...but I might be wrong so it's better to just check somewhere on google.. ^-^;
I've played kinect games, and it looks like miku miku dance is more accurate than kinect adventures.
yep
The whole beginning I was like "Miku, how unlady like to be kicking up your skirt like that" xD
@fielrichard101 Infact, it said 24-26 on the last animation, wich is terrible, it said 206 on the single character, wich again, is nothing for such a simple thing.
Real-time motion capture available to the public? The future looks badass.
@Deathblood2294 Kinect has an input lag of 150 ms (as measured in Kinect Sports). Killzone 2 has an input lag of on average 150 ms (as measured by Eurogamer). KZ2's lag can go up as high as 190 ms. Most shooters have a lag of 60 ms to 100 ms (CoD, Halo, etc.).
I see the future of gaming
This is amazing! *u* すげえ~
This is so fantastic - thank you! I have got all installed and working BUT I have a huge lag in the movement and the MMD freezes and stutters. I am running this is Windows 7 on a monster of a machine so don't know why this would be happening?
This is cool, but I think it would be cooler if this was webcam compatible, so you don't need Kinect.
Miles5X5 now it works with a webcam www.kickstarter.com/projects/571292281/mashme-become-an-animated-character
@OokamiiKiba you need to buy the adapter plug for about 40 dollars
seems like it needs to skip some frames of animation to ease the jittering issues
The mapping of the body is the best by far.
Opens up a whole new door for indie devs
do you know why miku's leaning like that at the end? It's because someone gotta have to get the kinect remote to stop the dance :3
Its interesting how you can see the maleness of the actions in a female avatar.
this is freaking awesome. I work atm with Kinect too for School Diplom work... it's kinda difficult... specially cuz there is barely and programmexamples in java >.<
Which language did you use?
これはすごい！
MikuMikuDanceの最新版はKinectによるモーションキャプチャーが可能に！
I love ! I want a Kinect ! I'm a french user of MMD ! Yeah ! (o)//
GOOD WORK !!
This is great! I downloaded the files and it is precise and great.
She moves very fluidly.
How do you get the model to spin around? I've literally been looking for a solution for almost a week and i still don't know how people do it...
use the centre bone
you know what you should startsing idk if its possable, you might have to work with mmd's programing as well. make it so multiple people can control multiple models. so that there can be two or more people acting out a scene or dance.
this is amazing how responsive it is!
Who needs Mafia 2's face tech, when YOU, can be the anime character.
I think this can deliver an awesome dancing game for Kinect on 360 or PC
Awesome vid man just got one question we'er you using a xbox 360 kinect or the pc kinect? Wondering because I was th inking of buying a pc kinect 250$ but its for miku so I would.
Noticed that there does not appear to be independent head and neck motion. Are the facial expressions from prior motion data?
i like how she went straight through that person next to her
Dude... my prom dance routine, how did you know?! lolz
apparently, you have to right click and hit "run as administrator." It doesn;t work for me, though.
@PHValtiel you just need to calibrate it by standing still and away with nothing in the background. Also, are you running 64 bit windows or 32 bit?
so this is th elejendery program that what you dance she dances too..... too bad you can only BUY it
@jenden44 Sure. :3 Glad to help!
Can you pose models using this or is it just for motion data? I honestly believe that this is an innovative use of technology.
Now all we need is someone to make
Just Dance Miku
I'd buy it for sure, And play it for hrs and hrs
I'm getting a Kinect in like two days, just for this. Only reason I bought it ;3
@xRukiaxIchigofanx No I've never tried it before you need to download drivers for kinect so it could work on your pc. Then i believe you will need to download some sort of software for the kinect so you can use it with MMD. Try to google these stuff up like," PC kinect software or drivers." Something alone those lines.
@philnolan3d Kinect and MikuMikuDance is just the beginning anyway. Give it a few years and we'll kiss current motion capture technology goodbye.
i almost pissed myself laughing when she started dancing trying to keep up
This is unintentionally hilarious.
Okay. I now need
1. MMD (I've wanted it all my life DX)
2. Kinect.
@fwaggylover Nope. The Kinect is a 3 dimensional camera that recognizes ligaments from your body.
Most webcams can't even track a human face :P
Wow, no ping-pong balls needed or anything!
can anyone tell me the difference between mmd 64bit vs the multi model edition?
i currently have multi model but was wondering if i should get the outher one.
@Ampereox its a driver you can download it here
openni.org/?q=node/2 you must choose which one best for your computer. plus you need a kinect device too.. even i am not buy it yet. coz its too expensive!! hope this will help you. (my english is really2 bad sorry)
@GJSpot Okay thanks ^^ This seams to be really confusing to me
@Scyeth You expected a hacked system to take out everything that bugs out?
I like it when she goes over to the invisible computer XD