I also had to go down this rabbit hole ot figure this out a few years ago. I'm happy you've created an easy-to-follow tutorial here. I hope to see more cartoony characters in the future with this.
i was making a mask for my avatar and couldnt figure out how to make both meshes move together without combining them in blender. this video helped a lot , very informative and easy to understand. Thank you :3
the visemes control is on the fx animator, where you would put toggles and stuff-- unless vrchat updated it and screwed everything over again. but i dont think i specified that in this video. thank you!
@@_colorbars_ I used the same-ish method as you did at first, by my toon shader was causing some weird shading on the mouth, so as a programmer and a game developer that uses unity almost daily, I knew there was a better way to do 2D face cards using scrolling UVs. The method I found was to to put all of the mouths on the same texture as the base texture and that meant I also needed to move the UV cords for the rest down and slightly shrunk. Then in a shader I would have a "VisemeIndex" int that was controlled by the animations. The shader would then move the UV cords for the mouth to the correct mouth. But this way, there's no floating mouth in front, its one material, skinned mesh renderer, one texture, will always work with ANY shader(you'd just need to add the logic), and best of all will work on the quest because there's no transparency. A link to my GitHub to see how it works if you are curious: github.com/TheRealCanad1an/Better2DMouths
I love the hard sketch mouth. I'd be interested to see how you plan to improve it but I think the style looks great! Ty for the tutorial I've been trying to figure this out for a while.
thank you so much for making this, I pretty much lost hope in finding any kind of documentation on 2d visemes before finding this, and i combed through almost every documentation and tutorial on the web on this its worth noting though as someone who had an identical method with the invisible plane: I strongly recommend keeping the mouth plane aligned with the face mesh as best as possible if you wanna also upload this to quest, because the quest version does not allow invisible textures. It really sucks having a cool mouth texture only for it to be a weird out of place square for half the userbase (assuming anyone else cares about their things being cross platform) one question though, say i wanted to have the mouth textures transparent on pc and the skin color on quest. would it be better to make an entirely new set of animations with the alt textures, or swap the textures within with the existing animations? or would this be possible at all with a single avatar?
You'd have to make new animations for the new quest materials unfortunately. Either that or replace the textures in the unity project for the transparent images with the skin color ones; It'll be easy if you name the skin color textures the same as the transparent ones, deleted the transparent ones from the project, and inserted the skin color ones. It replaces the texture in the material and you dont need to change the animations. Easy way to convert the materials to quest shader is by using VRCquesttools, though it'll only convert to toon lit. Fast n easy tho. Glad it helped!
This is a great video and have watched many times. I'm still stuck on just one more part! I haven't been able to find much documentation on it but its just the whole process of importing images, turning an image into a separate material, linking them to their blendshape and all that. I have all the materials! Named blendshapes, textures, and a seperate plane, just lost on the what exactly to do with my pngs and making them into a material just for the plane.
hiya! im glad you like the video! There are two steps to doing this, 1. In blender the plane Must have its own texture aside from the rest of the model. I show you that here 8:30 In unity, when you apply a texture to a model, it automatically creates a corresponding material. However, you can also right click and press Create > Material, in the "albedo" slot you can click and drag your png. Your material inspector will look a little different from mine since i had a specific shader, but standard allows for transparency. All you have to do is go to the top of the shader options and click the drop down for "rendering mode" and select Cutout.
I have done blinking for a similar character. It's a bit more complicated and i by no means have perfected it, but i do have a method! i'll see if i can get time this year to make a video on it
So how would I make the lip sync appear on the avatar when I speak? I noticed that you have the lip sync set to "viseme blend shape" in your avatar descriptor, but when I do that none of the visemes appear in any of the options below it.
Set it to "viseme paramater". Sorry for the confusion and the delay in response! As long as you've set up the visemes in the avatar's fx layer like they are in the video, it should work.
This is a life saver!!! I tried making blend shapes with individual face plates for mine and they ended up clipping through the model. Do you think these visemes can be switched out for expressions?
do you mean using expressions instead of visemes or pausing visemes for expressions? for the first one, yes, for the second one, im not so sure how. I'm no programmer but I assume there's a way to stop the visemes while you have a gesture on. If you wanna take the lazy route, see if you can make a blendshape that hides the mouth and exchanges it for one with the expression.
Thank you so much! I've been looking for a tutorial like this. One question, do you know how to make it random? like, if I wanted to change my eye image to blinking image randomly?
i've asked around and picking from a random set of materials isn't really a thing vrchat can do? I'd have to do more research. It's hard to get answers to those kinds of questions. I DO however have an avatar with a 2d blinking animation, and thats just a sprite sheet of different images of the eyes with uv offsets with a LONG looping animation (2 minutes) to give it the illusion that the character is looking around and blinking at random intervals. I'll try to get a video of that together; hard to sit down and focus long enough to record
Hiya! Love the tutorial so far!! Huge help to me as im not to big of a fan of modelling 3d mouths with blendshapes, so this was a huge help to go for 2d mouths. Although i do have a problem i cant seem to fix yet, testing the visemes, the mouth does not return to the base viseme, it stays stuck to the last viseme played, has that happened with you, if so is there a fix? [Edited] OOP I got it to work!!
isn't it easier and optimized to create multiple planes with all your visemes textures on them and use classic blendshapes instead of all this, it's automatically recognise in unity if i remember right (and if it's named correctly bafore exporting) Because for quest users you won't be able to use as much materiels as you do here
not necessarily- you would still need to assign separate materials to all those planes, which in turn, isn't optimizing it any further than it already is. It's just adding more vertices. it'd be more optimized* if you used a single sprite sheet, made as many planes as you needed, and UV offset the texture on each plane in your animations so that they all have the visemes on the sprite sheet. You'd still need to use viseme parameters for this. *optimized here meaning uploadable for quest. the most optimized version of this method would be to use a single plane and a single sprite sheet, animating with uv offsets.
@@_colorbars_ no need to use multiple materials, one with all the expressions (expression sheet texture) and juste uv on the 11 or 13 different visemes textures, a unique 1024 or 2048 png and so blendshape for all visemes with the corresponding planes going in and out of your model instantly, that was indeed what i meant
@@_colorbars_ do you know how to do the same method with uv offsets in unity? it's what i'm currently trying to do! i also have another set of mouths specifically for happy expressions, but i feel like that would be more complicated and require actual coding lol
@@bashartz Yeah! just animate your offsets. If you press record and then change your offset in a shader, that should do the trick. Something like poyomi will require you to enable a part of your shader to be animated. (finding out how to tile a sheet would be the trickiest part in this, im still figuring that out.) For having another set of mouths, it's a little complicated to explain in text, but what you could do is have two different sheets- one for neutral, one for happy. Have two on different planes that you can switch out for that expression. I'd recommend toggles in your radial menu for this, but it'd be interesting to see how it could work with just the finger gestures I suppose!
You'd need to make a sprite sheet and animate offsets for each viseme. I can't really supply a how-to video at the moment, but its not too difficult- there are videos showing how to animate offsets in unity.
sorry for replying so late- there are plenty of tutorials on youtube that start from the very bottom! I don't have any to reccomend at the moment, as vrchat has changed quite a bit since I followed any, but sippbox does a good job.
You'd have to share more details in order for me to help you out; I've never experienced this issue before. Check to see if your parameters match up with the way I've configured them
im trying to upload but its saying the image is too big for the actions and moods you probably went over this in the video but there are no time stamps
if the set up is similar vrchat's sdk, then I don't see why not. I don't know anything about vseeface/vrm files- the whole reason this is possible is because vrchat has "Viseme parameters", which allow you to assign any animation to the lip sync.
@@_colorbars_ ah ok thank you I do not think it is possible I will stick to placing my emotions inside my characters head and then figuring out different things
sil is Silence. As explained in the video, each viseme has a number assigned to it, Sil being 0. Since the set up for viseme parameters is a Int that returns to 0 when not detecting input (IE, you aren't talking or are muted), Sil is set at 0
what about if the mouth images were all in one texture map then you make multipleplanes on the face to assign each mouth image, then make shapekeys showing one at a time ...think that could work? the mouth animations look so alive and cool.
yeah that could work! would add more polygons, but it wouldn't be that bad. You'd have to make the animation transition/exit times 0 though, so each plane shows up instantly.
![2D Visemes & Blinking in VR Chat Guide [Quest/Fallback Compatible]](http://i.ytimg.com/vi/l--1c8QGpVw/mqdefault.jpg)
Very informative, been making avatars, for years and didn't know how 2d visemes worked I'll be sharing this around
I also had to go down this rabbit hole ot figure this out a few years ago. I'm happy you've created an easy-to-follow tutorial here. I hope to see more cartoony characters in the future with this.
this is indeed a vdeo, thank you milk
i was making a mask for my avatar and couldnt figure out how to make both meshes move together without combining them in blender. this video helped a lot , very informative and easy to understand. Thank you :3
For the newest version I had to put the animator controller under FX in the playable Layers tab. Hope that helps someone.
the visemes control is on the fx animator, where you would put toggles and stuff-- unless vrchat updated it and screwed everything over again. but i dont think i specified that in this video. thank you!
@@_colorbars_ I used the same-ish method as you did at first, by my toon shader was causing some weird shading on the mouth, so as a programmer and a game developer that uses unity almost daily, I knew there was a better way to do 2D face cards using scrolling UVs. The method I found was to to put all of the mouths on the same texture as the base texture and that meant I also needed to move the UV cords for the rest down and slightly shrunk. Then in a shader I would have a "VisemeIndex" int that was controlled by the animations. The shader would then move the UV cords for the mouth to the correct mouth. But this way, there's no floating mouth in front, its one material, skinned mesh renderer, one texture, will always work with ANY shader(you'd just need to add the logic), and best of all will work on the quest because there's no transparency. A link to my GitHub to see how it works if you are curious: github.com/TheRealCanad1an/Better2DMouths
Any way to add mmd compatibility?
I love the hard sketch mouth. I'd be interested to see how you plan to improve it but I think the style looks great! Ty for the tutorial I've been trying to figure this out for a while.
This will be awesome sauce for making my low-poly models. Thank you so much!!
I actually have a question, though. Can you make your silent position a drawing as well?
thank you so much for making this, I pretty much lost hope in finding any kind of documentation on 2d visemes before finding this, and i combed through almost every documentation and tutorial on the web on this
its worth noting though as someone who had an identical method with the invisible plane: I strongly recommend keeping the mouth plane aligned with the face mesh as best as possible if you wanna also upload this to quest, because the quest version does not allow invisible textures. It really sucks having a cool mouth texture only for it to be a weird out of place square for half the userbase (assuming anyone else cares about their things being cross platform)
one question though, say i wanted to have the mouth textures transparent on pc and the skin color on quest. would it be better to make an entirely new set of animations with the alt textures, or swap the textures within with the existing animations? or would this be possible at all with a single avatar?
You'd have to make new animations for the new quest materials unfortunately. Either that or replace the textures in the unity project for the transparent images with the skin color ones; It'll be easy if you name the skin color textures the same as the transparent ones, deleted the transparent ones from the project, and inserted the skin color ones. It replaces the texture in the material and you dont need to change the animations.
Easy way to convert the materials to quest shader is by using VRCquesttools, though it'll only convert to toon lit. Fast n easy tho.
Glad it helped!
absolute life saver, only tutorial for this i could find
This was so helpful!!! Thank you I've been trying to find something like this since forever!
bro deserves more subscribers,
this was SOOO helpful, thank you so much!
this is awesome tysm, i've been scrounging around at 4 am looking for a solution like this
An integer is whole numbers as in 1 2 3 ect. a float is also numbers in between as in 1.02 5.66343 65465.234 ect
cool avatar bro and good vid
thank you issacvoregames
Thank you!
dude what the hell, you're the guy that did stuff for Joel lol. weird to see you here. much love
Great tutorial! Worked pretty good for my avatar. :)
the standart material has an uv offset field which im p sure can be animated 🤔
thank you now i can animate my little skrunklies
love that for them
This is a great video and have watched many times. I'm still stuck on just one more part! I haven't been able to find much documentation on it but its just the whole process of importing images, turning an image into a separate material, linking them to their blendshape and all that. I have all the materials! Named blendshapes, textures, and a seperate plane, just lost on the what exactly to do with my pngs and making them into a material just for the plane.
hiya! im glad you like the video!
There are two steps to doing this, 1. In blender the plane Must have its own texture aside from the rest of the model. I show you that here 8:30
In unity, when you apply a texture to a model, it automatically creates a corresponding material. However, you can also right click and press Create > Material, in the "albedo" slot you can click and drag your png.
Your material inspector will look a little different from mine since i had a specific shader, but standard allows for transparency. All you have to do is go to the top of the shader options and click the drop down for "rendering mode" and select Cutout.
could you possibly make a tutorial video for blinking? and thank you so much for this tutorial !! :]
I have done blinking for a similar character. It's a bit more complicated and i by no means have perfected it, but i do have a method! i'll see if i can get time this year to make a video on it
thank you so much!!!
So how would I make the lip sync appear on the avatar when I speak? I noticed that you have the lip sync set to "viseme blend shape" in your avatar descriptor, but when I do that none of the visemes appear in any of the options below it.
Set it to "viseme paramater". Sorry for the confusion and the delay in response!
As long as you've set up the visemes in the avatar's fx layer like they are in the video, it should work.
This really helped me out - thank you!
This is a life saver!!! I tried making blend shapes with individual face plates for mine and they ended up clipping through the model. Do you think these visemes can be switched out for expressions?
do you mean using expressions instead of visemes or pausing visemes for expressions?
for the first one, yes, for the second one, im not so sure how. I'm no programmer but I assume there's a way to stop the visemes while you have a gesture on. If you wanna take the lazy route, see if you can make a blendshape that hides the mouth and exchanges it for one with the expression.
Thank you so much! I've been looking for a tutorial like this.
One question, do you know how to make it random? like, if I wanted to change my eye image to blinking image randomly?
i've asked around and picking from a random set of materials isn't really a thing vrchat can do? I'd have to do more research. It's hard to get answers to those kinds of questions. I DO however have an avatar with a 2d blinking animation, and thats just a sprite sheet of different images of the eyes with uv offsets with a LONG looping animation (2 minutes) to give it the illusion that the character is looking around and blinking at random intervals. I'll try to get a video of that together; hard to sit down and focus long enough to record
Hiya! Love the tutorial so far!! Huge help to me as im not to big of a fan of modelling 3d mouths with blendshapes, so this was a huge help to go for 2d mouths. Although i do have a problem i cant seem to fix yet, testing the visemes, the mouth does not return to the base viseme, it stays stuck to the last viseme played, has that happened with you, if so is there a fix?
[Edited] OOP I got it to work!!
glad it works for you!!
(sometimes saying the issue out loud helps you through it lol. Coding ducks and all)
They’re AWESOME
i hope both sides of ur pillow are comfortably cold, TYYY
What do i do if i only use two mouths like closed and open?
then use the mouth open for every viseme and the closed for the silent one. i think it should work!
isn't it easier and optimized to create multiple planes with all your visemes textures on them and use classic blendshapes instead of all this, it's automatically recognise in unity if i remember right (and if it's named correctly bafore exporting) Because for quest users you won't be able to use as much materiels as you do here
not necessarily-
you would still need to assign separate materials to all those planes, which in turn, isn't optimizing it any further than it already is. It's just adding more vertices.
it'd be more optimized* if you used a single sprite sheet, made as many planes as you needed, and UV offset the texture on each plane in your animations so that they all have the visemes on the sprite sheet. You'd still need to use viseme parameters for this.
*optimized here meaning uploadable for quest. the most optimized version of this method would be to use a single plane and a single sprite sheet, animating with uv offsets.
@@_colorbars_ no need to use multiple materials, one with all the expressions (expression sheet texture) and juste uv on the 11 or 13 different visemes textures, a unique 1024 or 2048 png and so blendshape for all visemes with the corresponding planes going in and out of your model instantly, that was indeed what i meant
@@_colorbars_ do you know how to do the same method with uv offsets in unity? it's what i'm currently trying to do! i also have another set of mouths specifically for happy expressions, but i feel like that would be more complicated and require actual coding lol
@@bashartz Yeah! just animate your offsets. If you press record and then change your offset in a shader, that should do the trick. Something like poyomi will require you to enable a part of your shader to be animated.
(finding out how to tile a sheet would be the trickiest part in this, im still figuring that out.)
For having another set of mouths, it's a little complicated to explain in text, but what you could do is have two different sheets- one for neutral, one for happy. Have two on different planes that you can switch out for that expression.
I'd recommend toggles in your radial menu for this, but it'd be interesting to see how it could work with just the finger gestures I suppose!
saved my life
Any way to add mmd compatibility?
im struggling to get this to work, i followed this step by step, it works in unity, but not in vrchat?
good stuff. i tried doing this a couple months ago and it kind of sucked
Big help! Thx
do you have a UV offset version of this? I'm trying to make my model oculus compatible for my friend
You'd need to make a sprite sheet and animate offsets for each viseme. I can't really supply a how-to video at the moment, but its not too difficult- there are videos showing how to animate offsets in unity.
INSTANT SUB!!!!
I made it so hard for myself lol animating material changes
would it be possible to make a tutorial thats super beginner friendly? i made and rigged a model but i dont know how to use unity that well
sorry for replying so late- there are plenty of tutorials on youtube that start from the very bottom! I don't have any to reccomend at the moment, as vrchat has changed quite a bit since I followed any, but sippbox does a good job.
Thank you a lot ^^
Sometimes the visemes get stuck
You'd have to share more details in order for me to help you out; I've never experienced this issue before. Check to see if your parameters match up with the way I've configured them
Thankuuuuuuu
im trying to upload but its saying the image is too big for the actions and moods you probably went over this in the video but there are no time stamps
i've never heard of this error, could you explain more?
also I didn't go over expressions in this video
never mind it was with another issue fixed it and the lip syncing is working perfectly@@_colorbars_
does this work for a vrm file? and does it work in vseeface?
if the set up is similar vrchat's sdk, then I don't see why not. I don't know anything about vseeface/vrm files- the whole reason this is possible is because vrchat has "Viseme parameters", which allow you to assign any animation to the lip sync.
@@_colorbars_ ah ok thank you I do not think it is possible
I will stick to placing my emotions inside my characters head and then figuring out different things
what is sil?
sil is Silence. As explained in the video, each viseme has a number assigned to it, Sil being 0. Since the set up for viseme parameters is a Int that returns to 0 when not detecting input (IE, you aren't talking or are muted), Sil is set at 0
@@_colorbars_ ohh thank you
why cant you just show all the different mouths you did so we know which one is for every viseme?
there are a lot of helpful tutorials online about lip sync animations
@@_colorbars_ not for 2d visemes
@@Optimusprime-ve7xi they work the same as keyframes for any 2d animation
what about if the mouth images were all in one texture map then you make multipleplanes on the face to assign each mouth image, then make shapekeys showing one at a time ...think that could work?
the mouth animations look so alive and cool.
yeah that could work! would add more polygons, but it wouldn't be that bad. You'd have to make the animation transition/exit times 0 though, so each plane shows up instantly.
@@_colorbars_ yes it works , found a tutorial that did exactly that.
thanks