GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians

Project: shenhanqian.gi...
We introduce GaussianAvatars, a new method to create photorealistic head avatars that are fully controllable in terms of expression, pose, and viewpoint. The core idea is a dynamic 3D representation based on 3D Gaussian splats that are rigged to a parametric morphable face model. This combination facilitates photorealistic rendering while allowing for precise animation control via the underlying parametric model, e.g., through expression transfer from a driving sequence or by manually changing the morphable model parameters. We parameterize each splat by a local coordinate frame of a triangle and optimize for explicit displacement offset to obtain a more accurate geometric representation. During avatar reconstruction, we jointly optimize for the morphable model parameters and Gaussian splat parameters in an end-to-end fashion. We demonstrate the animation capabilities of our photorealistic avatar in several challenging scenarios. For instance, we show reenactments from a driving video, where our method outperforms existing works by a significant margin.

[@mikestaub]

This is incredible. Very smart to combine the different rendering pipelines to get the best of both worlds, bravo!

[@vishalanime]

Ah man this won't go public anytime soon, but just another crazy mic drop from Prof. Niessner 👌🏽

[@spirobel2.0]

dis is amazing thanks! looking forward to the source code release 😍

[@manu.vision]

Amazing, can’t wait to play with this!

[@Eric.Tomaszewski]

Amazing project. Is there a github code to try pls?

[@naninano8813]

The space is heating up! arxiv 2312.03029 (Gaussian Head Avatars) published at around same time, shows similar results and approach

[@evesattler1570]

has the gaussian head avatar code been published yet? It`s marked as public on their git but the code only consists of a readme file as of yet

[@ameerazam3269]

When you guys release code ?

[@邓珺礼]

what is the software you used , control joints and expressions

[@DaveDFX]

Amazing ! Game changer for avatars

[@MisterWealth]

When will the code be made available?

[@RayRift]

Wow! Quite amazing!

[@technoshaman001]

I want an avatar like this of myself, how do I become part of this program?

[@r.m8146]

Revolutionary.

[@Mwesi]

Wow; excellent work!

[@jameslangridge8849]

So could the finished model be rigged for vr chat or a vtuber or something. If so content creation is about to get wild 😄

[@Danuxsy]

I literally just saw another gaussian video and I thought "couldn't you combine polygons with gaussians?" and then I saw this lol xD

[@VritanshKamal]

Incredible ! !

[@modelisationLolito]

Great job

[@wilbertandrews]

I'd like to try this 👍🏻

[@carlosmunnoz4188]

Awwwesome mate! 🔥🔥🔥

[@rallyworld3417]

Impressive

[@narathipthisso4969]

Great 👍🏻 😊

[@ganeshtamang1354]

wow love it

[@haikeye1425]

Great!

[@Instant_Nerf]

What’s happening anymore

[@mrburns366]

So.. LA Noire 2? 😁

[@hos42]

Why do I feel an impending sense of doom for our society?

[@JT-Works]

The end is just the beginning