Matthew Chan, NVIDIA Research
ABSTRACT: Traditionally, creating 3D digital humans requires lengthy efforts by digital artists, and often costly 3D scanning by special multi-view scanners. Learn how recent generative AI technologies allow the learning of photorealistic 3D representations from a collection of in-the-wild 2D images, such as internet photos. We'll dive deep into our recent work called “EG3D” and “WYSIWYG”, which can synthesize wide varieties of photorealistic 3D humans in real time. We'll also show how 3D synthetic data from a pre-trained 3D generative model can be used to train another AI model for challenging image synthesis tasks. To this end, we present our recent work called “LP3D,” which can synthesize photorealistic neural radiance field (NeRF) models from a single RGB image in real time. We'll demonstrate how these AI-driven human synthesis methods can make the advanced capabilities, such as 3D video conferencing, accessible to anyone and enable new applications in the future.
BIO: Matthew Chan joined NVIDIA as a research engineer in 2022. They primarily work at the intersection between graphics and generative models, specifically how they relate to 3D scene synthesis, reconstruction, and understanding. They graduated from University of Maryland, College Park in 2021 with a bachelor's degree in mathematics and computer science.
research.nvidia.com/labs/amri/author/matthew-chan/
research.nvidia.com/labs/nxp/wysiwyg/
Joint event by Silicon Valley ACM SIGGRAPH (SVSIGGRAPH), San Francisco Bay Area ACM (SFBayACM) and Los Angeles ACM SIGGRAPH (LASIGGRAPH).
www.sfbayacm.org/
lasiggraph.org/
www.meetup.com/sv-siggraph/events/299800871/
0:00 ACM Chapters Intros
5:42 Speaker Intro
6:36 Presentation
7:27 Long History of Telepresence Efforts...
10:31 Generative Models
13:37 3D-aware GAN: Unsupervised Learning Of Photorealistic 3D Faces
34:03 Conditional 3D Face Synthesis
46:03 AI-Mediated 3D Telepresence
51:11 Questions and Answers Q&A