Audio2Gesture - Part 1: Introduction

Note: This video may not reflect the current shipping version
In this tutorial, we'll provide an overview of Audio2Gesture that covers the basic usage of the extension in the Machinima application, and how to initialize the extension and create a pipeline.
Watch Part 2: Audio2Gesture - Part 2: Offline Pipeline and Creating a Scene with the Sequencer ( • Audio2Gesture - Part 2... )
Watch Part 3: Audio2Gesture - Part 3:Streaming Pipeline and Integration with Animation Graph ( • Audio2Gesture - Part 3... )
---------------------------------------------------------------
NVIDIA Omniverse™ is an open platform built for virtual collaboration and real-time photorealistic simulation. Complex creator, designer, and engineering visual workflows are transformed as users and teams connect design tools, assets, and projects for collaborative iteration in a virtual world.
Download Omniverse today → nvda.ws/3gNZsu0.
Try Omniverse Enterprise free for 30 days → nvda.ws/30uI0aq
See the full collection of tutorials at NVIDIA On-Demand → nvda.ws/2WluMH2
CONNECT WITH US ON SOCIAL
Twitter: / nvidiaomniverse
Instagram: / nvidiaomniverse
LinkedIn: / nvidia-omniverse
Discord: / discord
#nvidiaomniverse
#pixarusd
#openusd
#machinima

[@ryansalazaracosta2596]

Great stuff. I been working whit A2F thist last 2 months and I think is beautiful. Now I will go to A2G. Thanks!

[@WiderMotion]

Great, thanks!

[@HoneyDrops2023]

Is there a way to one day apply this to animation, as in using AI to create motion cap --that would be amazing.

[@untitledprojectSG]

this does, it animates a skeleton, its exactly what your asking for

[@Netsurai]

It is strange that the Meta Human Skeleton cannot be imported into Omniverse from the beginning. Looks like the Omniverse developers have no interest or time? To implement (please) at least a UE4/5 standard skeleton. That's a shame, now 2000+ people can play around with this “completely unstandardized” OmniVerse skeleton. I like to call it the “Snake Human” skeleton because it has way too many body joints for most animation projects.
Unfortunately it's always the same, every animation app makes its own human skeleton, unfortunately in 2022 there is no minimum base standard in 3D apps, the only standard I know was BioVision Motion Capture (.BVH). We even messed weeks around in UE4/5 to retraget UE to Meta Human. Let's hope that in 2024 an AI tool can solve this mess or all 3D animation tools implement at least something along the lines of the old BVH as new Human Standard Skeleton.
Sometimes I think that these developers regrettably have no idea what the end user really needs! They code the greatest things in, but they don't use the app in practice …. Exporting OmniVerse to UE 5 would be a very useful field test, right?
Unfortunately, there is still a lack of compatibility here. Still, UE 5 could be an ideal platform to show the world something like Audio2Gesture. But please not with your Omniverse “Snake Human” Skeleton, where I would first have to rearrange joints for days. ;)