SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation

Physically-simulated models for human motion can generate high-quality responsive character animations, often in real-time. Natural language serves as a flexible interface for controlling these models, allowing expert and non-expert users to quickly create and edit their animations. Many recent physics-based animation methods, including those that use text interfaces, train control policies using reinforcement learning (RL).
However, scaling these methods beyond several hundred motions has remained challenging. Meanwhile, kinematic animation models are able to successfully learn from thousands of diverse motions by leveraging supervised learning methods. Inspired by these successes, in this work we introduce SuperPADL, a scalable framework for physics-based text-to-motion that leverages both RL and supervised learning to train controllers on thousands of diverse motion clips.
SuperPADL is trained in stages using progressive distillation, starting with a large number of specialized experts using RL. These experts are then iteratively distilled into larger, more robust policies using a combination of reinforcement learning and supervised learning.
Our trained SuperPADL controller runs in real time on a consumer GPU, and can reproduce motions with high fidelity from a dataset of over 5000 skills. Moreover, our policy can naturally transition between skills, allowing for users to interactively craft complicated animations. We experimentally demonstrate that SuperPADL significantly outperforms RL-based baselines at this large data scale.
Learn more: arxiv.org/abs/...
blogs.nvidia.c...
#nvidiaresearch #generativeai #siggraph2024

[@RyanBrownUmd]

Looks excellent!! Would love to try it out, will the public be able to try out this research with published model weights or api at some point in the future? Or is only the research being published?

[@connorharris1900]

I still haven’t seen an acceptable physics model that demonstrates the interaction of all the “laws” and complete integration into the fabric of a given “reality” I do believe you can get there

[@shirkit5798]

This looks impressive! Is there a way to the public to try it out?

[@TheAntonio52688]

I'm really impressed, i want to ask if it is possible to use now?

[@alexei5231]

could be used for robot training

[@LOC-Ness]

this could make creating active ragdoll animation easier

[@lunars9793]

Looks better!

[@hengshan-y6w]

is this open source ?

[@hengshan-y6w]

is this open source ?