"Pose-Appearance Disentanglement in Hybrid Neural Fields" - Sam Buchanan, Research at TTIC

Originally presented on: Friday, March 22nd ,2024 at 12:30am CT, TTIC, 6045 S. Kenwood Avenue, 5th Floor, Room 530
Title: "Pose-Appearance Disentanglement in Hybrid Neural Fields"
Speaker: Sam Buchanan, TTIC
Abstract: Factored feature volumes offer a simple way to build more compact, efficient, and interpretable neural fields, but also introduce biases that are not necessarily beneficial for real-world data. In this work, we (1) characterize the undesirable biases that these architectures have for axis-aligned signals, and (2) explore how learning a set of canonicalizing transformations can improve representations by removing these biases. We prove in a two-dimensional model problem that simultaneously learning these transformations together with scene appearance succeeds with drastically improved efficiency. We validate the resulting architectures, which we call TILTED, using image, signed distance, and radiance field reconstruction tasks, where we observe improvements across quality, robustness, compactness, and runtime. Results demonstrate that TILTED can enable capabilities comparable to baselines that are 2x larger, while highlighting weaknesses of neural field evaluation procedures.
#deeplearning #theory #nerf #computervision