Spring 2024 GRASP SFI-Andrew Owens, University of Michigan, “Multimodal Learning from the Bottom Up”

ABSTRACT
Today’s machine perception systems rely extensively on supervision provided by humans, such as natural language. I will talk about our efforts to make systems that, instead, learn from two ubiquitous sources of unlabeled sensory data: visual motion and cross-modal associations between senses. First, I will discuss our work on creating unified self-supervised motion analysis methods that can address both object tracking and optical flow tasks. I will then discuss how these same techniques can be applied to localizing sound sources in video, and how tactile sensing data can be used to train multimodal visual-tactile models. Finally, I will talk about our recent work on subverting visual perception systems, by creating “multi-view” optical illusions: images that change their appearance under a transformation, such as a flip or rotation.
PRESENTER
Andrew Owens is an assistant professor at The University of Michigan in the department of Electrical Engineering and Computer Science. Prior to that, he was a postdoctoral scholar at UC Berkeley. He received a Ph.D. in Electrical Engineering and Computer Science from MIT in 2016. He is a recipient of a Computer Vision and Pattern Recognition (CVPR) Best Paper Honorable Mention Award.