Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

Automating the assembly of objects from their parts is a complex problem with innumerable applications in manufacturing, maintenance, and recycling. Unlike existing research, which is limited to target segmentation, pose regression, or using fixed target blueprints, our work presents a holistic multi-level framework for part assembly planning consisting of part assembly sequence inference, part motion planning, and robot contact optimization. We present the Part Assembly Sequence Transformer (PAST) -- a sequence-to-sequence neural network -- to infer assembly sequences recursively from a target blueprint. To train PAST, we introduce D4PAS: a large-scale Dataset for Part Assembly Sequences consisting of physically valid sequences for industrial objects. We then use a motion planner and optimization to generate part movements and contact selection. Experimental results show that our learning-based approach matches the performance of the state-of-the-art simulation-based method but with significantly reduced computational time.