Learning and planning with relational abstractions

ECE Seminar
Feb 19, 2024, 4:30 pm5:30 pm
B205 Engineering Quadrangle



Event Description


Decision-making in robotics domains is complicated by continuous state and action spaces, long horizons, and sparse feedback. One way to address these challenges is to perform bilevel planning, where decision-making is decomposed into reasoning about “what to do” (task planning) and “how to do it” (continuous optimization). Bilevel planning is powerful, but it requires multiple types of domain-specific abstractions that are often difficult to design by hand. In this talk, I will give an overview of my work on learning these abstractions from data. This work represents the first unified system for learning all the abstractions needed for bilevel planning. In addition to learning to plan, I will also discuss planning to learn, where the robot uses planning to collect additional data that it can use to improve its abstractions. My long-term goal is to create a virtuous cycle where learning improves planning and planning improves learning, leading to a very general library of abstractions and a broadly competent robot.


Tom Silver is a final year PhD student at MIT EECS advised by Leslie Kaelbling and Josh Tenenbaum. His research is at the intersection of machine learning and planning with applications to robotics and often uses techniques from task and motion planning, program synthesis, and neuro-symbolic learning. Before graduate school, he was a researcher at Vicarious AI and received his B.A. from Harvard with highest honors in computer science and mathematics in 2016. He has also interned at Google Research (Brain Robotics) and currently splits his time between MIT and the Boston Dynamics AI Institute. His work is supported by an NSF fellowship and an MIT presidential fellowship.


Electrical and Computer Engineering (Jaime Fernández Fisac)