The series continues in Spring 2022 featuring speakers from both Princeton and other institutions. It is scheduled on Fridays at 3-4PM EST, and will be either in-person (EQuad D221) for Princeton speakers or over Zoom (link) for external speakers.

If you have a Princeton email, please fill out the form to subscribe to the robotics-seminar mailing list.

April 22, 2022 - Mac Schwager, Stanford - Location: Zoom

Reimagining Robot Autonomy with Neural Environment Representations

Abstract: New developments in computer vision and deep learning have led to the rise of neural environment representations: 3D maps that are stored as deep networks that spatially register occupancy, color, texture, and other physical properties.  These environment models can generate photo-realistic synthetic images from unseen view points, and can store 3D information in exquisite detail.  In this talk, I investigate the question: How can robots use neural environment representations for perception, motion planning, manipulation, and simulation?  I will show recent work from my lab in which we build a robot navigation pipeline using a Neural Radiance Field (NeRF) map of an environment. We develop a trajectory optimization algorithm that interfaces with the NeRF model to find dynamically feasible, collision-free trajectories for a robot moving through a NeRF world.  We also develop an optimization-based state estimator that uses the NeRF model to give full dynamic state estimates for a robot from only on board images. I will also discuss preliminary results using NeRF models for grasp planning, and for tracking the poses of multiple 3D objects in a scene. Finally, I will discuss a new differentiable robot physics simulator called Dojo that can use NeRFs as a geometry description for objects, leading to physically interpretable motion prediction from NeRF models.  I will conclude with future opportunities and challenges in integrating neural environment representations into the robot autonomy stack.

Bio: Mac Schwager is an Associate Professor of Aeronautics and Astronautics at Stanford University.  He directs the Multi-robot Systems Lab (MSL) where he studies distributed algorithms for control, perception, and learning in groups of robots and autonomous systems.  He is interested in a range of applications including cooperative surveillance with teams of UAVs, autonomous driving in traffic, cooperative robotic
manipulation, distributed SLAM, distributed trajectory planning, and autonomous drone and car racing.  He obtained his BS degree from
Stanford, and his MS and PhD degrees from MIT.  He was a postdoctoral researcher at the University of Pennsylvania and at MIT.  Prior to joining Stanford, he was an assistant professor at Boston University from 2012 to 2015.  He received the NSF CAREER award in 2014, the DARPA YFA in 2018, and has received numerous best paper awards in conferences and journals including the IEEE Transactions on Robotics best paper award in 2016, the Best Paper Award in Robot Manipulation in ICRA 2018, and the Best Paper Award in Multi-Robot Systems in ICRA 2020.

April 8, 2022 - Stephen Tu - Learning from many trajectories

March 25, 2022 - Karen Leung - Towards the Unification of Autonomous Vehicle Safety Concepts: A Reachability Perspective

March 11, 2022 - Reza Moini - Bio-inspired Design and Additive Manufacturing of Architected Cement-based Materials

Feb 25, 2022 - Aimy Wissa - Bio-Inspired Locomotion Strategies across Mediums: From Feather-Inspired Flow Control to Beetle-Inspired Jumping

Feb 11, 2022 - Jordan Taylor - The steep part of the learning curve: how cognitive strategies shape motor skill acquisition

Dec 3, 2021 - Naomi Leonard - Collective Intelligence and Multi-Robot System

Nov 19, 2021 - Aaron Ames - Safety-Critical Control of Dynamic Robots

Nov 5, 2021 - Chuchu Fan - Building Dependable Autonomous Systems through Learning Certified Decisions and Control

Oct 8, 2021 - Karthik Narasimhan - Language-guided policy learning for better generalization and safety

Sep 24, 2021 - Daniel Cohen - Living microrobots: controlling cellular swarms and the waterbear as a potential microrobot chassis

Sep 17, 2021 - Michael Posa - Contact-Rich Robotics: Learning, Impact-Invariant Control, and Tactile Feedback

Feb 11, 2021 - Jia Deng - Optimization Inspired Deep Architectures for Multiview 3D

Feb 25, 2021 - Stefana Parascho - Rethinking Architectural Robotics

Mar 11, 2021 - Naveen Verma - AI Meets Large-scale Sensing: preserving and exploiting structure of the real world to enhance machine perception

April 8, 2021 - Jaime Fernandez Fisac - Safe Robots in the Wild: maintaining safety by planning through uncertainty and interaction

April 22, 2021 - Olga Russakovsky, Zhiwei Deng - Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation

May 6, 2021 - Bartolomeo Stellato - Data-Driven Embedded Optimization for Control

Allen Ren, Alexandra Bodrova, Nate Simon, Anirudha Majumdar