
Open-world Segmentation and Tracking in 3D
Laura Leal-Taixé of NVIDIA presents this Stanford AA289 Robotics and Autonomous Systems Seminar talk on training models to detect and segment 3D data, such as Lidar point clouds, without hand-annotating data in the 3D domain. She walks through methods for generating and using pseudo-labels to substitute for costly 3D annotation, then demos NVIDIA's Segment Anything in Lidar system, showing how open-world segmentation can generalize to unseen object categories. The talk situates the work within broader open-world perception problems in autonomous systems, where models must detect and track objects that were never part of a fixed training taxonomy. Aimed at a graduate robotics audience, it covers the technical pipeline from pseudo-label generation to model training and evaluation, with a live demonstration illustrating the system's practical performance on real Lidar scans.