Skip to main content

Towards Structured Physical AI Models

ECE Seminar

-
Location: EER 3.646
Speaker:
Roei Herzig
UC Berkeley

Abstract: Current AI systems can synthesize videos, pass the bar exam, and write 
code. Despite these advances, robots still struggle with basic physical tasks, like folding 
a shirt, that humans perform naturally. This disparity stems from the Robotic Data Gap: 
Robotics has no internet. While digital AI trains on billions of hours of web data, robot 
learning relies on small, costly datasets that are difficult to standardize and highly 
heterogeneous. In contrast, humans are remarkably data-efficient, generalizing 
effortlessly from limited experience. This raises a key research question: Can we bridge 
this gap by building Physical AI systems that perceive, reason, and adapt to the 
physical world, driving data efficiency and scalable generalization?


In this talk, I will present recent efforts in Physical AI to integrate physical inductive 
biases, allowing robots to generalize beyond their limited training data. I will highlight 
ongoing work that incorporates structured representations, such as motion particles, 
object geometries, symmetries, and affordances, into learning-based robotic models. 
My work, spanning from manipulation arms to humanoids, demonstrates that this 
structured approach is the key to unlocking data-efficient Embodied AI despite the 
constraints of real-world data scarcity.


Bio: Roei Herzig is a Postdoctoral Scholar at UC Berkeley and a Research Scientist at 
the MIT-IBM Watson AI Lab. He is advised by Professor Trevor Darrell and collaborates 
with Professors Jitendra Malik, Shankar Sastry, and Deva Ramanan. Roei earned his 
PhD from Tel Aviv University under the supervision of Professor Amir Globerson. His 
research develops models and learning algorithms for Physical AI, grounded in 
structural priors, using robots as the ultimate testbed. He has been recognized with 
several distinctions, including the 2023 Dissertation Award for the best AI thesis in Israel 
and the Israeli Excellence in Data Science Postdoctoral Fellowship