Small Models for Big Worlds: Notes on Leslie Kaelbling's IROS 2026 Talk

Author

Published

Nov 9, 2026

Share

•
Table of Contents

Leslie Kaelbling gave a talk at the IROS 2026 workshop on compositional robotics in Pittsburgh on September 27, called “Small models for big worlds.” She opened by giving it a second title: “Robotics is not solved.” Her point: robotics still has hard open problems, and compositionality and modularity may help with them.

I enjoy Leslie’s talks a lot. She typically gives contrarian views in her talks, and every word counts (using her own words, “like a program”). This talk also very concisely describes why compositionality/modularity are helpful in just first 10 minutes. I’ll try to share a faithful note below.

Leslie Kaelbling at the podium beside her title slide, "Small models for big worlds."

Leslie Kaelbling (MIT), right, beside the podium, with her title slide on the left. All photos in this post are mine.

Modules are good and bad

She started with vocabulary, because “world model” gets used for two different things:

Slide: "Modular systems are good and bad."

Five things modules are good for, and two costs.

Compositional is more than modular

Slide: "Compositional systems."

Her definition of a compositional system.

For her, a compositional system has:

Where are the seams?

Slide: "Compositional seams in the physical world."

Candidate seams on her slide: objects, chunks of space, agents, qualities, and meta-qualities of modules such as certainty, computation time, memory and energy.

Then the big question: “What are the seams in the physical world?” In other words, where can we pull a problem apart so that learning and reasoning get easier?

Two slides: a world state model with objects, rooms and other agents, and a factored world transition model.

Left: a world state model, with objects, rooms and other agents; it can be uncertain. Right: a world transition model built from factored rules over objects and their properties.

Both kinds of world model can be compositional:

A modular system that plans

Slide titled "TipTop: A Modular Open-Vocabulary Robot Manipulation System That Plans" (her spelling).

Her slide introducing TiPToP by Will Shen, Nishanth Kumar and colleagues (the slide spells it “TipTop”).

Next, a recent system from her group:

Why is Astra so good at robotics?

Slide: "Why is Astra so good at robotics?"

Four reasons, which she said are in order of importance.

People are freaking out that Astra can do everything. She joked that maybe we should all just go to the beach. Her guess at why it’s so good, most important first:

  1. A compositional model of the world, in natural language. “Natural language tells us what the compositional structure of our world is, because humans developed it to talk about the world efficiently.”
  2. Recursive composition of tool calls, like a program that calls subroutines.
  3. Good tools from roboticists. “It’s also good because of us”: we’ve built great general tools, like inverse kinematics, that it can call.
  4. A lot of robotics data in training.

Robotics is not solved

Slide: "Robotics is not solved!!"

What an intelligent autonomous robot still needs.

Then she came back to her second title. An intelligent autonomous robot needs to:

“These are all things that I think we have not yet worked out.”

Recent work from her lab

Slide: "Some recent MIT LIS work in these directions."

Three recent projects from her lab at MIT.

Then three recent projects that, as she put it, fill in some of the gaps Astra doesn’t cover. Each one carries a view:

Her summary: to solve a big problem with a small brain, “take the big problem and turn it into a sequence of small problems.”

Architectures, from 1959 to 2026

Five slides: 1959 Pandemonium, 1981 Albus, 1985 subsumption, 2025 agentic architecture, 2026 Retriever.

Five slides from her short history of robot architectures. Years as on her slides.

Near the end, she walked through a few representative architectures.

Then she went back in time:

The "2026: Retriever!" slide.

Her last architecture slide, “2026: Retriever!”, with the system diagram from our paper and the author names. Project page, paper.

My take: It’s great to see Leslie put our work in the lineage after the classic hierarchical robot architectures! I’m pretty happy that Leslie likes the work.

She closed the history with: “We have to all start taking seriously the idea that our computation happens in time and figure out how to manage that.”

From the questions

My take

What I keep thinking about