If Agentic Robotics Is the Solution, What Is the Problem?

Author

Published

Nov 11, 2026

Share

•
Table of Contents

Everyone’s talking about agentic robotics now. But if agentic robotics is the solution, what is the problem?

Why should you read this? I think there will be new methods every month, even new trends and new areas. Only the pieces that solve critical problems in the field stay, and those problems stay around for a long time. So instead of chasing the methods, I think we should chase the problems. If you’re setting goals, it’s better to aim at those problems and work backward to find your methods.

So in this post I want to talk about what an agentic approach can tackle, what it still can’t tackle well, and what new capabilities changed the landscape a bit.

By agentic robotics, I mean a big model that plans, calls tools and skills, and acts in a loop: it looks, decides, acts, and checks what happened.

Robots already learn skills, and language models help them plan

Shakey the robot, with its camera, range finder and on-board logic labeled.

Shakey, SRI International, 1972: already seeing, planning and acting. Photo: SRI International, CC BY-SA 3.0.

What changed? The models are now good enough to finally close the loop

I think this is the key thing. Now the models are good enough to finally close the loop, from high level to low level, across different time scales and control scales.

A few years ago, language models mostly picked from the robot’s skill list, or wrote code on top of it. Now robot models output dexterous motor commands and think before they act. And a general model like OpenAI’s GPT-6 Astra can write a robot’s joint commands itself, or pick a humanoid’s skills to work in a kitchen the robot explored first.

So the hype is: plug Astra into a robot and you don’t need a trained policy anymore. But I think the tests are more interesting than that. It makes good task decisions, but finger coordination and walking still fail when it controls them directly. In one walking test, each call took about 40 seconds, and the simulation had to pause and wait. It often did better with a learned policy like π0.5, or a skill library, underneath. One evaluation says Astra “has largely crossed the threshold of embodied competence, but not yet that of practical embodied intelligence”.

At IROS 2026, Leslie Kaelbling gave her guess for why Astra is so good at robotics, most important first:

So this makes agentic robot systems finally viable. But what they can and can’t tackle well still depends on the problem.

Different assumptions mean different problems

Agentic robotics is just one of a few different solutions, and they don’t necessarily conflict with each other. I don’t think it’s a paradigm shift, but there are useful components inside, like the compositional, hierarchical architecture. So we still need to look at the problems. We make different assumptions and use different setups, so we’re often studying different problems. And if you change the assumptions, it’s really hard to carry a solution over.

So I think we need to say what problem we’re studying. It can be short: the goal, what the robot takes in and puts out, what skills it has, what it doesn’t know, how fast each part has to run, and how we test it against the strongest simple baseline. As Tom Silver puts it, “framing the problem is itself a contribution.”

What changes when we make different assumptions? For locomotion, we built the robot’s body, so we can model it pretty well. In manipulation, the robot also handles objects whose mass, friction and shape it doesn’t know.

So I keep coming back to the questions from textbooks like Russell and Norvig and Algorithms for Decision Making:

Design questions in two loops. Different methods answer different questions. Illustration.

Illustration. The inner loop is the robot deciding: perception, planning, skills. The outer loop is us designing the system: which parts we write by hand or learn, and what teaches them. Different methods sit in different places in these loops.

Robots don’t have internet-scale data, so a deeper understanding of the problem matters more

Imitation learning works well near its training data, and it can generalize, like robots that clean new homes. But it can only cover a certain amount of the data distribution. It’s a stretch to ask it to generalize to scenarios too different from its training data, and small mistakes can add up step by step.

And robots don’t have internet-scale data. Collecting demonstrations on a real robot takes robot time, and the variations keep adding up: objects, scenes, instructions, robot bodies, task lengths.

So I think one way is to clearly understand the structure of the problem, like its compositional structure. Language has compositional structure, so a few thousand words can make infinite combinations. I think the physical world has structure like this too, and it lets us generalize in a compositional way. Not every task decomposes cleanly, though.

I want to use that structure both in training and at test time.

Pick tasks that actually have the problem

If the task only needs imitation of trajectories, what benefit are we trying to show with an agentic approach? Does the task actually have the problem we’re trying to solve? And what does the agent add over a strong simple baseline?

A short task may not gain much from longer reasoning. A fixed work cell may only need a fast policy. Cleaning a room is different. It’s long horizon, objects are hidden, and people walk around.

One way to check: give the robot the information it needs, then hide it. Or keep the world still, then let it change. Compare with a strong simple baseline, and check whether an off-the-shelf agent already solves it.

My bet is to decompose the loop, learn from the proper data at scale, and think online

If we’re studying tasks with long horizons, partial observability and compositional structure, I think an agentic approach makes sense. For pure locomotion or one short skill, we probably don’t need it. Or that policy can just be one skill inside a robot agent.

Here’s the direction I want to explore:

  1. Break down the whole loop in a principled way, like in Retriever and the work of George Konidaris, Leslie Kaelbling and Tomás Lozano-Pérez.
  2. Find efficient supervision for each piece. Some pieces may need data the robot collects itself, as in the era of experience.
  3. Learn the proper models with the proper data, at scale of data and compute, as The Bitter Lesson argues. For some pieces, we have a lot more data in a certain modality. For example, there’s a lot more video data, so people use it to train video models.
  4. Compose the system. I think getting the pieces to work together is its own problem.
  5. Spend more compute online on reasoning and planning, like planning in the now, AlphaZero and MuZero. I think we can use more search at test time, with a simple set of rules to recombine what the robot already learned.

What problem are you working on, and what assumptions does it depend on?