Everyone’s talking about agentic robotics now. But if agentic robotics is the solution, what is the problem?
Why should you read this? I think there will be new methods every month, even new trends and new areas. Only the pieces that solve critical problems in the field stay, and those problems stay around for a long time. So instead of chasing the methods, I think we should chase the problems. If you’re setting goals, it’s better to aim at those problems and work backward to find your methods.
So in this post I want to talk about what an agentic approach can tackle, what it still can’t tackle well, and what new capabilities changed the landscape a bit.
By agentic robotics, I mean a big model that plans, calls tools and skills, and acts in a loop: it looks, decides, acts, and checks what happened.
Robots already learn skills, and language models help them plan
- Robots learn a lot from data. They handle tasks with many ways to do them, like Implicit BC and Diffusion Policy, and do fine two-handed work, like ALOHA and ALOHA Unleashed. They use knowledge from the web, like RT-2 and OpenVLA, learn from lots of different robots, like Open X-Embodiment and Octo, and even work in homes they’ve never seen. Legged robots learn agile skills in simulation and walk on rough terrain, and robots navigate just from camera images.
- Language models help with planning and code. They break instructions into steps, like zero-shot planners and PaLM-E, pick steps the robot can actually do, like SayCan and Text2Motion, adjust when something goes wrong, like Inner Monologue and REFLECT, and write robot code, like Code as Policies and ProgPrompt.
- And the loop itself isn’t new. Shakey already saw, planned and acted around 1970. Robots have long layered reflexes under higher-level behaviors, planned steps and motions together, and reasoned about what they can’t see.

Shakey, SRI International, 1972: already seeing, planning and acting. Photo: SRI International, CC BY-SA 3.0.
What changed? The models are now good enough to finally close the loop
I think this is the key thing. Now the models are good enough to finally close the loop, from high level to low level, across different time scales and control scales.
A few years ago, language models mostly picked from the robot’s skill list, or wrote code on top of it. Now robot models output dexterous motor commands and think before they act. And a general model like OpenAI’s GPT-6 Astra can write a robot’s joint commands itself, or pick a humanoid’s skills to work in a kitchen the robot explored first.
So the hype is: plug Astra into a robot and you don’t need a trained policy anymore. But I think the tests are more interesting than that. It makes good task decisions, but finger coordination and walking still fail when it controls them directly. In one walking test, each call took about 40 seconds, and the simulation had to pause and wait. It often did better with a learned policy like π0.5, or a skill library, underneath. One evaluation says Astra “has largely crossed the threshold of embodied competence, but not yet that of practical embodied intelligence”.
At IROS 2026, Leslie Kaelbling gave her guess for why Astra is so good at robotics, most important first:
- Language gives it a compositional model of the world.
- Its computational structure is a recursive composition of tool calls, like a program calling subroutines.
- It calls great tools roboticists built: “it’s also good because of us”.
- It has been trained on a lot of robotics data.
So this makes agentic robot systems finally viable. But what they can and can’t tackle well still depends on the problem.
Different assumptions mean different problems
Agentic robotics is just one of a few different solutions, and they don’t necessarily conflict with each other. I don’t think it’s a paradigm shift, but there are useful components inside, like the compositional, hierarchical architecture. So we still need to look at the problems. We make different assumptions and use different setups, so we’re often studying different problems. And if you change the assumptions, it’s really hard to carry a solution over.
So I think we need to say what problem we’re studying. It can be short: the goal, what the robot takes in and puts out, what skills it has, what it doesn’t know, how fast each part has to run, and how we test it against the strongest simple baseline. As Tom Silver puts it, “framing the problem is itself a contribution.”
What changes when we make different assumptions? For locomotion, we built the robot’s body, so we can model it pretty well. In manipulation, the robot also handles objects whose mass, friction and shape it doesn’t know.
So I keep coming back to the questions from textbooks like Russell and Norvig and Algorithms for Decision Making:
- Can the robot see what it needs?
- Does the world wait while it thinks?
- Do we know the dynamics?
- Do the test tasks look like the training tasks?
- Is the robot alone?
- Can it reset and try again?

Illustration. The inner loop is the robot deciding: perception, planning, skills. The outer loop is us designing the system: which parts we write by hand or learn, and what teaches them. Different methods sit in different places in these loops.
Robots don’t have internet-scale data, so a deeper understanding of the problem matters more
Imitation learning works well near its training data, and it can generalize, like robots that clean new homes. But it can only cover a certain amount of the data distribution. It’s a stretch to ask it to generalize to scenarios too different from its training data, and small mistakes can add up step by step.
And robots don’t have internet-scale data. Collecting demonstrations on a real robot takes robot time, and the variations keep adding up: objects, scenes, instructions, robot bodies, task lengths.
So I think one way is to clearly understand the structure of the problem, like its compositional structure. Language has compositional structure, so a few thousand words can make infinite combinations. I think the physical world has structure like this too, and it lets us generalize in a compositional way. Not every task decomposes cleanly, though.
- In time: planning puts steps the robot already knows into new sequences, so we don’t need a demonstration for every long task.
- In space: knowing objects and how they relate, like the spice in the drawer, can help in a new scene. The hard part is connecting those concepts to what the camera actually sees.
I want to use that structure both in training and at test time.
Pick tasks that actually have the problem
If the task only needs imitation of trajectories, what benefit are we trying to show with an agentic approach? Does the task actually have the problem we’re trying to solve? And what does the agent add over a strong simple baseline?
A short task may not gain much from longer reasoning. A fixed work cell may only need a fast policy. Cleaning a room is different. It’s long horizon, objects are hidden, and people walk around.
One way to check: give the robot the information it needs, then hide it. Or keep the world still, then let it change. Compare with a strong simple baseline, and check whether an off-the-shelf agent already solves it.
My bet is to decompose the loop, learn from the proper data at scale, and think online
If we’re studying tasks with long horizons, partial observability and compositional structure, I think an agentic approach makes sense. For pure locomotion or one short skill, we probably don’t need it. Or that policy can just be one skill inside a robot agent.
Here’s the direction I want to explore:
- Break down the whole loop in a principled way, like in Retriever and the work of George Konidaris, Leslie Kaelbling and Tomás Lozano-Pérez.
- Find efficient supervision for each piece. Some pieces may need data the robot collects itself, as in the era of experience.
- Learn the proper models with the proper data, at scale of data and compute, as The Bitter Lesson argues. For some pieces, we have a lot more data in a certain modality. For example, there’s a lot more video data, so people use it to train video models.
- Compose the system. I think getting the pieces to work together is its own problem.
- Spend more compute online on reasoning and planning, like planning in the now, AlphaZero and MuZero. I think we can use more search at test time, with a simple set of rules to recombine what the robot already learned.
What problem are you working on, and what assumptions does it depend on?