Teach and Grow: An Agent-Centered Architecture for General Robot Learning

1. Key Themes

Avoiding the "Retraining Tax" via Explicit Skill Blocks

The paper identifies a fundamental flaw in monolithic Vision-Language-Action (VLA) models: when they fail on an out-of-distribution (OOD) object or contact, fixing it requires new data collection, model updating, and regression testing across all previously supported behaviors. TGL solves this by turning a few demonstrations into reusable "Skill Blocks"—closed-loop behaviors that can be added, tested, and retained locally without rewriting the foundation model. As stated in Section I: "This mismatch between a local gap and a global update is the retraining tax... General robotics therefore needs an architecture that can add and validate local competence without rewriting unrelated behavior."

Agent-Centered Dynamic Composition and Feedback

Instead of a fixed execution pipeline, TGL places a multimodal AI agent above the workflow. The agent retrieves and composes Skill Blocks, selects executors (learned policies, geometric planners, or controllers), and observes the physical outcome to revise the plan dynamically. Section IV.C notes: "A conventional VLA or WAM can react to new observations while still operating inside a workflow fixed before the episode... TGL adds this semantic control layer. After each verified effect, the Agent may continue, acquire more evidence, select another executor, replace the remaining blocks, request targeted teaching, or stop."

The "Teach-and-Grow" Lifetime Scaling Law Hypothesis

The paper proposes a new scaling law based on "effective reusable experience" (X), rather than model parameters or offline data. It hypothesizes that as a robot accumulates validated, reusable skills, the error on future tasks and the teaching required to learn them will decrease as power laws. Section I states: "if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier."

2. Contrarian Perspectives

End-to-End VLA Scaling is Bottlenecked by Physical Data Production, Not Just Model Size

While the industry trend is to scale up VLA models and datasets, this paper argues that physical coverage grows combinatorially and cannot be solved by web-scale semantics alone. Appendix A points out: "Web-scale semantics may identify the intended object or instruction, but they do not by themselves supply the geometry, dynamics, or contact behavior needed for reliable execution." The paper argues that the cost of producing physical data for dense coverage scales as $r^F$ (where $F$ is the number of interacting factors), making the end-to-end route practically unsustainable for general robotics.

Demonstrations are Evidence for Structure, Not Trajectories to Imitate

Standard imitation learning treats demonstrations as action targets to copy. TGL treats them as evidence to infer semantic subgoals and reusable strategies, discarding the original motion. Appendix B states: "The demonstrations do not determine the final action policy. They establish an initial hypothesis about task structure: which semantic effects matter, how they are ordered, and which relations survive across demonstrations." This means a robot can learn from human video or different embodiments without needing to match their exact kinematics.

3. Companies Identified

OpenAI

Description: AI research and product company. Why relevant: The paper draws inspiration from coding agents like Codex for the agentic architecture, and uses "gpt-5.6-sol" as the multimodal agent in their experimental stack (Appendix I). Quotes: "Systems such as Codex and Claude Code inspect a changing workspace, form a plan, call tools, read the result, and revise their strategy until the goal is complete" (Section I). "The online traces use gpt-5.6-sol with seed 0 and structured decisions" (Appendix I).

Anthropic

Description: AI safety and research company. Why relevant: Claude Code is cited alongside Codex as the inspiration for the agentic loop that TGL adapts for physical robotics. Quotes: "Systems such as Codex and Claude Code inspect a changing workspace, form a plan, call tools, read the result, and revise their strategy until the goal is complete" (Section I).

Physical Intelligence (π0)

Description: Robotics foundation model company. Why relevant: Referenced as a leading generalist VLA policy whose approach (scaling model and dataset) is challenged by the TGL architecture. Quotes: "π0: A vision-language-action flow model for general robot control" (References). The paper positions TGL as an alternative to the "dominant vision of general-purpose robotics" held by models like RT-1, RT-2, and π0.

AgiBot

Description: Embodied AI and robotics data company. Why relevant: Their dataset, AgiBot World, is cited as a major advance in robot data, but also as evidence of the massive infrastructure needed to create physical data substrates. Quotes: "AgiBot World are major advances... their scale also makes visible how much new infrastructure and interaction must be created before robot data can resemble a mature web-scale substrate" (Appendix A).

4. People Identified

Chang Nie, Zhe Liu, and Hesheng Wang

Lab/Institution: School of Automation and Intelligent Sensing, Shanghai Jiao Tong University. Why notable: The authors of the paper, proposing a paradigm shift from monolithic VLA models to an agent-centered, skill-block architecture for lifelong robot learning. Quotes: "We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks" (Abstract).

5. Operating Insights

Separate Semantic Reasoning from Physical Execution

CTOs should avoid asking language models to directly issue motor commands. The architecture deliberately separates the Agent (which handles semantic state transitions, route planning, and outcome verification) from robot-native executors (which handle metric motion, contact, and control). Section VI.C advises: "Placing the Agent above the workflow does not mean asking a language model to issue every motor command. Most control should remain inside fast, tested executors." This prevents language-level plans from being treated as motor authority while allowing physical evidence to alter the plan.

Implement Human-Editable, Structured Experience Memory

Instead of absorbing all failures into opaque model weights, maintain an explicit, structured text memory of failures, diagnoses, and repairs. This allows operators to inspect, debug, and edit robot knowledge directly. Appendix F notes: "Because the memory is explicit, humans can inspect and edit it. An operator can correct an overgeneralized rule, forbid a recovery, or mark a tool version incompatible. This editability is difficult to achieve when every experience is absorbed only into distributed parameters."

6. Overlooked Insights

Near-Additive Cost Structure for Capability Expansion

A buried but massive implication for robotics startups is the cost structure of adding new capabilities. While end-to-end models face convexly growing costs due to global regression testing and combinatorial data coverage, TGL's cost grows near-linearly. Appendix G states: "If block scope and validation stay local, each $\kappa_k$ remains bounded. With hierarchical retrieval satisfying $C^{cum}_{retrieve}(K) = O(K)$, cumulative cost is at most linear in $K$. The robot then pays mainly for the missing behavior and its local interfaces." This fundamentally changes the unit economics of deploying general robots.

The "Slow Teacher, Fast Student" Distillation Path

The paper outlines a practical deployment strategy where the slow, agentic route handles novel tasks and generates verified trajectories, which are then distilled into a fast VLA or diffusion policy for routine execution. Appendix H explains: "novel task → few-shot agent solution → verified experience → distilled fast student → routine deployment." This means companies don't have to choose between agentic reasoning and fast VLA execution; they can use the agentic system as a data-generation and verification engine to safely train their fast policies.