Zhang Xiaojun | 张小珺Jùn｜商业访谈录 Summary

1. Key Themes

The Genealogy of Modern Robot Learning: Two Warring Schools

The episode traces a detailed intellectual lineage of robotics — a "traditional" school rooted in Carnegie Mellon (planning, control, physics modeling) and a "machine learning" school rooted in Berkeley/Stanford. K maps the family tree explicitly: Matt Mason → Siyuan Feng (locomotion) and K's own advisor (path planning); Andrew Ng → Peter Abbeel → Sergey Levine (RL for robots) → K herself. The tension between these schools shaped careers and bets.

"My advisor is from the CMU lineage, doing path planning... he emphasized everyone should be a full-stack person. You need to know how to build robots, do dynamics analysis, path planning, control. But in his lab, I was the only one doing machine learning-style approaches." 00:47:46

Pi's Three-Paper Research Arc: Capability → Generalization → Performance

K presents Pi's main publications as a deliberate sequence of answering the field's hardest questions, one at a time.

"The first paper is Pi0, its keyword should be capability. The second paper is Pi0.5, its keyword is generalization. And the most recent one is Pi0.6-star, its keyword is performance." 00:55:21

"Pi0.6 — I also participated very heavily in this project because we are the reinforcement learning team, this was our first published work... if an agent can roughly accomplish a task, you go collect its experience data, let it collect experience in the real world, put that data back into its training pool, and it can actually achieve better results than before from those fixed collected datasets." 02:00:07

Reinforcement Learning as the Path Beyond Imitation

K draws a sharp distinction between imitation learning (copying human demonstrations) and reinforcement learning (self-directed improvement through experience), arguing that only RL can surpass human-level performance.

"Imitation learning has an innate hope for generalization... But as I went deeper into imitation learning research, I felt unsatisfied. If you're always just copying others, you can't innovate, can't have breakthroughs. So I moved into reinforcement learning — RL emphasizes how through your own exploration, you can break through original performance and push the ceiling higher." 00:46:27

"I want to give an extreme example: our Pi0.6 star project, in the folding task — the robot's performance clearly surpassed even the best data collectors as the starting point." 02:11:11

Real-World Data vs. Simulation: A Principled Bet

K takes a clear stance that real machine (真机) data is currently irreplaceable, while acknowledging the appeal of simulation — and explains exactly why simulation falls short for manipulation.

"If you want to do anything related to soft, flexible things, clothing, friction, viscosity — those complex physical properties — you actually might not even be able to build such a simulator. So you can't even talk about that data." 02:05:35

"I am a believer in real machine data. But I personally believe that when robots are cheap and deployed widely, much of the concern about data today won't matter — because you can let so many robots collect data during deployment, and all of it serves you." 02:01:34

The "Experience Data" Insight: Robots Teaching Themselves

Pi0.6-star introduced the concept of using a robot's own rollout data — its autonomous attempts, including failures — to improve the model, dramatically reducing dependence on expensive human teleoperation.

"Previously when collecting real machine data, there's a robot arm, and you let it do things via teleoperation. But it can also go do things on its own, collect all that data. If you can replace the teleoperation source with an already-trained large model letting it run, the cost of data should drop a lot." 02:02:05

Pi's Philosophy: Pure Research First, Commercialization Later

K reveals that Pi deliberately avoids early commercialization, learning from a cautionary tale in the field about distraction.

"Peter Abbeel founded a robotics startup called Covariant in 2015 or 2016, saying they'd do general machine learning for robots. But at some point they went deep into logistics and warehousing... From the perspective of developing a large model, looking back, the early commercialization was a distraction. You lose the general, generalizable nature, and instead put energy into many commercialization-related things without really pursuing the root problem. So Pi, I think, was influenced by this experience — really emphasizing: we don't want to think too much about commercialization." 03:17:46

China's Hardware Dominance vs. America's Software Edge

K gives a frank, non-diplomatic assessment of the competitive landscape.

"I think China's industrial chain and manufacturing advantages are truly enormous. It's hard to imagine a robot company assembling a robot where not a single component is Chinese — that's nearly impossible... China having this supply chain manufacturing advantage — the iteration speed [for robots] will be different... I don't even know how America would catch up if they tried." 03:16:18

"China companies may emphasize pragmatism and commercial return more. Pi is quite unique — for now we should completely not consider commercialization, because that might be a distraction." 03:18:45

Claude Code and the Transformation of Human Collaboration

K makes a quietly profound observation about how AI coding tools are restructuring not just productivity but human relationships at work.

"Before I would go ask the person responsible for a certain module... why not just ask Claude Code directly? I can replace what used to require human communication with an AI agent interaction... even including our many current work processes — one person controlling three or four agents, letting the agents do a lot of the work. This has somewhat changed the human-to-human relationships in team collaboration." 00:07:31

"I think it should be about three or four times [productivity improvement]. But research is constantly firing a shot and seeing where the bullet lands, then adjusting." 02:31:37

The Unique Research Culture at Pi: Academic Openness in a Startup

K describes Pi as unusually open for an industry lab — holding reading groups, allowing office visits from peers, publishing frequently — in stark contrast to Tesla, Google DeepMind, and others.

"Pi should be considered a fairly unique place — we still have some academic influence, feeling that sharing and publishing is part of your research. Other companies, from my current sense, are not this open... Pi's office doesn't even have that strict security. Sometimes you can bring industry friends to come take a look, introduce things, have this collision of ideas." 01:41:41

2. Contrarian Perspectives

The Humanoid Robot Form Factor Is Overrated — For Now

K explicitly states she would not have joined Pi if they were building humanoids, and argues that non-humanoid robots can accomplish harder tasks faster.

"We don't do humanoid. Because if they said they were doing humanoid, I wouldn't have come — you know? I'm very worried that different backgrounds of people going to do humanoid... my research focus is doing better tasks. I personally firmly believe: you don't need humanoid to do better tasks. And from Pi's perspective, you can make it faster without humanoid. Look — folding clothes, making coffee — we did all of these without humanoid first." 02:33:03

Simulation Data Is Currently Useless for Real Manipulation Tasks

Against the prevailing hype around sim-to-real transfer, K argues the gap remains unbridgeable for the tasks that actually matter.

"Folding clothes in simulation — this should still be a semi-frontier problem. Semi-frontier because I haven't seen a single company that can do it very well — train well in simulation, add a tiny bit of real machine data, and perform well. We're not there. And if you want to do anything related to soft, flexible, clothing, friction, viscosity... you might not even be able to build such a simulator." 02:05:06

The Robot Field Has No Meaningful "Leaderboard" — Making Frontier Claims Unreliable

K argues that without standardized evaluation, every company's claims of frontier progress are essentially unverifiable and incomparable.

"The evaluation problem directly leads to an unclear sense of frontier. Unlike other fields that have a leaderboard — everyone runs on this huge dataset, it's a race, you can clearly see who's first. In robotics, talking about frontier is hard because evaluation is so difficult... each company internally has their own evaluation system, and these priorities aren't even the same." 01:48:50

Reward Function Design Is the Wrong Frame — Task Communication Is the Real Problem

K reframes the classic RL debate away from reward engineering toward a broader question of how to communicate intent to an agent.

"In my view, it's not a problem of writing a reward function — it's a problem of communicating to the agent what you want it to do. The reward function is more of a representation method... In natural language RL, the most useful thing was verifiable tasks. Verifiable tasks can exist independently of a so-called reward function. I believe conveying good and bad doesn't necessarily require a reward function." 02:53:00

Early Commercialization Kills General Robotics Research

Drawing on the Covariant example, K argues that going to market too early is structurally incompatible with building a general foundation model.

"Because of early commercialization, they lost the general, generalizable nature — instead putting energy into many commercialization-related things without really pursuing the root problems... Pi, I think, was influenced by this experience." 03:17:46

3. Companies Identified

Physical Intelligence (Pi / π)

Research-first robotics company in San Francisco focused on building a universal "robot brain" — a foundation model that works across different hardware morphologies. Founded by Sergey Levine, Chelsea Finn, and Karl Hausman ~March 2024. Valued at over $5 billion within two years.

"Pi's model is trained on data from different morphologies and can be used on different robot hardware... The human controls the same brain to operate many different morphologies — whether driving a car, operating an excavator, or controlling a robot arm. I think that's the most essential definition of a universal brain." 01:45:54

Skild AI

Founded by Deepak Pathak and Abhinav Gupta (both CMU), described as a peer "brain company" to Pi, focusing more on locomotion and humanoid/quadruped morphologies alongside manipulation.

"Skild is also hardware-agnostic, but they have more exploration of legged humanoids and quadrupeds. Both Pi and Skild have pretty similar slogans — they want to make a universal foundation model brain for embodied intelligence." 01:28:06

Figure

Humanoid robot company founded ~2022 by a non-technical serial entrepreneur. K highlights the founder's bet on a field he didn't come from as unusually interesting.

"Figure's founder, as far as I know, doesn't have a technical background — he has previous successful entrepreneurial experience. I think he's a very interesting person because he could in 2022 commit to a new field he wasn't that familiar with because he believed in it." 01:29:54

1X (OneX)

Humanoid company distinguished by using Series Elastic Actuators rather than rigid electric motors.

"1X's founder, I've heard, has deeply cultivated this robot foundation... they do Series Elastic Actuator humanoids — relatively unique." 01:30:23

Dyna (Dynabot/referenced as Dana)

Described as more commercially deployment-focused than Pi or Skild.

"If I were to use one label to describe them, they more emphasize commercial deployment and landing. You can see their folding clothes demo... they might be among the earlier ones to move toward the deployment side." 01:30:37

Boston Dynamics

Cited as the pinnacle of traditional control-based robotics; founder Marc Raibert highlighted for his contrarian fashion choice as a form of intellectual rebellion.

"He made the first robot in the world that could [do backflips] at high speed... influenced a whole generation of scholars." 00:57:33

Tesla (Optimus)

Identified as the most aggressive humanoid bet, with strong hardware credibility from automotive manufacturing.

"Tesla is probably the most aggressive company betting on humanoid... They have a very strong humanoid belief... and they have so much money to burn on robot hardware R&D." 01:39:39

Covariant

Peter Abbeel's robotics startup, now focused on warehouse/logistics. Used as a cautionary tale about premature commercialization distracting from general model research.

"Peter Abbeel founded Covariant in 2015 or 2016... at some point they went deep into logistics and warehousing. From the perspective of developing a large model, looking back, the early commercialization was a distraction." 03:17:46

Generalist AI

Startup using Yumi-style data collection (wearable gripper for human-gathered robot data), leaning toward industrial deployment.

"Generalist AI would be slightly more biased toward industrial scenario applications — inferred from their released videos." 01:33:10

Sunday Robotics

Startup also using Yumi-style data collection, distinguished by cuter robot designs and home-use focus.

"Sunday's released videos show more home-use scenario representation — even their robot is quite cute — currently the cutest robot I've seen." 01:33:20

NVIDIA (Gear Lab)

Running a robotics research lab led by Fei-Fei Li's former students Zhuang Chu (Zhuoyu) and Jim Fan (Linxi Fan).

"Fei-Fei Li brought many students who have also devoted themselves to robotics research — like Zhuoyu and Jim Fan, both Fei-Fei's students — they're leading NVIDIA's Gear Lab, overseeing how to do robotics at NVIDIA." 01:03:45

Google DeepMind (Robotics)

Cited as leveraging its Gemini multimodal model for robotics, treating the robot body as another modality.

"Google, because they have their own large model Gemini... maybe for them, robotics can be a part of this modality, providing spatial perception, spatial planning, or spatial control capabilities. I think when they pursue the robotics path, they're more thinking: how does robotics become part of this multimodal large model." 01:40:20

4. People Identified

Sergey Levine

Co-founder of Pi; former Berkeley professor. Described as a prolific storyteller, legendary paper polisher ("Sergey GPT"), and heavy science fiction reader who cites sci-fi in nearly every paper.

"Sergey is a heavy science fiction addict. If you read Pi's papers, many have a quote from a sci-fi novel — all written by Sergey. He's a professor who tells stories extremely well. And internally, the joke is that before Claude GPT appeared, Berkeley people already had Sergey GPT — send him your draft, and he gives back a super polished version, fast and excellent." 01:01:47

Chelsea Finn

Co-founder of Pi; former Berkeley professor. Distinguished by extreme discipline (wakes at 4am, swims daily) and an almost animalistic physical intuition about robot motion and data quality.

"Chelsea is extremely disciplined — every day he wakes at 4am and goes swimming. He has a very well-planned life, very inspiring. And he's a person with animal-like intuition — talking about robot-specific motions and performance, he has very fast intuitive reasoning. You can feel he has very deep understanding of these motion tasks." 01:02:17

"Chelsea was responsible for Pi's data collection for a period — he has very deep understanding of what kind of data to collect. Many of Pi0's tasks were pushed forward by Chelsea because he believed they could be done when others weren't sure." 01:02:39

Karl Hausman

CEO of Pi. Handles company infrastructure and operations, enabling the research team to focus purely on research. K describes him as the Sam Altman analog to Sergey's research leadership.

"Karl manages many of the company's infrastructure things — you can think of our company having a very large portion of work that is research, but also a very large portion that is ensuring this research can be carried out efficiently and quickly. Karl is mainly responsible for this." 02:32:26

Peter Abbeel

Berkeley professor; Sergey Levine's advisor; AI pioneer who brought machine learning to robotics. Also founder of Covariant.

"Peter Abbeel should be one of the earliest pioneer professors to apply machine learning to robotics. He's now at Berkeley and also simultaneously leads Amazon's research division... He and Sergey, you could say, are the people who really developed and spread reinforcement learning in machine learning for robotics." 00:59:01

Matt Mason

Described as K's "ancestral teacher" in the robotics lineage; CMU manipulation professor. Known for the insight that dexterity lives in the brain, not the hand.

"His most famous statement is: we currently over-emphasize dexterity in the hardware fingers and joint construction, but he thinks the key to dexterity is not in the hand itself but in how the brain controls even a simple structure to do complex tasks. I think my research has been influenced a lot by him." 00:56:04

Mark Raibert

Founder of Boston Dynamics; famous for the Hawaiian shirt rebellion against academic formality.

"He's an older man with white hair, always appearing in public in Hawaiian shirts. The story goes that in his youth, when he wore a relaxed shirt to a conference, someone criticized him. From that point on, he said 'I'm going to wear this' and promote it. He has a rebellious spirit." 01:36:14

Deepak Pathak

CMU professor; known for Curiosity-driven reinforcement learning; co-founded Skild AI with Abhinav Gupta.

"Deepak is more classified as one of the famous scholars doing reinforcement learning... he and Abhinav now jointly founded Skild, also a very influential robotics company in Silicon Valley, also doing embodied intelligence brain." 01:06:02

Abhinav Gupta

CMU professor; visionary, wildly creative researcher who pioneered using cheap wearable tools for robot data collection — a forerunner of Yumi. Co-founded Skild.

"He's a person with very creative thinking — you cannot predict what he'll say next second. His thinking is very jumpy. Very early he wanted to do a self-learning robot... He said let's buy something for $5 on Amazon — a long stick with a small gripper at the end — collect data with this. That has an eerily similar concept to today's Yumi." 01:04:45

Siyuan Feng

CMU professor; former schoolmate of K; published Omni-Target; expert in legged locomotion.

"Siyuan Feng is a CMU professor. When he was at my school before, we became friends. I learned a lot about locomotion from him. Their group has published Omni-Target." 03:22:32

Abhishek Gupta (distinct from Abhinav)

Sergey Levine's student who came to K's university (University of Washington) and became K's "junior advisor," connecting her to Pi.

"Abhishek came to our school and became my junior advisor... our collaboration was very close. He would say 'I have this idea,' I'd say 'I have this idea,' we'd mix them together... Because Abhishek is Sergey's student, when Sergey was co-founding Pi, Abhishek may have told him about me — this person who is graduating, doing this." 02:36:12

Shuran Song (宋书然)

Female robotics scholar at Columbia/Stanford; her lab produced Diffusion Policy, a landmark work for robot learning.

"Shuran Song is a female scholar I deeply respect. Her group produced Diffusion Policy — that came from Shuran's group. And ACT and Aloha came from Chelsea's group. I think both are milestone papers that profoundly influenced the current form of machine learning for robotics." 01:03:45

Li Bo (李博)

K's first mentor; went on to become a professor at University of Chicago; now doing a startup in Silicon Valley. She selected K based on K's background in game theory for adversarial machine learning.

"My first academic mentor, Li Bo — she graduated and went to University of Chicago as a professor, now also doing a startup in Silicon Valley. She chose me — I guess maybe because I was studying economics and liked game theory, while her research topic was Adversarial Machine Learning, which needs adversarial game-theoretic thinking." 00:42:40

Jitendra Malik

Berkeley computer vision professor; famous for the quote that sparked cross-field robotics invasion.

"'Robotics is far too important to be left for roboticists.' This has many interpretations — one not-so-friendly one is: roboticists can't do it, we need to come do it. But a friendlier one: to do robotics, you need people from different fields working together. I think this has instructive meaning for today's robot foundation model development." 01:23:42

Andrew Ng (吴恩达)

Cited as the entry-point teacher for K's generation of ML researchers through his online courses.

"When I started learning, there weren't many ML textbooks — we all watched Sergey's videos and Andrew Ng's online courses. Andrew Ng's online courses were basically everyone's entry-level ML textbook when I was in university." 00:58:30

5. Operating Insights

Tackle the Hardest Version of a Problem First — Then Generalize

K deliberately chose chopsticks as her robot's end-effector precisely because it's the hardest possible manipulation tool. Her logic: if your algorithm works in the hardest case, generalization to easier cases follows naturally.

"I said: if the algorithm can succeed on chopsticks, it proves that on other methods it should be even easier to succeed. If you're going to do something, do the hardest version to succeed. Then what's left is probably just engineering problems." 01:16:04

Use Internal "Olympics" Events to Push the Boundary of What's Possible

Pi runs internal competitions where researchers and data collectors team up to attempt tasks that seem impossible — using teleoperation to find the frontier, then training models to replicate it.

"We'd periodically let researchers and data collection staff form teams to play an Olympics — see who can operate a robot to do very difficult, unimaginable tasks. Since it's humans operating, you can push the boundary enormously. What a human can complete, can a robot complete? This is a very good activity." 02:13:47

Before Asking a Colleague, Ask the AI Agent First

K has restructured her entire workflow around Claude Code, reducing human-to-human communication by routing technical questions through AI first — and reports 3-4x productivity gains.

"Before, I would rather go ask the person responsible for that module — explain the situation, ask if it's feasible. Why not just ask Claude Code directly? I can replace what used to require human communication... One person now controls three or four agents and lets the agents do lots of work." 00:07:31

Hold Pre- and Post-Project Reading Groups to Align Team Belief

Pi runs structured reading groups at two points for each research project: once at the start (to align everyone on prior work and generate shared excitement) and once near completion (to get broad feedback before publishing). This spreads capability across the team and surfaces blind spots.

"For Pi0.6-star, there were two reading group sessions. First, to tell everyone what we want to explore, what prior work has done... Then near maturity, another one: here's where we've gotten, what do you think? Then there are more free-form ones — like when Claude Code came out, immediately a reading group: who in our company uses Claude Code best? Get on stage." 02:30:34

6. Overlooked Insights

The Scaling Law for Homes: You May Only Need ~100 Environments to Generalize to New Ones

Buried in K's description of Pi0.5 is a potentially landmark empirical finding: there may be a finite, achievable number of home environments after which performance on new homes plateaus — meaning true generalization to arbitrary homes is tractable, not requiring infinite data.

"At the time, a fairly exciting point was: we collected data in roughly 100 homes. Then you start removing some of these homes and see at what point there's still growth. We felt this scaling was slowing — meaning you don't really need to collect data in every person's home in the world before you can work in their home. There's actually a number, and once you reach it, maybe that's enough." 01:58:21

This is a critical empirical result that the podcast participants did not stop to discuss — but if it holds, it fundamentally changes the data economics of consumer robotics. It implies that home robot generalization is not a "boil the ocean" problem. Any investor evaluating data moats in robotics should probe this finding immediately. It also implies that early movers who hit that ~100-home threshold first may have a durable lead, while latecomers face diminishing returns from additional data collection — a very different competitive dynamic than "more data always wins."

Robots Self-Assembling / Self-Reproducing as the Next Milestone — and K Wanted to Research It

In a single throwaway sentence near the end of the interview, K names robot self-assembly/self-reproduction as what she considers a true civilizational milestone for the field — and reveals she pitched it as a research agenda when exploring faculty positions.

"I've always wanted to do is robots that can build themselves. I think this is a milestone in robot development — because it's like a species with continuity, able to reproduce through self-replication. Robots being able to assemble and build themselves is a form of reproduction. This is a topic I very much want to work on — it was something I said when looking for faculty positions." 03:26:24

This is not currently a funded research agenda at any of the named labs. Yet K — a core Pi researcher and RL specialist — explicitly identifies it as the field's next major frontier. For a deep-tech investor or a lab looking to recruit exceptional researchers, this signals an underexplored research niche that sits at the intersection of robotics, self-replication, and manufacturing automation. The first lab or company to demonstrate even a primitive version of robot-assisted robot assembly could establish a qualitatively new capability category.