The Race to Create General-Purpose Robots | Karol Hausman & Lachy Groom on TBPN
- 01Generalization is Robotics' Defining Challenge
- 02Real-World Data Collection Over Simulation
- 03End-to-End Learning is Already Here and Necessary
1. Key Themes
Generalization is Robotics' Defining Challenge
The breakthrough isn't about what robots can physically do (agility or dexterity), but about generalization - getting robots to work in completely new environments. Physical Intelligence's Pi-05 model represents a paradigm shift where robots can enter unfamiliar homes and perform complex tasks like cleaning kitchens with 50-80% success rates, up from essentially 0% previously.
"The biggest challenge in robotics so far hasn't really been agility or dexterity. What robots can do. Generalization. Similar to what we've seen in language before... The challenge we set for ourselves is to take a robot to a completely new home. It's never seen before and ask you to do a complex long horizon task like clean a bedroom or clean a kitchen." - Karol Hausman [00:01:09]
"We've been quite surprised by how few different environments you need to see to be able to generalize to anyone... We actually got really reassured that this path could really work." - Karol Hausman [00:11:33]
Real-World Data Collection Over Simulation
Unlike self-driving cars where simulation has proven valuable, manipulation tasks require real-world data because the challenge isn't modeling the robot's body but modeling the infinite variety of objects and environments it must interact with. Pi has discovered that diverse real-world data across multiple robot types and environments all contribute to each other's performance.
"It hasn't worked nearly as well for manipulating objects or working with your hands. I think the reason for that is then the difficulty isn't about how do you move your hands. It's more about the world that you're manipulating. That is much harder to simulate... This is the data that is the most important, the data of physical interactions. This is the data that is not on the internet." - Karol Hausman [00:08:18]
"Looking at the past successes of machine learning of AI, it seems that the best successes are where you take real world data and large diversity of that data and learn directly on that. You don't try to find some kind of proxy or some kind of simulated environment... You just go after the problem head on and that's what we're doing here." - Karol Hausman [00:09:17]
End-to-End Learning is Already Here and Necessary
Pi's demonstrations are already fully end-to-end - taking camera inputs and directly outputting actions. This isn't just about being on a "scaling law," but because programmatic approaches have fundamentally failed after decades of attempts. The world is too complex for if-statements and prescribed logic.
"End-to-end robotics is already here. Everything we've shown so far is fully entilent where you take camera input in and view other sensors and output actions directly... I think there is another reason to do end-to-end learning, which is this is something the only thing that has a chance of working. If there was a way to just pre-program your robot and write a really good C++ code to do all kinds of different things like folding laundry, we would have done it a long time ago." - Karol Hausman [00:16:29]
2. Contrarian Perspectives
Robotics Will Eventually Generate More Training Data Than the Internet
Most assume language models will always be the primary driver of AI compute and data needs. However, robots can generate unlimited real-world interaction data continuously, unlike the finite internet. This could fundamentally shift where most model training occurs.
"I think that's one thing that I realized since starting the company is that robots generate a ton of data and you don't need that many to generate data that is close to the levels that LLAM companies use for their models and there is no ceiling to it... I think over time it's quite likely that places are going to switch a little bit where most of the models including LLAMs and BLM are going to be using real-world data collected through robots because that's the data that has no ceiling and it's very active as opposed to just pass observations of what people wrote on the internet." - Karol Hausman [00:22:18]
Alignment Tax Should Drive All Hiring Decisions
Rather than hiring for diversity of thought or bringing in "fresh perspectives," Pi explicitly optimizes for zero "alignment tax" - only hiring people already completely aligned with their mission who have dedicated their lives to robotics. This allows them to move much faster with minimal communication overhead.
"We talk about the alignment tax. When we're bringing someone on, how much work is there to align them around our mission, our way of seeing the world and almost everyone that joins, there's like basically nothing... That just allows us to move so much quicker. It's like we need to communicate a fraction of what the average company needs to communicate to someone. We don't really need to inspire people or motivate them because they're so inspired and so self-motivated." - Speaker 03 [00:19:34]
Research Labs Should Operate Like Startups, Not Academia
Pi intentionally doesn't operate like a traditional research environment. They have clear goals, move extremely quickly, and don't follow typical academic research patterns - yet they're solving fundamental research problems that require breakthroughs.
"I'd never worked in a research environment. I don't have priors. I don't know what a research environment actually looks like... we feel like every other company I've worked with that moves extremely quickly and has a clear set of goals and direction... I've never seen more alignment than I've seen in Pi." - Speaker 03 [00:18:41]
This is a 15+ Year Problem, Not a Tomorrow Problem
While Twitter shows endless impressive robot demos, Pi explicitly warns investors that robotics won't be solved quickly - drawing parallels to self-driving's 15-year arc. They view their competition as "science itself" rather than other companies.
"There are all these self-driving companies that have died over the past 15 years and one thing that we actually like to remind people is that this is not coming tomorrow... There's fundamental research breakthroughs that that we need to make and much like self-driving how to it's what like a 15-year arc at this point there is a very high likelihood that robotics is the same way... we think our greatest competition and science itself it's not like this company or that company." - Speaker 03 [00:28:25]
3. Companies Identified
Physical Intelligence (Pi)
Building foundation models that can control any robot to perform any task, with recent Pi-05 demo showing robots successfully cleaning kitchens and making beds in completely new homes.
"We want to build a model that can control any robot to do any task... The challenge we set for ourselves is to take a robot to a completely new home. It's never seen before and ask you to do a complex long horizon task like clean a bedroom or clean a kitchen." - Karol Hausman [00:00:27]
Waymo
Referenced as having fewer cars on the road than Tesla but achieving superior autonomous driving performance, demonstrating that passive data collection advantage doesn't guarantee best results.
"Tesla has this incredible advantage with how much data they're collecting and passively yet Waymo is so much better so far and it has so many fewer cars on the roads." - Speaker 03 [00:27:26]
Google (ARM Farm Project)
Pioneered early robotic learning with static robotic arms practicing grasping in bins of diverse objects, proving learning approaches work but revealing they were too slow without pre-existing knowledge.
"The idea of the ARM farm was to set up many different stations where you have static robots, static robotic arms where they just practice and they learn from experience... On one hand, it was a big success because of that. It was clear that this learning approach is something that can truly work... On the other hand, it was also disappointing in that it took really long time." - Karol Hausman [00:13:21]
4. Operating Insights
Scale What Shows Returns, Be Flexible on Approach
Rather than committing to one data collection or deployment strategy, Pi will scale whatever shows benefits - whether producing more robots, expanding teleoperation teams, or commercial deployment. They maintain optionality across all possible paths.
"All of the above. Basically where we find there's benefits to scale at this stage, we'll scale it. We'll figure out a way and whether that's producing more robots, giving them to people, whether it's scaling up our operations team and the folks that tele-operate these robots ourselves, whether it's going on commercially deploying these into environments where they're doing economically useful or viable costs." - Speaker 03 [00:10:42]
Algorithmic Improvement Can Reduce Data Requirements
Don't just scale data collection blindly - there's massive value in algorithmic development that makes existing data more useful and reduces the necessity of scale.
"We also think there's so much though to do on just the algorithmic development that can make the data far more useful, that can reduce the necessities of scale, but we're structured such that we can go into every avenue." - Speaker 03 [00:11:03]
Build on Open Source VLMs as Multi-Modal Foundation
Pi's models are built on top of vision-language models, creating truly multi-modal systems where robot actions are just "another language" the model speaks alongside text and vision.
"The models we've been releasing they're actually built on top of vision language models so the models that are truly what they call multi-model where you can talk to it you can ask it what they see in the image and every now and then you can ask it to perform actions too... robot actions is just like yet another language that these models can speak." - Karol Hausman [00:26:02]
5. Overlooked Insights
Consumer Overnight Operations as Reliability Workaround
A barely mentioned but potentially transformative insight: consumer robots don't need 98% reliability if they operate overnight while you sleep. Tasks like laundry folding, meal prep, and house tidying can all happen during hours when failures don't matter and humans can review results in the morning.
"I think the other cool thing about the home and consumer use case is there's so much that could also just happen overnight. There's like while you're sleeping, your laundry is folded, your meals are cooked for like the you know, they're prepped for the week ahead, your house is tidied." - Speaker 03 [00:04:34]
This completely reframes the reliability problem - instead of needing near-perfect performance for real-time tasks, robots can attempt tasks with human review cycles built into natural sleep patterns. This could accelerate consumer adoption by years.
Tariffs Creating Forcing Function for US Robotics Supply Chain
The tariff situation, while initially seeming negative, is actually arriving at the perfect time - before robotics scales to mass production. It's creating urgency to build domestic actuator manufacturers and supply chains while companies are still in R&D phase, potentially giving the US a long-term structural advantage.
"The good thing is, I mean, good and bad. It's also subscale right now. Most of the money is being spent on R&D rather than scale production... I think the good thing is that given its subscale, there's a lot of time to build out US supply chains and it's putting a lot of focus on figuring out can we get US actuators? Can we start to create companies that are developing those?" - Speaker 03 [00:20:47]
This timing accident could force the creation of an entire domestic robotics manufacturing ecosystem before anyone has locked in overseas dependencies at scale.