Fei Fei Li: The Race to Build World Models For AI

1. Key Themes

New View Prediction as a Novel AI Primitive

The core technical thesis of the episode is that World Labs has identified a new foundational primitive for AI training — distinct from next-token prediction (LLMs) or next-frame prediction (video models). Fei-Fei Li frames this explicitly: "We know LLMs are built on next token prediction. We've seen video models as being built on next frame prediction. Atlas is really new view prediction" 00:00:10. This primitive takes a "spatial context" of views and lets a virtual camera be pointed at any point in space-time to generate what the world should look like there.

Unifying Generation and Reconstruction — A Half-Century-Old Field Split

Fei-Fei Li stresses that computer vision has historically split into separate subfields (generation, recognition, 3D reconstruction), each with its own conferences and tracks. Atlas is notable for merging generation and reconstruction into one model. "Sitting here, having been in this field for decades, I cannot tell you how many PhD theses have been written on the problem of reconstruction or novel view synthesis... This is an elegant model that combines or unifies the problem of reconstruction and generation by anchoring on viewpoints and the viewpoint estimation. And that's just incredibly powerful" 00:07:52.

Dense-to-Sparse Reconstruction Is a 50-100x Efficiency Unlock

Justin Johnson describes how traditional 3D reconstruction required "dense" capture — hundreds of photos to cover every crevice of a space — making it impractical for casual or commercial use. Atlas collapses this to a handful of images. "I want like 100, 200, 300 photos of this room to capture it. And what we're trying to do is bring that down to like three... We're saying like 50, 100x reduction" 00:15:52. This reframes 3D capture from a specialist/scientific-imaging task into something consumer-accessible.

Scaling Law Conviction Preceded Proof

The founders describe genuine uncertainty going into training — there was no established scaling law for spatial intelligence when the company started, yet all three co-founders held strong conviction it would work. "There was no scaling law of spatial intelligence" 00:13:00, and later: "I think three of us have total conviction about the scaling law" 00:21:40. Fei-Fei Li adds nuance: she was convinced it would work eventually, but not that it would work this fast: "I was not sure it was going to work this well at this fast" 00:22:23.

Compute, Not Data or Architecture, Is the Current Bottleneck

Justin Johnson states plainly that Atlas is nowhere near its scaling ceiling. "No, no, we're at the beginning" 00:22:54, and "we're basically limited by compute at this point... each time we made the model bigger and each time we trained it for longer, each time we put it on more chips, like it got significantly better" 00:22:56. The release timing itself was compute-constrained: "the thing that was limiting it was not the scale or the data or anything like that. It was literally like we had a deadline of when we wanted to release this thing" 00:23:26.

Robotics' Core Bottleneck Is Data, Not Chips (Yet)

Fei-Fei Li identifies data — specifically the difficulty of dense real-world capture and scenario randomization — as robotics' central constraint today, positioning Atlas (via the Cinex acquisition) as the solution. "We should zoom out and recognize the biggest problem right now in robotics is actually data. One day it'll be chips. But for now it's data" 00:30:43.

Dynamics (Time/Motion) Were Deliberately Deferred, Not Missing

Contrary to external feedback that the model lacks motion, the founders reveal dynamics are already latently present in Atlas's pre-training — the current release simply emphasized spatial over temporal post-training. "The pre-training checkpoint already has a lot of latent dynamics in it... this is something we're going to improve quite a lot going forward" 00:37:52.

The "AI-Completeness" of Spatial Intelligence

Drawing an analogy to Turing completeness, Fei-Fei Li and Ben Mildenhall argue new view prediction (like next-token prediction) is "AI complete" — a primitive general enough to encode any intelligence task if solved fully. Fei-Fei Li: "This primitive... if I could solve this AI task in its full, broadest generality, it would solve any intelligence problem... new view prediction, this primitive that we have in Atlas, especially generative new view prediction, this is also AI complete" 00:41:42. Fei-Fei Li adds an evolutionary framing: "new viewpoint prediction is exactly evolution had to solve by making animals move... Nature gave animals eyes... But nature didn't give trees eyes. Why? Because when you move, you see a new viewpoint" 00:42:44.

2. Contrarian Perspectives

Going Direct-to-3D Was Not the Obvious Winning Bet

Justin Johnson, despite having spent his career on NeRF-style dense reconstruction, admits the field genuinely didn't know whether synthesizing multiple views first (then building 3D) or going direct-to-3D would win: "There's been a lot of uncertainty in the field around like which of those approaches kind of will win out" 00:13:34. World Labs bet on the multi-view synthesis path rather than direct 3D generation — a contrarian choice relative to much of the field's direct-3D efforts.

Marble's Gaussian Splat Output Was a Bottleneck, Not a Feature

Rather than doubling down on their first product's output format, the team concluded Gaussian splats — despite being useful and efficient — actually constrained the model's generality, prompting a full architecture rethink for Atlas. "That was kind of a bottleneck in the previous marble model... the fundamental primitive is not like, could generate a Gaussian splat world. The fundamental primitive is... new view prediction" 00:11:44.

More Views Isn't Actually the Answer — Generation Is Required Even With Perfect Capture

The intuitive assumption is that enough photographs solve reconstruction. Fei-Fei Li argues this is fundamentally false — even a world-class capture effort will always miss spots (under microphones, between chair legs), meaning generation/imagination is mathematically necessary, not just a nice-to-have: "Even the world expert on doing these dense captures is still going to miss some spots... You're always going to miss something no matter how many views you get" 00:18:01.

Exposing the Model to MORE Dynamics Produces Better Static Output

Counterintuitively, the team found that to get high-quality static/frozen-time reconstructions (like the bullet-time effect), the model needed to be trained on dynamic data, not scrubbed static data. "Even in the case where I want static output in the end, the best way to get it is actually expose the model to dynamics... let the model figure out how to factor out the dynamic stuff" 00:36:56. This runs against the instinct to curate purely static training sets for static tasks.

Adding Editability/Control Without Care Turns a Frontier Model Into a Party Trick

Justin Johnson pushes back on the naive product instinct to just add more control knobs to a model. He argues quality must never degrade for control to be added, or the feature becomes worthless: "You've got to add control but not compromise the quality of the model or it just becomes a party trick, basically... No one is going to seriously think about swapping their cutting-edge frontier video model usage for your model if you give them extra knobs but the quality degrades" 00:39:17.

3. Companies Identified

World Labs — The company itself, co-founded by Fei-Fei Li, Justin Johnson, and Ben Mildenhall, building spatial intelligence / world models. Mentioned throughout as having just launched Atlas, described by hosts as "probably the most significant model launch this year" 00:35:25. Two and a half years old.

Atlas (World Labs product) — The newly launched frontier world model combining generation, reconstruction, and simulation via new view prediction. Praised for unifying previously separate CV subfields: "This is an elegant model that combines or unifies the problem of reconstruction and generation" 00:08:18.

Marble (World Labs product) — The prior-generation world model (predecessor to Atlas), focused on generating 3D worlds as Gaussian splats. Still powers World Labs' current product; described as useful for creatives but architecturally limited/static: "Marble had this kind of fundamental blocker of like you couldn't really jam more than honestly like a couple images in" 00:19:12.

Cinex — A robotics company acquired by World Labs (folded into its robotics team), whose core technology was real-to-sim-to-real transfer for robotic policy training. Mentioned as the origin of World Labs' robotics roadmap: "We acquired this company that was formerly known as Cinex. And what is their key technology? Right now their key technology is a system that goes from real to sim and then sim to real" 00:29:25.

4. People Identified

Fei-Fei Li — Co-founder of World Labs, longtime computer vision academic ("having been in this field for decades" 00:07:52). Identified for her foundational framing of spatial intelligence and AI-completeness thesis, and for driving the acquisition/robotics strategy. Credited with total conviction on the scaling law hypothesis from the company's earliest, most uncertain days.

Justin Johnson — Co-founder of World Labs, leads model training/scaling efforts ("Justin's team needs a lot of chips" 00:11:12). Identified for his deep technical judgment on the generation-vs-reconstruction tradeoff and for his product philosophy around control/editability without quality compromise. Personally tested Atlas against his own historical failed 3D captures with strong results: "I've taken a bunch of my own old captures that never worked before and then put them through the system... imagine angles that I never would have gotten from a traditional kind of like Nerf or Splat type reconstruction" 00:16:41.

Ben Mildenhall — Co-founder of World Labs, inventor of NeRF (Neural Radiance Fields), described by Martin Casado as bringing deep 3D reconstruction expertise to the team ("being the creator of Nerf and doing a lot of 3D and reconstruction" 00:13:18). Central to validating that Atlas's outputs were genuinely 3D-consistent rather than merely visually plausible.

Ilya (Sutskever, implied) — Referenced by Fei-Fei Li for the canonical "mystery novel" thought experiment illustrating why next-token prediction is AI-complete: "there's the classic example, I think, from Ilya, where... the final sentence of the mystery novel is, like, and the killer was, predict the next token" 00:41:42.

5. Operating Insights

Build Conviction Through a Scaling Ladder Before Committing Big Compute

Rather than betting everything on an untested architecture at full scale, the team deliberately ran a sequence of progressively larger models to validate the approach before committing to the largest run. "You've got to like climb the scaling ladder, right? You've got to like work your way up and like do smaller experiments, do smaller models, like to build your conviction on what's going to work and what's going to scale" 00:12:40.

Ship on a Deadline, Not on "Perfect Scale" — Constrain Scope to Hit Dates

Even with more headroom available, World Labs deliberately capped model size/training time to hit a release date rather than chasing marginal gains indefinitely: "It was literally like we had a deadline of when we wanted to release this thing and therefore we backed up what we could afford to train in order for that deadline" 00:23:26. This is a disciplined operating choice — treating shipping cadence as a hard constraint against open-ended scaling temptation.

Watch for the "Overnight Slack" Signal — Small-Scale Emergent Results De-Risk Big Bets

The team describes a specific moment — a smaller, non-final model unexpectedly producing a coherent fly-through under a garden table — that instantly converted the founders' uncertainty into full commitment. "That morning, the three of us looked at each other in the eyes and said, that's it. This is, we're going to build this. Like we made a decision within five seconds" 00:24:29. The operating lesson: define in advance what "signal" result would justify full resourcing, so the org can move decisively when it appears.

Treat Product Telemetry From Your Weaker Product as the Spec for Your Next Model

Instead of guessing what customers wanted, World Labs observed how users were actually circumventing Marble's limitations (screenshotting the 3D splat from different angles) and used that literal workaround behavior as the design brief for Atlas's core capability. "People would take that product, put an image in, get a full 3D scene as a Gaussian splat, take a couple screenshots of it from different points of view and leave... we can just make those images... that's data" 00:25:07.

6. Overlooked Insights

The Real Bottleneck in 3D/Design Workflows Is the "Translation Tax," Not Creativity

Buried in a discussion of use cases, Justin Johnson makes an offhand but economically significant observation: the most labor-intensive part of physical/architectural design isn't ideation but re-encoding human feedback into 3D software. "Taking feedback on a 3D design from kind of like verbal commentary or sketches... mapping that back into the 3D representation is like 95% of the work, right? You can have a meeting, get feedback, and then you go back and do a week of revisions" 00:28:03. This reframes the addressable market for World Labs-style tools away from "content generation" and toward eliminating a specific, quantifiable translation-labor tax that exists across architecture, construction, events (booth-building was mentioned), and game design — a much larger and more durable enterprise wedge than the creative/hobbyist use case that gets most of the attention.

Context-Length Scaling Has Never Been Seriously Applied to Visual Models — And That Gap Is the Opportunity

Justin Johnson notes that the LLM world fought pitched battles over context length (128k→1M tokens) and built a cultural intuition for its value, but "no one has pushed that at all on the image and video model side in the same kind of principled way... no one's out there trying to like put like an hour long video through and do a needle in a haystack retrieval" 00:18:47. This is a subtle but important point: it implies that visual/spatial models are roughly where language models were before the context-length race began, suggesting a whole class of scaling gains (and products built on "long-context vision") remains untapped — a potential investment/research frontier that the conversation doesn't dwell on but which follows directly from Atlas's "spatial context" mechanism already ingesting up to 100 frames.