Jensen Huang | The a16z Show Summary

1. Key Themes

Model Progress Is a Capital Game: Compute, Data, and Talent

Lambert's organizing frame is that every lab is a bundle of three forms of capital, and the open/closed gap maps directly onto who has the most of each. Execution, meaning making the org chart actually produce a model, is the fourth hidden variable.

"When I think about model progress, it comes down to different forms of capital that you put into them, and that is like compute, data, and talent." — Nathan Lambert 00:02:58

"Making the org chart work for building models is really hard, which is why you have companies have to do complete resets from the bottom up, and then it actually like kind of works sometimes. Like a company like Meta's like totally reset, and then they're like on a positive trajectory again." — Nathan Lambert 00:02:58

Closed Labs Win by Taking the Earliest, Most Expensive Bets, and Each Has a Distinct Culture

Lambert distinguishes OpenAI's "paradigm-creating" culture from Anthropic's "focus on what models will be able to do" culture, and tells aspiring researchers where the frontier actually is.

"If somebody wants to work at the frontier of AI, I mostly tell them to work at Anthropic and OpenAI, which is the easiest bet. I think Meta, Google, SpaceX AI, like, I think you still might be there, but it's harder to see from the outside." — Nathan Lambert 00:03:47

"OpenAI has done things like kickstart the reasoning model revolution. And I think that kind of breakthrough and that beautiful plot with the inference scaling laws is peak OpenAI." — Nathan Lambert 00:04:03

"Anthropic has been like very focused on the types of things the models are going to be able to do. And then their reward was this Claude Code explosion with their focus on coding for so long." — Nathan Lambert 00:04:22

Chinese Open-Weights Are a Commercial Distribution Strategy, Not Altruism or State Directive

The release of open models by Chinese labs is framed as a pragmatic go-to-market shortcut for startups that would otherwise be invisible to Western buyers.

"If they were to build their models and try to sell an API from China, nobody would know about them. Nobody would know about Moonshot AI. Very few people would know about 01.AI, and there's a very long list after this. And it was a very practical decision by them." — Nathan Lambert 00:04:46

"They're just trying to build a startup." — Nathan Lambert 00:05:13

Lisha Li echoes it: "it perpetuated because it was an excellent, excellent commercialization strategy." 00:08:07

Compute, Not Talent, Is the Chinese Labs' Binding Constraint

The talent-parity claim is stated emphatically, with a vivid anecdote on the compute gap.

"To say that China is anything but equal in talent, you're going to have a hard time because they just have so many people being trained at such a high level, and what they don't have is compute." — Nathan Lambert 00:09:19

"You talk to people there, and you're like, 'Oh, I heard an average OpenAI researcher has 5,000 H100 equivalents that they can access at any time,' and like their jaw just falls to the floor." — Nathan Lambert 00:09:38

Distillation Is Real but Overstated as an Explanation for Open-Model Catch-Up

Lambert argues distillation is a compute-substitute, but that the technical literature doesn't support it as the main driver of modern reasoning-model quality.

"Distillation is a way to effectively spend more money to get what is compute equivalent. Like outputs from a language model are a function of compute." — Nathan Lambert 00:09:56

"I'm not sure that it would have actually made our model a lot better." — Nathan Lambert, on whether Claude reasoning traces would have helped him build OLMo 00:11:25

"When I say that distillation helps them by a few months, it's based on the scientific literature that we just like best information we have." — Nathan Lambert 00:13:05

The Frontier Flywheel and Why Open and Closed Run on Different Exponentials

Lisha lays out the capital flywheel (compute, then frontier model, then revenue, then more capital, then IPO), and Lambert explains why open models are structurally slower: information and revenue don't flow back to the model builder.

"Open and closed models are on different timescales of exponentials, and the closed labs are so integrated, where they have efficient inference, they can serve the model at scale when they're ready to... and this has been the story of the year, where the revenue has just gone $0, $100 billion, and likely to grow." — Nathan Lambert 00:19:46

"You have an open-model builder, then you have the open-model inference companies, and you have the open-model post-training companies, and like there's information from those customers... that information doesn't just flow back to the model builder very easily." — Nathan Lambert 00:20:10

Tiered Market Structure: Best Closed Models Command Margin, Near-Frontier Becomes Open

"Over time there will be these tiers of open models where there's the best closed models, which you can demand the best margins for, and then there's the near frontier models, which is a step behind this, and those will be open because it's just like some sort of supply and demand there." — Nathan Lambert 00:06:04

Data and RL Environments Are the Real Progress Engine, and the Supply Chain Is Flawed

Frontier gains in non-verifiable domains come from buying high-quality data and building worlds for models to hill-climb, but the data industry is immature.

"A lot of the frontier progress in domains that are not fully verifiable is by buying data that is high quality, and giving the model worlds to act in, and hill-climb on." — Nathan Lambert 00:25:17

"I personally think that this RL environment explosion where there's just so much opportunity to make money is having a lot of people like cut corners, and this can induce things like reward hacking." — Nathan Lambert 00:28:40

"There are some people that contribute by doing a very narrow thing, which is you make a dataset, and you figure out what the behavior should be for your model in that area..." — Nathan Lambert 00:26:14

Demystifying Recursive Self-Improvement With Open Documentation

Trillium's RSI project is to publicly record how AI is actually being used to improve AI, and where it gets stuck.

"We're going to build an open-source stack for SFT, for mid-training, we're going to use SkyRL, MILES, all these things... we can store the trajectories for what we did, we can tell you what our tasks were, and why we chose them, and whether models work on them or not. And you just do this for a year of a lab." — Nathan Lambert 00:40:01

Fast Takeoff Is Bottlenecked by Exponential Compute for Linear Gains, but Societal Preparedness Is the Real Gap

"All the scaling laws are exponential increase in compute for linear increase in performance... It's hard for me to bet against this." — Nathan Lambert 00:55:28

"The AI capability plot doesn't necessarily worry me, but the AI capability plot in the pretty much non-existing preparation of cross-public cybersecurity and all of these things, like there needs to be more preparation in the assumption that this is coming." — Nathan Lambert 00:57:34

2. Contrarian Perspectives

Distillation Probably Isn't the Main Reason Chinese Labs Are Close

The dominant narrative blames distillation for open-model catch-up. Lambert, who has hands-on experience building open post-training recipes, argues the evidence doesn't support it and that "cope" is a factor.

"All the open model builders are stuck using like a really outdated Qwen model because you can't figure out how to get the teacher right. My hypothesis is that there's some like mid-training and something in the common pre-training data that's open that makes it hard to do this thing." — Nathan Lambert 00:11:09

"I think there's a bit of cope where people are like, 'Only we are good at building models.' And it's like there are so many companies. There's like 20 to 30 companies that are building good models." — Nathan Lambert 00:13:24

He also notes Xiaomi-style week-long open RL runs don't map onto the distillation literature: "where exactly does distillation go into this?" 00:11:56

Crack Down on Distillation Would Do More Harm Than Good

Despite the national-security pressure, Lambert argues policy action is both legally weak and strategically counterproductive, and he aligns with Jensen Huang's framing.

"I don't even think terms of service are a signed contract. It's just like I don't think you can sue them in the same way, you just can cancel their access." — Nathan Lambert 00:14:27

"You won't regulate distillation with policy from Congress. You will probably just do federal action to make their life annoying... you're going to cause more harm than good if you do anything about distillation, because they're not going to stop." — Nathan Lambert 00:15:54

Jensen Huang's clip: "People distill my products every single day. They take it and they strip it down to its raw parts. They test everything. That's called competition." 00:00:21

Closed Labs Don't Have the Secrecy Advantage They Think They Do

Contrary to the "frontier is hidden and pulling away" fear, Lambert argues information leaks and the real moat is serving efficiency.

"The security of these labs in terms of information is not as high as they think. Like Silicon Valley is famous for like ideas traveling, like what is it like one beer or something?" — Nathan Lambert 00:52:47

"What they can win on is putting more inference into their infrastructure, which lets them serve at a lower price and make higher margins." — Nathan Lambert 00:53:14

Closed Labs Are Not Apple: Harness Lock-In Didn't Materialize

Lambert had predicted vertically integrated model+harness would be a durable wall, but was wrong.

"I thought years ago that say the closed labs would be like Apple... But it turned out that so many people have different harnesses for different things that they like to do, that the models companies have been pressurized to make it so that their model can work in all sorts of different harnesses." — Nathan Lambert 00:50:49

Pre-Training Is Now the More Approachable Frontier; Post-Training Is the "Wild West"

Against the intuition that pre-training is the capital-gated moat, Lambert finds it more tractable than post-training.

"Doing controlled, like let's just start pre-training our model, seems a lot more approachable to me than I don't really know much about language models, I'm going to figure out this whole post-training thing, where it's just like the Wild West." — Nathan Lambert 00:49:15

"You don't have to spend $1 billion to get a very good base model... it's not that you need $10 billion to like have a relevant model." — Nathan Lambert 00:45:29, 00:46:18

3. Companies Identified

OpenAI — Frontier closed lab. Mentioned as the originator of the reasoning-model paradigm and a heavy investor in data, including robotics.

"That kind of breakthrough and that beautiful plot with the inference scaling laws is peak OpenAI." — Nathan Lambert 00:04:03 On robotics: "I think this was a cycle of like 6 to 8 months ago, they started hiring people and buying a lot of data in robotics. And then, the model magically 6 months later is really good at it." — Nathan Lambert 00:22:43

Anthropic — Frontier closed lab behind Claude. Praised for focus and coding.

"Their reward was this Claude Code explosion with their focus on coding for so long." — Nathan Lambert 00:04:22 Together with OpenAI: "the most forward-looking and the most willing to spend egregious amounts to make a step towards the direction that people think is impossible." — Nathan Lambert 00:04:46

Meta — Mentioned as having completely reset its AI org and now trending positively, and as a natural open-weights player given complementary revenue.

"A company like Meta's like totally reset, and then they're like on a positive trajectory again." — Nathan Lambert 00:03:30 "Mark wrote the best blog post on this. Like when Llama 3 came out, he wrote like the best like, 'Why open is the path forward.'" — Nathan Lambert 00:47:45

Google — Listed among frontier labs with the most capital.

"The closed frontier labs of OpenAIs, Anthropics, Googles have clearly had the most of these." — Nathan Lambert 00:02:58

SpaceX AI — Named alongside Meta and Google as a place where "you still might be there, but it's harder to see from the outside." 00:03:47

Moonshot AI (Kimi) — Chinese open-model lab, cited as an example of why open release creates visibility. Lambert also notes Kimi was caught serving Claude in its products.

"Nobody would know about Moonshot AI." — Nathan Lambert 00:04:46 "Even cases where Kimi was serving Claude in their products to users." — Nathan Lambert 00:12:30

01.AI — Chinese lab; example of open release as a visibility strategy. 00:04:46

Xiaomi — Cited as running a week-long open RL run and then releasing the model with marketing savvy.

"You have the Xiaomis of the world doing this week-long open RL run where they're like, 'Here's what we did,' and then they drop the model." — Nathan Lambert 00:11:25

Qwen (Alibaba) — The default open teacher/base model that open post-training recipes are stuck on. 00:11:09

GLM / Kimi K3 — Open-source models Lisha says startups use to reach the Pareto frontier. 00:19:22

Decagon and Harvey — Startups Lisha cites as doing heavy fine-tuning on open models.

"I remember from like Decagon and Harvey who is trying to do a lot of fine-tuning. There's like a huge amount of startups doing post-training as a service." — Lisha Li 00:18:53

Cursor (Composer) — Lisha cites Cursor's finding that a planner-plus-fine-tuned executor matched two frontier models at far lower cost.

"Using two frontier models, you know, one for planner, one for execution, is not as good, or is equivalently as good, but far more costly, than just using Composer with maybe Opus... as planner." — Lisha Li 00:24:10

Thinking Machines (Tinker, Inkling) — Praised for transparency about a non-best first model, then compounding via Tinker and a second release.

"They're like, 'This wasn't the best model. We're proud of it.'... now they've released their second model and they're getting going, and they have Tinker." — Nathan Lambert 00:47:01

Prime Intellect — Named as a company enterprises go to for fine-tuning services alongside Thinking Machines. 00:21:13

SSI (Safe Superintelligence) — Cited as the lone example of real information security.

"Which is why SSI is so impressive. Nobody knows what's going on." — Lisha Li 00:53:02

NVIDIA — Referenced for chip improvements expanding compute.

"NVIDIA's chips will get way faster, there's more compute built out." — Nathan Lambert 00:52:31

Ai2 (OLMo) — Lambert's prior lab; built OLMo; its post-training is still among leading open recipes though cycle time lagged. 00:11:49, 00:49:37

Eleuther, Hugging Face, Marin / OpenAthena (Stanford) — Other open labs that could do RSI documentation. 00:40:01

Open RSI — Academic consortium that just launched an open recursive-self-improvement effort. 00:43:47

Megatron — Open-source training stack that many new labs fork and never push upstream. 00:46:00

SkyRL, MILES — Open-source RL/post-training tooling Trillium will use. 00:40:01

Linear — Task-tracking tool mentioned as a way to document RSI task choices. 00:40:44

Trillium — Lambert's new non-profit open research lab, named for the spring ephemeral flower that blooms before the canopy leafs out.

"Trillium is a spring ephemeral flower, which it blooms before the canopy has grown its leaves... they're like essential for spreading nutrients through the cycle of growth." — Nathan Lambert 00:33:40

4. People Identified

Nathan Lambert — Founder of Trillium; former post-training lead at Ai2; author of an RLHF book and a widely read educator on post-training. His thesis on open models and the grounded RSI view run through the episode. Lisha notes he has "done a great amount of education to the public on" post-training. 00:02:37

Lisha Li — a16z partner and the host; describes investing in both frontier models and startups because specialized-data tasks create "many, many winners." 00:54:20

Jensen Huang — NVIDIA CEO; clipped as framing distillation as competition.

"People distill my products every single day. They take it and they strip it down to its raw parts. They test everything. That's called competition." — Jensen Huang 00:15:25

Hannaneh Hajishirzi — Referred to as "Hannaneh," Lambert's co-lead of post-training at Ai2, originator of the framing that prompts are the key unit.

"I remember at AI2 like with Hannaneh who I co-led post-training there with, we would always say like prompts are the key unit of post-training." — Nathan Lambert 00:10:41

Hamish Ivison — UW/Ai2 student Lambert advised on tmux terminal-task synthetic environments, who declined to sell the data for $1M in favor of open release.

"I was joking with the author, 'Hey, you could probably sell this data for $1 million and then release the paper.' And they're like, 'No, we care about doing the right thing.'" — Nathan Lambert 00:32:46

Mark Zuckerberg — Referred to as "Mark"; wrote the Llama 3-era blog post on why open is the path forward, and is seen as "coming back to the line." 00:47:45, 00:47:53

An Eastern European contributor — An unnamed volunteer who helped improve code for Lambert's book "for the right thing," cited as evidence open communities attract real contributors. 00:36:44

5. Operating Insights

Fast Following Is a Documented Pipeline: Data Vendors Resell Exclusives After the Contract Window

Lambert describes the mechanism by which frontier know-how diffuses into the open ecosystem on a lag. For an operator, this is a planning heuristic: any proprietary dataset advantage has a roughly one-year half-life if a vendor is involved.

"OpenAI and Anthropic spend way more to build out this function of a data industry, and then those data companies, a year later, might sell them to everybody. They might sell them everywhere. They need to maximize their return on their investment, and they had some contractual period." — Nathan Lambert 00:06:58

Data Vendors and Labs Are Misaligned on What They're Optimizing

The labs' researchers obsess over eliminating reward hacking; the vendors spend proportionally more on world-building and revenue. The opportunity for operators and startups is to bridge model-behavior expertise and environment production.

"All these company researchers spending so much time on it, and then all the data companies like buy our data, and I think proportionally spend less time on model behavior, more time on world building and revenue. And somehow these two parties need to meet together to actually make higher quality data." — Nathan Lambert 00:29:14

Environment Quality Comes From Reading Every Data Point and Using Rollouts to Find Bugs

A concrete recipe for synthetic RL environment creation that is cheap relative to its leverage.

"You prompt a frontier model to generate a type of problem, and you use a few different prompts to generate diversity, and then you use model rollouts and other things to find the bugs in them, and eliminate the ones with bugs. And this process is not magic. It's like, you read every data point, your data will get better." — Nathan Lambert 00:32:06

The Data Industry Has Opsec Rough Edges Worth Auditing

Lambert flags data supply-chain leakage as a real risk and a vendor-selection criterion.

"Data companies that use contractors overseas. Those people are through an intermediary, they don't know they're selling it, they just upload it to GitHub." — Nathan Lambert 00:31:19

To Break Into a New Technical Field, Hand-Write Code for Months

"You have to write some code by hand, and you have to do this for months, and then you have to wait 6 to 18 months for you to get noticed." — Nathan Lambert 00:37:47

6. Overlooked Insights

Efficiency Gains May Be the Real Near-Term "Recursive" Win, with Jevons-Paradox Ambiguity

In passing, Lambert states that price cuts of 2x every 8 months for 3 years compound to roughly 1% of the cost within about 4 years, and that models will be genuinely good at cutting their own serving costs because of the billions at stake. The unresolved question, raised only as an aside, is whether lower cost leads to higher margins via Jevons paradox or lower revenue as the market fills. This reframes the RSI debate: the verifiable, measurable loop (serving efficiency, architecture tweaks) is where self-improvement is most plausible, not "solving knowledge work."

"What does it mean for GPT-4o or Astra, and these bigger models, to reduce in price 2x every 8 months for the next 3 years? Like you start doing this math, and it's 1% of the cost in whatever, like 4 years, and it's like are they going to be able to have higher margins due to Jevons paradox there, or their revenue is going to decrease because the market fills up?" — Nathan Lambert 00:27:05

"I do think the models will be really good at lowering serving costs. These companies have billions of dollars on the line... But it's not necessarily like how do you solve knowledge work?" — Nathan Lambert 00:42:00

The Open-Source Ecosystem Is Already Being Flooded With Agent-Generated Noise, Raising the Value of Verified, Trusted Open Infrastructure

Tucked into a tangent about GitHub issues is a significant second-order effect: AI agents generating fake research results and plausible-sounding issues, making maintainer attention and credible curation scarcer. That raises the strategic value of nonprofit labs like Trillium as trusted sources of open standards, and of tools that filter agent-generated contributions.

"People open the most crazy GitHub issues now... they open an issue, and they're like, 'I found this thing in your dataset,' and I'm like, 'Well, if you looked at the paper, we said that in like the second sentence.' And it's just like a lot of language model output, and that's a new thing for the open community." — Nathan Lambert 00:35:56