20VC: Taking on NVIDIA: Fractile CEO on the Future of Chips | What Everyone Gets Wrong About Capex Spending | The Winners and Losers in Neo Clouds with Walter Goodwin

1. Key Themes

The Memory Wall Is the Core Physics Bottleneck in LLM Inference

Walter explains that compute (FLOPs) has scaled roughly 25,000x faster than memory bandwidth over 25 years, which makes autoregressive LLM generation bandwidth-limited rather than math-limited. This is the technical foundation of Fractile's bet.

"We've scaled it about a million-fold in the last 25 years on these chips. Bandwidth, by contrast, is kind of much more intrinsically physics-limited. We've scaled it about 40-fold in the same time. So, you know, it's lagged behind the flop scaling about 25,000 times." — Walter Goodwin 00:18:57

"The reason that if you take a very large frontier model, it might generate at 80 tokens per second, 80 words per second, is going to be not because of the limited maths capability on the chip, but because you don't have nearly enough bandwidth to that memory." — Walter Goodwin 00:18:57

Test-Time Compute Makes Inference the Dominant Workload (and Speed the Missing Capability)

Walter's founding thesis four years ago was that training is a fixed cost and inference is "the only part that matters." Reasoning models and agent swarms have validated it. The hardest problems, such as Millennium Prize problems, require enormous sequential token generation, and no current silicon does that fast enough.

"The hard logic of building something with a high fixed cost is that you must believe you're going to be doing enough deployment to justify that." — Walter Goodwin 00:42:52

"That Navier-Stokes solution was, I think, 80 hours of execution from 10,000 agents in a swarm." — Walter Goodwin 00:43:25

"The many, many millions of sequential token generation means you're waiting days and days and days to get one of these responses. And it's a space which, somewhat tantalizingly, hasn't been cracked at all by any of the silicon, any of the chips that we have today." — Walter Goodwin 00:24:54

The Chip Market Is Splitting Between HBM-Based and SRAM-Based Chips, and Neither Does Everything

HBM chips (GPUs, TPUs, Jalapeno, Etched, MatX) are boxed in by the same bandwidth specs. SRAM chips (Groq, Cerebras) are fast but capacity-constrained. Fractile's bet is 3D-stacking DRAM on logic to get "SRAM-level bandwidth" with "DRAM-level capacity."

"They are fundamentally kind of boxed in by the like Top Trump specs, right? You can get only so many flops, and you can get only so much bandwidth. The uh HBM that sits off to the side of them is exactly the same piece of silicon. It's the same module from the same vendors for each and every one of those chips." — Walter Goodwin 00:24:54

"You suck the SRAM upstairs basically, and you have a you have a DRAM layer stacked on top. All of that compute die on the base is now free to put Tensor Cores down... you get the SRAM-level bandwidth from the 3D access, and you get the capacity from the DRAM. And so suddenly, you have this kind of, you know, holy trinity." — Walter Goodwin 00:58:09

Energy and Data Center Capacity Is the Binding Constraint, Which Raises the Value of Efficient Chips

The bottleneck has rotated from GPUs to HBM and logic silicon to land and power. Scarcity of megawatts raises the premium on tokens-per-megawatt.

"If you want to be completely agnostic and invest in, you know, you're long AI, you know, bu- buy energy basically, because that is obviously the kind of most fundamental." — Walter Goodwin 00:36:03

"If you're running that on an HBM-based chip, you're doing so at this kind of extraordinarily inefficient point on a kind of tradeoff curve. On these chips, you can either be sort of slow and efficient in generating tokens or fast, but deeply inefficient. Sort of far fewer tokens per second per megawatt of power." — Walter Goodwin 00:37:46

Vertical Integration Is a Margin Defense, but the Chip Layer Retains Durable Value

Everyone from labs to neo clouds is verticalizing to avoid commoditization and fight Nvidia's margins. Walter argues the chip layer is structurally protected because of long design lag and long deployment lifetimes.

"That lag to design the chip, to get the chip out, to have it built, and then the degree to which it has to have a useful lifetime in a data center means that you don't have quite that same kind of fully efficient race, you know, potentially a race to the bottom in some ways, in the chip space as you do elsewhere." — Walter Goodwin 00:50:47

Frontier Intelligence Isn't Commoditizing, but Neo Cloud Compute Is

Walter distinguishes between layers: frontier models and even well-executed open-model serving retain margins, while generic competent neo cloud capacity is fungible.

"Frontier intelligence is not yet commoditizing. We see, you know, large margins there." — Walter Goodwin 00:08:09

"One neo-cloud that is competent is much like another neo-cloud that is competent. The reason it's so exciting today is we don't yet know which ones are competent, so I think there's still a lot of space there for there to be huge winners, and and maybe sort of huge losers as well." — Walter Goodwin 00:08:09

The Back-End Partner Platform (Broadcom) Homogenizes Custom Silicon

Most custom accelerators (TPU, MTIA, Jalapeno, MatX, Etched) are rumored to share Broadcom as the back-end partner, the same TSMC nodes, and the same HBM. Fractile's in-house physical design and direct foundry relationships are positioned as a structural advantage.

"Charlie Kawwas... held up a kind of Lego model of the next-gen platform, and he said, 'This is going to be in production with six major customers in 2028.'... You get to change the color of the paint, basically." — Walter Goodwin 00:30:33

"You've gone and found a way to use resources that others haven't. So, in our case, a different process node, maybe a slightly different DRAM node or DRAM generation... the opportunity to build gigawatts of capacity sort of down the back of the sofa with things that other people... have already moved away from." — Walter Goodwin 00:30:34

Heterogeneous Chip Orchestration Is Likely a Transient Phase

Disaggregated inference (e.g., Nvidia pairing Rubin GPUs with Groq chips) creates stranded capacity risk because the mix is fixed at CapEx time while workloads shift.

"You're very likely to end up in a place where some portion of that capacity in your data center is stranded. You're just not filling it because you've got the wrong mix." — Walter Goodwin 00:56:38

"Heterogeneity is is actually a really challenging thing, and I think it's a transient response to the fact that you don't today have a chip that can kind of do it all." — Walter Goodwin 00:57:07

The Market Is Underpricing the Sustainability of the Buildout

Walter argues the dot-com trauma means markets price in a bust, while GPUs are fully utilized and algorithmic efficiency gains are compounding.

"We're so traumatized by the dot-com bubble that we've priced in exuberance right from the outset." — Walter Goodwin 00:00:00

"Far from the fiber buildout of 2000, 2001, we have complete, full utilization of the GPUs that are powered on in data centers. We don't have dark GPU fleets." — Walter Goodwin 00:48:19

Tapeout Is Over-Fetishized; Manufacturability, Yield, and Systems Integration Are the Real Work

Walter pushes back on the chip startup and VC obsession with tapeout dates, pointing to what comes after: yield, repair/redundancy, reliability, and rack-scale bring-up.

"There's been this sort of accidental shared obsession between sort of chip startup founders and the the venture capitalists that are backing them on this phrase, cuz it's such a tangible moment." — Walter Goodwin 00:34:14

"The thing that we're continually shooting for, which we call Task 0, is, you know, the day that a particular reference workload runs on an entire rack of Fractile chips." — Walter Goodwin 00:32:17

2. Contrarian Perspectives

Nvidia at $10 Trillion in Five Years Is "Cheap," Because the Market Is Pricing a Bust

Most investors see Nvidia as peak-cycle risk. Walter says confidently that it will be a $10 trillion business and claims it trades below Cisco's post-crash forward multiple.

"Nvidia today trades Uh this is a sound bite, so it may be unscored, but uh I believe it trades at a lower forward multiple than Cisco after the crash, you know, after the dot-com bubble burst." — Walter Goodwin 01:09:45

"We're so traumatized by the dot-com bubble that we've priced in exuberance right from the outset. And so, you know, I think we're not yet appreciating how sustained this buildout will be." — Walter Goodwin 01:10:01

Don't Build the Business You'd Be Expected To: Pure Chip Company Versus Vertical Integration

The consensus is that everyone is moving full stack. Walter explicitly commits Fractile to never monetizing models and never competing with customers, while still staffing ML researchers so the chip bets are right.

"We never intend to compete with our customers. We will never monetize internal models, because that's how we've chosen to build this company." — Walter Goodwin 00:05:09

Prefer Being an Inference Provider Over a Neo Cloud

Neo clouds are the hot asset class. Walter is more bullish on inference specialists (Modal, Fireworks, Baseten, Together) because they haven't locked up capacity and can pivot to new chips.

"I would rather be an inference provider today, you know, a specialist in running these models, getting them up, than I would a neo cloud, actually." — Walter Goodwin 01:01:04

"They haven't yet locked up huge amounts of their own capacity... They have the potential to bet on a new chip as it comes up." — Walter Goodwin 01:01:04

Orchestration Software Over Heterogeneous Chips Is a Danger Zone

A fashionable thesis holds that value accrues to those orchestrating many chip types. Walter says that value could collapse as a single better chip removes the need for heterogeneity.

"I think there is a danger for anybody that is working, you know, purely in the software layer... I think a lot of that could collapse back down over time, you know, that that there isn't a fundamental, existential reason why we desperately need to have seven different platforms to run a workload." — Walter Goodwin 00:53:57

Per-Company Fine-Tuned Models Are a Transient Advantage

Pushing back on the idea (raised by Harry from Satya Nadella's comments) that every company will have its own proprietary-data model, Walter says fine-tuning advantages vanish with the next open model release.

"Generally, the advantage of having something that is internally fine-tuned will disappear as soon as the next release of the next open model arrives." — Walter Goodwin 00:52:39

3. Companies Identified

Fractile AI inference chip and rack-scale systems company, founded by Walter Goodwin four years ago. Just raised a $720M Series C, co-led by Thrive, Founders Fund, Lightspeed and Redpoint, at a $7B valuation. Why mentioned: the subject of the episode, building 3D-stacked DRAM-on-logic chips for much faster, higher-capacity inference.

"We're a frontier AI company of sorts, but we make chips. We make these big rack-scale systems that are, you know, fully focused on advancing the frontier of AI capabilities." — Walter Goodwin 00:04:12

Nvidia Dominant GPU maker. Why mentioned: the incumbent; praised for volume and un-opinionated general-purpose chips; acquired Groq; Walter predicts a $10 trillion valuation in five years.

"Nvidia is an extraordinary business, and they build by many metrics, in a sense, the best AI chips in the world, right? And the best systems." — Walter Goodwin 00:13:43

Groq SRAM-based fast-inference chip company, acquired by Nvidia. Why mentioned: pioneering fast inference, but capacity-limited; part of the Nvidia disaggregated deployment.

"This SRAM play from Groq was the right solution at that moment to be able to run LLMs faster. Today, though... You cannot solve LLM inference with these SRAM-based chips." — Walter Goodwin 00:59:58

Cerebras SRAM-based wafer-scale chip company. Why mentioned: has a deal with OpenAI; cited as a new silicon platform and a possible catalyst for Nvidia's Groq deal.

"There is another rumor that obviously Cerebras got a deal with OpenAI. I think that can be an alarming moment if you're running a multi-trillion dollar GPU company." — Walter Goodwin 00:59:30

TSMC World's leading foundry. Why mentioned: the supplier Walter would invest in.

"It is cheap at the price. I mean, I would I would buy that stock every day of the week. Uh all of the world's chips that are at the frontier, or the majority of the world's chips at the frontier, are built there." — Walter Goodwin 01:06:36

Broadcom Back-end design partner for custom AI accelerators. Why mentioned: the intermediary whose platform homogenizes TPU, MTIA, Jalapeno and others, contrasted with Fractile's in-house approach.

"You know, it's a kind of you get to change the color of the paint, basically." — Walter Goodwin 00:30:33

DeepSeek Chinese frontier lab. Why mentioned: open-sourced its inference stack, achieving much higher throughput on GPUs, evidence that full-stack ownership preserves margins even in open model serving; Harry noted 80% efficiency gains with no hardware change.

"DeepSeek had done so well in building an inference stack to run their models on GPUs that they were getting, you know, much higher throughputs, much better interactivity, better speeds than everybody else who had built a DeepSeek inference server." — Walter Goodwin 00:08:09

Fluidstack Neo cloud. Why mentioned: first neo cloud to step away from Nvidia by deploying Google TPUs, with an "extraordinary year."

"They were, in some ways, the first neo-cloud to kind of decide that they were going to step bodily away from Nvidia, and they deployed a bunch of Google TPUs, and they've had an extraordinary year." — Walter Goodwin 00:12:08

Nebius Neo cloud, mentioned by Harry. Why mentioned: example of a neo cloud moving into model routing and inference to avoid commoditization.

"We had the founder of Nebius on the show, who was talking about the movement into model routing, and inference for them as a core business." — Harry Stebbings 00:11:01

OpenAI Frontier lab building its own chip ("Jalapeno") and an early reasoning-model publisher. Why mentioned: vertical integration example; signed a Cerebras deal; Noam Brown's team published the first reasoning model.

"If you think about what, you know, OpenAI did with Jalapeno in nine months, it suggests some form of commoditization of the layer." — Harry Stebbings 00:06:55

Google (TPU) Why mentioned: TPUs are among the Broadcom-platform custom chips and were deployed by Fluidstack.

Meta (MTIA) Why mentioned: custom chip program; Fractile hired Dan Rabinovitsj from Meta for systems buildout.

Etched Specialized transformer chip company. Why mentioned: cited as an HBM-based competitor with clever architecture but bounded by the same memory specs.

"Etched, MatX, they have often really brilliant kind of architectures on the chip... But they are fundamentally kind of boxed in by the like Top Trump specs." — Walter Goodwin 00:22:51

MatX HBM-based chip startup. Why mentioned: rumored to use Broadcom, same HBM and TSMC nodes as other custom chips.

D-Matrix Chip company pursuing a different architecture in the same 3D DRAM space. Why mentioned: the competitor Walter would invest in.

"I'm a big believer in 3D DRAM, and one other company called D-Matrix is pursuing a different architectural direction in the same space. I think both companies are going to be very successful." — Walter Goodwin 01:07:59

Inherent London-based neo lab started by Walter's friends, focused on automating science and research. Why mentioned: the customer Walter would invest in.

"Friends of mine have started Inherent in London... what it takes to build systems that automate science, automate research." — Walter Goodwin 01:07:12

Modal, Fireworks, Baseten, Together Inference providers. Why mentioned: Walter is bullish on all and prefers them to neo clouds.

"They still have the capacity to dodge and weave, right? They They haven't yet locked up huge amounts of their own capacity." — Walter Goodwin 01:01:04

Anthropic Why mentioned: named by Harry among labs building their own chips.

Nscale UK neo cloud. Why mentioned: Harry referenced the public backlash when Microsoft pulled out of the Nscale project.

Microsoft Why mentioned: pulled out of the Nscale project; Satya Nadella's view on per-company models was raised.

Atlassian Why mentioned: Mike Cannon-Brookes predicted inference costs will drop dramatically in 3 to 5 years.

Eight Sleep, MongoDB, Framer Episode sponsors; MongoDB cited as running 40M ElevenLabs agents and 75% of the Fortune 100's critical apps.

ElevenLabs Why mentioned (sponsor read): runs 40 million agents on MongoDB.

Cisco Why mentioned: benchmark for the dot-com crash forward multiple versus Nvidia.

Fuse Why mentioned: Harry referenced Alan Chang at Fuse in the context of energy.

4. People Identified

Walter Goodwin Founder and CEO of Fractile. Why mentioned: guest; first-principles inference thesis four years ahead of consensus.

"It was sort of what I set the company up to do four years ago, is build chips that are better at LLM inference." — Walter Goodwin 00:24:54

Jensen Huang CEO of Nvidia. Why mentioned: described as "quite positive-sum" and a smart guy who wants smart people; said nice things about Fractile in London.

"He said some nice things about Fractile last week when he was in London, you know, and and is, in some ways, a bit of an ecosystem player." — Walter Goodwin 00:58:48

Noam Brown OpenAI researcher. Why mentioned: his team published the first reasoning model, validating Walter's test-time compute bet.

"I'm, you know, very grateful to Noam Brown and the team at OpenAI that were the first to kind of, you know, publish one of these." — Walter Goodwin 00:42:52

Dan Rabinovitsj Ex-Meta AI compute systems leader, recently joined Fractile. Why mentioned: driving the systems buildout from chip to rack to networking.

"I'm very excited that we were joined just a few weeks ago by Dan Rabinovitsj from Meta, who worked on a lot of their AI compute systems there." — Walter Goodwin 01:04:47

Charlie Kawwas Broadcom leader. Why mentioned: showed the Lego model of the next-gen platform shared by six customers.

Demis Hassabis Why mentioned: the board member Walter would most like.

"Demis Hassabis is both an academic hero of mine and a bit of a British hero, and I think he should uh he should come aboard." — Walter Goodwin 01:09:06

Satya Nadella Why mentioned: Harry relayed his view of per-company specialized models and evals as a value segment.

Mike Cannon-Brookes Atlassian co-founder. Why mentioned: predicted inference costs will fall dramatically in 3 to 5 years.

Sarah Guo Why mentioned: Harry referenced Walter's appearance on No Priors with "our mutual friend Sarah."

Mark Zuckerberg, Andrew Huberman, Charles Leclerc Why mentioned: sponsor read (Eight Sleep).

Alan Chang Fuse. Why mentioned: Harry praised him in the energy context.

5. Operating Insights

Pull the Hardest Part of the Supply Chain In-House, Then Use It to Reach Different Resources

Fractile's in-house physical design and direct foundry relationships mean they weren't confined to the Broadcom platform's building blocks, and they could use different process nodes and DRAM generations than frontier HBM buyers. This sidesteps the gold rush for finite TSMC and HBM capacity.

"Fractile's always had that in-house... it's become a really core strategic advantage for us, because it means we've both always had direct communion with our manufacturing partners. We've had to do that legwork from the outset." — Walter Goodwin 00:28:00

Build the Internal Expertise of the Layer Above and Below You, Without Competing With It

Fractile employs ML researchers running small experiments, not to build products but to avoid placing the wrong chip bets and to give customers architectural guidance.

"It's completely essential that we have a bunch of folks internally who are ML researchers, who are looking at how model architectures might evolve... You have to synthesize everything that you're learning across the ecosystem, and you have to bring that back then to be obsessive about the part of the space that you're building." — Walter Goodwin 00:05:09

Stay Mission-Defined, Not Technology-Defined

Walter deliberately avoids becoming "the company that cracked 3D DRAM," keeping the org primed to change response if the workload shifts.

"One thing which we're cautious not to become is the kind of the company that cracked 3D DRAM. We want to be challenged to continue to innovate on, you know, whatever comes next." — Walter Goodwin 00:22:51

Track a Customer-Relevant Milestone, Not a Vanity One

Instead of tapeout, Fractile's north star is "Task 0": a reference workload running on a full rack. This keeps the team focused on the post-tapeout work that decides success.

"The thing that we're continually shooting for, which we call Task 0, is... the day that a particular reference workload runs on an entire rack of Fractile chips." — Walter Goodwin 00:32:17

Use Capital to Lock Supply Years Ahead and Build Predictive Muscle

With the $720M, Walter says the main use is not first-silicon costs but maneuverability: long-term agreements on components, multi-generation roadmaps, and in-house experiments.

"The placing of long-term agreements over certain components that we need to have years into the future, and the ability, I think, to build more of that muscle of predictive power." — Walter Goodwin 01:04:47

6. Overlooked Insights

Chip Replacement Cycles Are the Hidden Demand Engine, Independent of New Data Centers

Almost as an aside, Walter notes that because megawatts are finite, operators must swap in more efficient chips on short cycles. That means chip demand doesn't depend on incremental data center builds, and it is why a new entrant can win share even if capacity is capped.

"If I have finite numbers of megawatts, I need to replace my chips every few years with better chips... every chip that you see in a data center today will definitely be out of those data centers 6 or 7 years tops. Many of them won't last more than a couple of years." — Walter Goodwin 01:10:01

Cheap, Older-Generation DRAM and Process Nodes Could Yield Gigawatts of Supply "Down the Back of the Sofa"

Buried in the Broadcom discussion is a supply-chain arbitrage: by designing for a different process node and DRAM generation, Fractile can tap capacity that frontier HBM buyers have already abandoned, sidestepping the scarcity everyone else is fighting over.

"You sort of find, you know, the opportunity to build gigawatts of capacity sort of down the back of the sofa with things that other people and this sort of, you know, set of frontier platforms have already moved away from." — Walter Goodwin 00:30:34