Elena Burger | The a16z Show Summary

1. Key Themes

Open-Source AI Has Crossed the Threshold from Curiosity to Critical Infrastructure

What began as an enthusiast phenomenon has become the backbone of the most innovative AI products. The shift happened roughly a year ago when leading application companies determined they could not differentiate by building on closed APIs alone.

"About a year ago where a bunch of smaller companies or like new application companies as they were trying to figure out how do I really build an AI without just being a wrapper on top of OpenAI? The answer to that question turned out to be open source. This is what Cursor did. This is what Decagon and Harvey are in the process of doing now... some of the most innovative products and applications now really depend on this very deeply." 00:07:13 — Matt Bornstein

VLLM Is the Invisible but Ubiquitous Layer Powering AI Deployment

VLLM functions as the operating system layer of open-source AI inference, running on half a million GPUs at any moment, supporting over a thousand model architectures, and being used as a benchmark by every major hardware vendor.

"VLLM is an inference engine. That means its job is to turn available GPUs into a running endpoint for intelligence. So that means it is kind of like databases and operating system and other critical software to power this economy... we support more than a thousand model architectures up to today... their newest chip will make sure VLLM can run on them. And in a lot of cases, they use VLLM as a benchmark to make sure it runs well on them." 00:08:25 — Simon Mo

Control, Not Just Cost, Is the Primary Driver of Enterprise Open-Source Adoption

While cost reduction gets the headlines, the deeper and more durable driver for enterprises choosing open-weight models is operational control — over SLAs, data retention, guard rails, and system performance.

"Control matters a lot over the last few years and then costs just start to matter over the last few months... for a voice agent company, they want to control their own model so that they can make sure the model actually responds by their required time. So the customer when they're on the phone, they can ensure the agent is responding according to a SLA. And this sometimes is only you can do with your controlled intelligence because you know the whole hardware you're running and the whole system you're monitoring versus relying on your critical infrastructure with a proprietary API where they might go down anytime or have violation of the contract anytime." 00:14:46 — Simon Mo

Open-Weight Model Licensing Is Evolving Into a Pharmaceutical-Style R&D Funding Model

Pure Apache-2-style open source is not economically sustainable for frontier model training, which costs hundreds of millions of dollars. Labs are now experimenting with commercial license terms that attach revenue-share obligations to large users, mirroring how pharmaceutical R&D is funded.

"I recently heard someone making an analogy to this to the pharmaceutical industry — it's almost like how do you make sure that the R&D process of new drugs are properly funded and is a proper sustainable method to making sure that people are willing to take big risk, big bets to go do research for new drugs... a portion of those revenue will flow back to continue to fund the next R&D effort. And this is where we're kind of seeing similar to the model development now." 00:22:56 — Simon Mo

The Open-Source Model Release Is a Multi-Party Co-Design Process, Not a Simple File Drop

Each frontier open-weight model release involves the model lab, hardware vendors, inference engine teams, Hugging Face, inference clouds, and hyperscalers — all coordinated in advance to ensure day-zero usability. The Mistral launch illustrated what happens without this coordination.

"Even from the beginning of 2023, 2024, if you remember when Mistral dropped their first model, they just drop a torrent link for P2P and then everybody's struggling and trying to get it up and running and then we're working behind the scenes with the Mistral team trying to get the inference engine support working in VLLM... and then after the weekend... Mistral and us just announced, here, you can run it on VLLM successfully, and everybody will be able to easily reuse a lot of the work and start building on top of it." 00:11:53 — Simon Mo

Proprietary Model Guard Rails Are Producing Costly False Positives That Are Actively Pushing Developers to Open-Weight Alternatives

Over-aggressive content moderation by closed-source APIs is not just a philosophical problem — it is causing real workflow disruption and accelerating migration to open-weight models even among technically sophisticated users.

"A lot of the Anthropic models are banning frontier AI user research and then when we're studying GPU kernels, even as an invalid memory access error, we are triggering the red line and so a lot of our developers are retreating from using Claude 5 because you have a two-hour job and you trigger the red line which is false positive and then you have to lose all of your work and so a lot of our developers are using Kimi K3 today even just making sure because it's similar quality and it has a guardrail that makes sense to us." 00:33:12 — Simon Mo

Environment Quality, Not Data Distillation, Is the Core Differentiator in Frontier Model Training

The key variable separating leading labs is their ability to build high-quality RL training environments — iterative feedback loops where models improve themselves — rather than access to any particular dataset or distillation from a competitor's outputs.

"Environment matters so much today... what I'm trying to do to really distill a copy from is about constructing it, understanding also understanding the learning process. You cannot distill how the model learns within environments... what's powering the progress is still just really smart people with very interesting algorithms, data, environment, and compute — they will produce the models." 00:43:23 — Simon Mo

Open-Weight Models Are at Capability Parity With Closed Frontier Models Today

Simon Mo, who is closer to the inference layer than almost anyone, makes the direct claim that the capability gap between open-weight and closed-source frontier models has effectively closed.

"Capability-wise, I don't really see a big gap. Not even today." 00:00:55 — Simon Mo

2. Contrarian Perspectives

Distillation From Closed Models Is Not What Is Powering Chinese AI Progress

The popular narrative — and implied policy assumption — that Chinese labs are primarily advancing through distillation of US frontier models is likely wrong. The actual driver is smart researchers building superior training environments.

"I will lean to the latter part... you cannot distill how the model learns within environments. A lot of these are just not doable today... I really don't think from currently what we're seeing this is a big cornerstone of what's powering the progress today. In the end what's powering the progress is still just really smart people with very interesting algorithms, data, environment, and compute — they will produce the models." 00:43:23 — Simon Mo

"It has really interesting policy implications... it's almost tempting if you're in the White House to say oh sure we'll just turn off distillation all our problems will be solved but like I think it's more the case that they're just smart people doing interesting things." 00:44:23 — Matt Bornstein

The Economics of Open-Source AI Are Fundamentally Broken Without New Licensing Models — and This Is a Feature, Not a Bug

Unlike open-source software, which can be maintained by volunteer engineers, frontier AI models require billions in compute. The emerging commercial licensing terms from Chinese labs like Moonshot and Minimax are not greed — they are a necessary funding mechanism analogous to pharmaceutical R&D, and restricting them would be counterproductive.

"It's not like there are no missing decimal points or commas... if there's no source of funding for Moonshot to continue to train models, we know where the funding will come from instead... that's actually worse for us. I think you've raised an interesting point that this is an important economic structure and we'll see more of this in the future." 00:27:56 — Matt Bornstein

The Content Moderation Problem at AI APIs May Be Structurally Unsolvable, Making Open-Weight Models the Default for Serious Work

Unlike social media platforms, which have a legal carve-out limiting their liability for user content, AI API providers face direct liability for outputs and have no equivalent exemption. This structurally pushes them toward over-moderation, which will systematically drive high-value professional use cases to open-weight models.

"Before social media people talked to each other and published articles... you centralize it all under one profit-seeking enterprise and all of a sudden the incentives change a lot... they don't have that carve-out, right — they don't have that exemption... and it seems like some of these companies have erred on the side of caution which is probably a reasonable thing to do but very frustrating as a user." 00:34:10 — Matt Bornstein

3. Companies Identified

VLLM / Infract

Open-source inference engine and the company commercializing it, co-founded by Simon Mo. Runs on half a million GPUs at any given moment, supports over a thousand model architectures, and is used as a hardware benchmark by NVIDIA, AMD, Google, Amazon, and Intel. Achieves 400–500 tokens per second in fast mode, 2-3x faster than closed-source fast modes.

"NVIDIA, AMD, Google, their newest chip will make sure VLLM can run on them. And in a lot of cases, they use VLLM as a benchmark... we're bridging almost the 10x gap." 00:00:21 — Simon Mo

Cursor

AI-powered coding tool cited as a canonical example of an application company that moved beyond closed-source APIs to build on open-weight models with custom mid-training and inference.

"This is what Cursor did... really strong application-level startups sort of made the determination we can't build just on closed source. We need to do our own mid-training, our own post-training, our own sort of inference and deployment tricks." 00:07:13 — Matt Bornstein

Decagon

AI customer support company cited alongside Harvey as being in the process of migrating from closed APIs to open-weight model infrastructure.

"This is what Decagon and Harvey are in the process of doing now." 00:07:13 — Matt Bornstein

Harvey

AI legal research company cited as actively building on open-weight models rather than closed-source APIs.

"This is what Decagon and Harvey are in the process of doing now." 00:07:13 — Matt Bornstein

Moonshot AI (Kimi)

Chinese AI lab behind Kimi K3, described as bringing an "Opus 4.8 level" model to open-weight infrastructure. Cited for building one of the best RL training environments for front-end coding, and for innovative architectural choices including removing rotary positional embeddings.

"Kimi K3 is not as expensive as Claude or GPT-4, but it is a lot more expensive than GLM 5.2... this model is bringing an Opus 4.8 level model to our own infrastructure that I can use, I can run, I can fine-tune." 00:16:32 — Simon Mo

Mistral

French AI lab cited for their pioneering and chaotic first model drop via P2P torrent link, which became the impetus for the co-designed, multi-party model release process now standard in the ecosystem.

"When Mistral dropped their first model, they just drop a torrent link for P2P and then everybody's struggling and trying to get it up and running." 00:11:53 — Simon Mo

Hugging Face

Model hub and AI platform cited both as a critical infrastructure partner in model releases (model format/hub vendor) and for their transparent public disclosure of using a Chinese open-weight model to contain a rogue AI cyber attack.

"For the Hugging Face incident, they break it down pretty well on their website and blogs — so really thanks for their transparency there." 00:32:12 — Simon Mo

Open Router

Inference routing startup cited as another company that started in the 2022–2023 era around open models and has grown into meaningful infrastructure.

"Open Router, Ollama — all of these different teams kind of got started around 2022 and 2023, some of them even before ChatGPT." 00:29:59 — Matt Bornstein (host)

Ollama

Consumer-facing open-source model runner cited as a peer company in the open inference ecosystem, also dating to the pre-ChatGPT era.

"Open Router, Ollama... all of these different teams kind of got started around 2022 and 2023." 00:29:59 — Matt Bornstein (host)

Minimax

Chinese AI lab cited for pioneering usage-based commercial licensing terms with their M2.7 model release, representing the next evolution beyond Meta's Llama license template.

"Starting from even Minimax recently when they're releasing the M2.7 model, they have a term specifically focusing on usage." 00:20:45 — Simon Mo

Anthropic

Closed-source AI lab cited as an example of over-aggressive content moderation that is actively driving developers away — specifically, Claude 5 was blocking legitimate GPU kernel research by Infract's own engineers.

"A lot of the Anthropic models are banning frontier AI user research... a lot of our developers are retreating from using Claude 5." 00:33:12 — Simon Mo

GitHub Copilot

Cited as one of the first AI products that people "just cannot live without," marking the moment in 2023 when open-weight model infrastructure became truly critical.

"Maybe 2023, when GitHub Copilot and ChatGPT are something that people just cannot live out with anymore." 00:06:15 — Simon Mo

Fireworks AI

Inference cloud cited as a company that built on top of Kimi's open-weight model, triggering commercial licensing discussions about derivative works.

"Big news back then was Fireworks and Cursor about how they build on top of Kimi model." 00:21:15 — Simon Mo

4. People Identified

Simon Mo

Co-founder and CEO of Infract, lead maintainer of VLLM. Built the inference engine that now runs on half a million GPUs, supports 1,000+ model architectures, and is used as a hardware benchmark by every major chip vendor. Has hands-on relationships with every major open-weight model lab globally.

"VLLM also works closely with all the hardware vendors. So that means across NVIDIA, AMD, Google, Amazon, Intel, and a lot more, their newest chip will make sure VLLM can run on them." 00:09:23 — Simon Mo

Ion Stoica (referenced as "Young" / co-founder)

Referenced as Infract co-founder and advisor with a deep background in open-source databases (Apache Spark), Any Scale (Ray), and the LMSYS Arena. Described as the strategic thinker behind how to build a company on top of a widely-used open-source project.

"Young as a co-founder, he has always been thinking about open source and where, how do you support open source better... his focus really stemmed from how do we build such a world where this software is being used by so, so many people on such a critical project." 00:36:49 — Simon Mo

Jenny (Su Jianlin, referenced by first name)

Researcher at Moonshot AI, inventor of Rotary Positional Embedding (RoPE), and now author of the technical rationale for removing RoPE in Kimi K3. Cited as an exemplar of the open scientific culture in open-weight research.

"The inventor of RoPE... yeah, Jenny... wrote the first paper introducing RoPE as a concept and then also wrote the explanation of why you don't need it as part of the technical report in this K3 model." 00:41:27 — Simon Mo

5. Operating Insights

Day-Zero Model Support as a Competitive Moat and Business Development Strategy

Infract turned the technical practice of immediately supporting new model releases into both a community moat and a business development tool. By proactively reaching out to model labs before public release and offering to integrate support into VLLM, they position themselves as the default infrastructure layer the moment a model goes live.

"Hi, we're the VLLM team. We would like to support your open-source model. We would like to offer in a way this kind of open-source but white-glove service to get your model running well on architecture... and then on the other end, we really have model labs that just don't know how this can work." 00:10:28 — Simon Mo

Performance Tiering as a Product Differentiation Strategy

Rather than offering a binary fast/slow mode like proprietary APIs, open-weight inference providers can offer up to ten distinct performance and cost tiers. This creates a powerful upsell surface and allows providers to serve wildly different use cases — from cheap batch workloads to 500-token-per-second premium developer flows — on the same underlying model.

"For proprietary models, there is regular mode and fast mode, and that's only the two switches here. But for open weight, when you're running it, every provider can offer potentially even 10 different levels of speed going from the slowest mode, which can be a lot cheaper, to 400 tokens per second, up to 500 in many cases for some workloads. And this is typically 2x or 3x faster than the fast mode out there today." 00:17:31 — Simon Mo

Use False Positives From Closed APIs as a Sales Trigger

Developers experiencing guard-rail false positives on closed-source APIs are highly motivated switchers. Infract's own team switched from Claude to Kimi K3 for GPU kernel research after being blocked during two-hour jobs. This pattern — identify the exact pain point of lost work due to false positives — is a repeatable sales motion.

"A lot of our developers are retreating from using Claude 5 because you have a two-hour job and you trigger the red line which is false positive and then you have to lose all of your work and so a lot of our developers are using Kimi K3 today even just making sure because it's similar quality and it has a guardrail that makes sense to us." 00:33:42 — Simon Mo

6. Overlooked Insights

The Removal of RoPE in Kimi K3 Is a Signal That Architectural First Principles Are Being Actively Overturned — and Open-Weight Labs Are Leading This

This was mentioned only in passing but is architecturally significant. Rotary Positional Embedding has been a foundational component of transformer models for years. The fact that its own inventor — now at Moonshot — published the case for removing it entirely in a production frontier model suggests that open-weight labs, not closed-source labs, are now the primary venue for foundational architecture research. This inverts the common assumption that closed labs like OpenAI and Anthropic lead on fundamentals.

"The inventor of RoPE... Jenny... wrote the first paper introducing RoPE as a concept and then also wrote the explanation of why you don't need it as part of the technical report in this K3 model. So when we read it, you have all these brilliant humble researchers that are able to really study how this works and really study the secret of training and pre-training and share it across the whole world and recognizing and iterating on their past." 00:41:27 — Simon Mo

A 99% Drop in GPU Prices Would Restore True Open-Source AI — and This Is Not Science Fiction

Mentioned as a throwaway thought experiment, but it carries enormous investment implications. If GPU compute continues on a cost-reduction curve even remotely approximating historical semiconductor trends, the economic barrier that currently makes frontier model training the exclusive domain of billion-dollar organizations could dissolve. This would trigger a Cambrian explosion of grassroots model development analogous to the early open-source software era, and would structurally disadvantage any business model predicated on the scarcity of compute access.

"The fun thought experiment is if GPUs dropped in price by 99%, then do we get back to a real open-source world?" 00:00:00 — Matt Bornstein "AlexNet, the first neural network to run on GPUs that we care about, ran on two GPUs — there are no missing decimal points or commas in there. Literally two. Now that would get literally nowhere." 00:27:56 — Matt Bornstein