BREAKING: SambaNova Hits… | Sourcery Newsletter Summary

1. Key Themes

Theme 1: Inference Is the New Frontier — And It Dwarfs Training in Scale

The AI industry is transitioning from a training-centric cost structure to an inference-dominated one, and the chip and infrastructure demands are categorically different.

"How do you actually deploy at scale so the whole planet can use it without burning up the planet, without running out of data center space, without blowing up your infrastructure cost? Because at scale, the number of chips deployed for inferencing will be orders of magnitude greater than whatever you're doing for training."

Per-token unit prices have dropped roughly an order of magnitude over two years while consumption rose more than 100x, meaning inference has moved from a lab expense to a product gross-margin line.

Theme 2: The Rack Is the New Unit of Competitive Advantage

Power density and physical footprint are becoming decisive competitive moats — not just raw model performance. The ability to run frontier models in existing data centers without liquid cooling or new construction changes the deployment calculus entirely.

"Instead of 130, 140 kilowatt rack of Nvidia GPU, we were outperforming them with a 10 kilowatt SN40 rack.. air-cooled. You didn't need liquid cooling upgrades."

"We're using standard Kubernetes, standard Red Hat Linux, standard Ethernet at the top for networking.. And so that allows people to go in to existing data centers, roll this thing in, pull out the old gear, & you're up & running with new services, which otherwise might take you nine months to a year, maybe as long as 18 months, to build a gigawatt data center."

Most data centers handle 30 kW racks; 120–140 kW effectively requires purpose-built facilities. A 10 kW air-cooled rack can enter markets that $100B data centers structurally cannot.

Theme 3: Speed Is the Premium Inference Product — Tokenmaxxing Is the Wrong Goal

The article identifies a coming bifurcation in inference markets: speed as a premium tier, with latency becoming the primary price driver in agentic workflows.

"If each of those 20 agents took two seconds, that's 40 seconds, you've already given up on that prompt, right? So the response time by the end user's expectation is, say, 1-2 seconds. Divide that by 20, it's 0.1 seconds per."

"When 5G showed up, nobody's signing up for 2G.. It's never been the case, 'Let me pay more for the slow.'"

The risk articulated: enterprises focused on maximizing token volume ("tokenmaxxing") without tracking per-rack margin or latency will misallocate AI spend.

Theme 4: Open Weights No Longer Means Cheap Inference

The release of Kimi K3 (2.8 trillion parameters) signals a structural shift: open-source models have crossed a size threshold where only a handful of operators can actually serve them, eliminating the commodity pricing pressure that defined the open-weights era.

Jamin Ball's framing: "Open weights were a cost lever and the 'license' was the discount." At 2.8T parameters that lever mostly disappears.

K3's weights alone come to roughly 1.4TB, requiring 10+ H200s just to load the model before touching the KV cache, and Moonshot's own launch materials recommend supernode configurations of 64 or more accelerators — the same minimum quantum problem Liang identifies for competitor hardware.

Theme 5: AI Sovereignty and the Return to On-Premises Deployment

Data privacy concerns and national sovereignty are driving regulated industries and governments away from shared cloud infrastructure toward dedicated, on-premises AI deployments — a structural tailwind for full-stack hardware vendors.

"I don't want my data trained into a model and have that model shipped worldwide. Can you imagine if your bank account information starts showing up in ChatGPT in some other place in the world without your permission?"

"I was talking to a CIO recently of a big bank.. they were never gonna go to a cloud, and it's like, 'See, I knew all along.' I was like, 'Yeah, it's cyclical.' A 20-year cycle, wait long enough, it'll come back."

SambaNova has signed sovereign agreements with operators in Australia (SCX), Europe (Infercom), the UK (Argyll), and Saudi Arabia (stc Group), and counts JPMorgan Chase and multiple US Department of Energy national laboratories as enterprise customers.

2. Contrarian Perspectives

Perspective 1: $50–100B Hyperscale Data Centers Are Not the Inevitable Future of AI Infrastructure

The consensus narrative is that AI requires ever-larger centralized data centers. Liang challenges this, arguing that latency economics and physical constraints in dense urban environments will force distributed, small-quantum deployments.

"I think you're gonna have some data centers like that, because I think there's still going to be large scale deployments.. And I think you're gonna find that the world's gonna be heterogeneous." "In terms of ultra-low latency in the big cities, you're gonna have to find smaller quantums."

The evidence: most existing data centers handle only 30 kW racks, making 120–140 kW GPU clusters physically inaccessible without multi-year, billion-dollar construction. A 10 kW air-cooled rack reaches those markets today without new facilities.

Perspective 2: Inference Providers Are Not Actually Making Money — Revenue ≠ Margin

While inference providers are reporting impressive revenue growth, Liang argues the economics beneath the surface are broken, and that infrastructure efficiency — not model capability — is the real competitive differentiator.

"For them to sustain themselves, as you know today, inference services, they're not making enough margin. They're generating lots of revenue, but you're not generating enough margin."

"People forget, as much as Nvidia costs, it's commodity. Because what you offer is the same as what your neighbor offers.. and your differentiation is, 'I can save you a little bit of money because maybe I got a discount from Nvidia.'"

SemiAnalysis reports inference providers including Fireworks, Baseten, and Fal are seeing widening margins on hyper-growth revenue — but Liang's counter is that GPU-only providers have no structural differentiation to sustain those margins long-term.

Perspective 3: Enterprise Cloud Migration Will Partially Reverse Within Two Years

Against the consensus that enterprise AI workloads migrate to cloud, Liang argues that commoditized cloud AI destroys enterprise differentiation and will drive a return to proprietary, on-premises deployments.

"If you actually transfer all of those services, that differentiation to all using the same exact model that's in the community, where does the differentiation come? And so what you find yourself is in a place of low margin for all these enterprises that historically had been able to enjoy much better margins."

He believes companies will recognize this dynamic "within two years" — and SambaNova's full-stack model (chips + systems + software + cloud sold directly) is specifically designed to capture that return.

3. Companies Identified

SambaNova Systems

Description: AI chip and systems company, Series F at $11B valuation, $2.5B total raised
Why mentioned: Central subject; maker of SN40/SN50 RDU chips; sells full-stack AI infrastructure to enterprises, sovereigns, and neoclouds
Quotes: "We take the biggest models in the world & we run them in the original precision. We don't quantize... We just run original precision, full precision, run it faster than anybody else."

Moonshot / Kimi

Description: Chinese AI lab; released Kimi K3, a 2.8 trillion parameter open mixture-of-experts model
Why mentioned: K3 is cited as a structural inflection point proving that open weights are no longer cheap to serve
Quotes: Arena.ai: "Kimi-K3 is now #1 in the Frontend Code Arena with 1679 pts, surpassing Claude Fable 5."

JPMorgan Chase

Description: Global bank deploying on-premises AI infrastructure
Why mentioned: Announced as a SambaNova enterprise customer, deploying SN40/SN50 to run AI workloads without third-party cloud
Quotes: Rodrigo told CNBC that "JPMorgan selected SambaNova to be the inference provider for the bank."

SoftBank Corp.

Description: Japanese telecom and technology conglomerate
Why mentioned: First deployment partner for the SN50; already hosts SambaCloud for developers; SoftBank Vision Fund 2 led SambaNova's 2021 Series D
Quotes: "SoftBank Corp. already hosts SambaCloud for developers in the region, and by anchoring new clusters on SN50 it positions SambaNova as the inference backbone for its sovereign AI initiatives."

Together.ai

Description: AI inference platform
Why mentioned: First commercial customer of SambaNova's disaggregated inference architecture (Nvidia for prefill, SambaNova for decode) at Vector Core Compute
Quotes: "Together.ai is the first commercial customer of the disaggregated architecture at VC2."

Vector Core Compute (VC2)

Description: Agentic neocloud announced June 2026, backed by Vista Equity and Cambium
Why mentioned: Built on SambaNova hardware; both Vista and Cambium are investors in SambaNova's current round
Quotes: "Vista Equity and Cambium announced Vector Core Compute, an agentic neocloud built on SambaNova hardware, in June 2026."

Baseten

Description: ML inference platform
Why mentioned: Resells SambaNova as an Nvidia alternative in multi-tenant environments
Quotes: "Baseten resells SambaNova as an Nvidia alternative in multi-tenant environments."

CoreWeave

Description: GPU cloud provider
Why mentioned: Used as evidence of infrastructure constraint scale: "crossed 1 GW of active power in the quarter and expanded contracted capacity past 3.5 GW, with capex raised to as much as $35 billion for 2026."

DeepSeek

Description: Chinese AI lab; maker of open-weights frontier models
Why mentioned: DeepSeek R1 671B and V4-Pro (1.6 trillion parameters) cited as benchmarks for the minimum quantum problem and scale of open models
Quotes: "Just to run that, the minimum for some of the other providers might be 10 to 20 racks."

MiniMax

Description: Chinese AI lab; maker of open-weights coding model M2.7
Why mentioned: Used in SambaNova's RAISE Summit live demo of disaggregated inference; SambaNova leads measured output speed for MiniMax M2.7 per Artificial Analysis
Quotes: "At RAISE, SambaNova ran MiniMax M2.7 on a setup that split the work between two kinds of chips."

OVHcloud

Description: European cloud provider
Why mentioned: Named as a SambaNova partner/customer in sovereign cloud deployments

General Atlantic

Description: Global growth equity firm
Why mentioned: Lead investor in SambaNova's $1B Series F at $11B valuation

Intel Capital

Description: Intel's strategic investment arm
Why mentioned: Strategic partner and investor; entered planned multi-year collaboration with SambaNova in February 2026 alongside SN50 launch

Artificial Analysis

Description: AI benchmarking and analytics firm
Why mentioned: Third-party verification of SambaNova's decode speed claims; cited for pricing spread analysis across inference providers

4. People Identified

Rodrigo Liang

Description: Co-founder and CEO of SambaNova Systems; 32-year career in semiconductor design
Why mentioned: Primary interview subject; architect of SambaNova's chip roadmap and business strategy
Quotes: "I've been in this industry for 32 years, building high performance chips for a long time. I've never seen the interest in semiconductors higher."

Jamin Ball

Description: Partner at Altimeter Capital; author of Clouded Judgement newsletter
Why mentioned: Provided pricing analysis on Kimi K3 and articulated the structural shift in open-weights economics
Quotes: "Open weights were a cost lever and the 'license' was the discount."

Gavin Baker

Description: Managing Partner at Atreides Management
Why mentioned: Added the "token wastrel" framing on Kimi K3's cost-per-task inefficiency and articulated why compressed model-layer margins flow downstream to infrastructure
Quotes: "Kimi K3 may be an important inflection point for AI. Potentially negative for Anthropic and OpenAI while being net positive for essentially every other company in the world."

5. Operating Insights

Insight 1: Measure AI ROI as Revenue Per Rack, Not Revenue Per Model

The article articulates a precise, operator-grade formula for evaluating AI infrastructure economics that most enterprises are currently ignoring.

"The revenue's generated per token.. If I put a rack of hardware, I'm just seeing how many tokens am I generating in a particular model, and that model has a price per token, right? Multiply that by 30 days per month, 24 hours per day, number of tokens per second, & you can figure how much money that rack is generating, & you look at how much it's costing you to operate."

For operators buying or reselling inference capacity, this reframes the buying decision from "which chip is fastest" to "which rack generates the best margin per month on the models I serve."

Insight 2: In Agentic Workflows, Latency Math Compounds — Design for 0.1s, Not 2s

Entrepreneurs building agentic products need to redesign their latency assumptions around the chain, not the individual call.

"If each of those 20 agents took two seconds, that's 40 seconds, you've already given up on that prompt, right? So the response time by the end user's expectation is, say, 1-2 seconds. Divide that by 20, it's 0.1 seconds per."

This has direct procurement implications: API providers that look cheap on a per-token basis but are slow on decode will silently destroy agentic product UX at scale.

Insight 3: Differentiation Comes From What You Do With AI, Not That You Use It

"Just using AI doesn't differentiate. How do you differentiate?"

"If you aren't paying attention, it becomes token maximizer, just like, 'Hey, just spend.'"

Enterprises need to identify the specific workflow where AI output replaces a high-cost human check (e.g., code review, compliance review) and price the ROI against that saved labor — not against token volume.

6. Overlooked Insights

Insight 1: Disaggregated Inference (Nvidia for Prefill + SambaNova for Decode) Is Live in Production and Delivers 2x Speed

Buried in the demo section is a commercially deployed architecture that is not yet widely discussed: splitting inference phases across chip types based on their computational characteristics.

"At RAISE, SambaNova ran MiniMax M2.7 on a setup that split the work between two kinds of chips. One Nvidia H200 rack with four GPUs handled prefill, and one SambaRack SN50 with 16 RDU chips handled decode... The COMPUTEX version of this... delivered 2x the inference speed of B200-only configurations as verified by Artificial Analysis."

This is not a theoretical architecture — it runs on standard vLLM, is live at Vector Core Compute, and is commercially available through Together.ai. Inference operators still running GPU-only configurations are leaving measurable performance on the table today.

Insight 2: Supply Chain, Not Engineering, Is SambaNova's Primary Constraint Over the Next 12 Months

The article notes that the $1B raise is explicitly earmarked for supply chain — not R&D, not sales — suggesting that unfulfilled demand is the actual binding constraint on growth.

"Liang said the capital is going to secure the supply chain, which he called the key constraint on fulfilling orders over the next 12 months, with the SN50 shipping in the second half of 2026."

For investors and customers, this signals that order backlog likely exceeds production capacity, and that early commitments or partnerships (as SoftBank Corp. has done) may secure preferential allocation of a constrained resource.