Teahose.
SIGN IN
NEW HERE — WHAT TEAHOSE DOES
We read the entire AI & tech firehose — so you don't have to.
PODPodcastsAll-In, No Priors, Acquired…
NEWNewslettersStratechery, Newcomer…
PAPPapersPhysical AI research
PHProduct Huntdaily launches
VCInvestor ScoutSequoia, a16z, Benchmark…
CLAUDE DISTILLS →
7 reads, 30 sec each — free, 6 AM ET.
+ a live graph of the companies, people & themes underneath.
HOME/THE GENERALIST/Why inference is harder than you…
NEWS
// NEWSLETTER ISSUE
THE GENERALIST

Why inference is harder than you think

DATE November 17, 2026SOURCE THE GENERALISTPARTICIPANTS THE GENERALIST
In this episode
// SUMMARY

1. Key Themes

Theme: Inference as AI's "final market"

Inference scales with AI's penetration of the economy

Srivastava's core thesis is that running AI is the durable, ever-growing business, regardless of how model capabilities evolve. The newsletter author sizes it at an enormous figure.

"If AI is going to eat the world, all that's going to matter is how that AI is run. And even in a world where there is AGI, the only thing left to do will be inference: to run the intelligence itself."

"If AI takes over more of the economy, then the inference opportunity could grow to a truly ludicrous size. Tuhin's back-of-the-napkin estimate: about $2 trillion of inference spend by 2030."

Demand is outrunning even insiders' forecasts

Baseten's own planning has been consistently too conservative, which signals how steep the demand curve is.

"Baseten's January 2025 forecast versus year-end actuals was off by four times, and its forecasts have been wrong every year since 2023. It's fascinating to me how difficult it is for even extremely sharp, knowledgeable humans to adjust to exponentials."

Theme: The compute squeeze is the binding constraint on AI growth

Compute availability, not demand, may cap near-term AI expansion

The conversation frames the supply crunch as potentially the single largest constraint, which also makes multi-cloud capacity management a core competency.

"...why the compute squeeze may be the single largest constraint on AI growth right now."

"The company... spans 20 clouds and 90 regions, and was most recently valued at $13 billion following a $1.5 billion funding round."

The timestamps also show dedicated segments on "Managing compute across clouds," "The compute crunch," and "Financing inference at scale," suggesting the capital intensity of inference is a major topic.

Theme: "Owned intelligence" and the closed-to-custom customer journey

Customers progress from closed APIs to custom fine-tuned models

The author describes a flywheel in which companies deploy open-weight models, capture production data and feedback, post-train, and redeploy, building compounding advantages.

"Deploy open-weight models, generate production data, capture feedback signals, post-train custom models, redeploy, repeat. The idea is that companies that close that loop accumulate compounding advantages."

"This is a theme I hear from an increasing number of companies."

Theme: Inference is deceptively hard, creating room for specialists

Hyperscalers are structurally disadvantaged by their own org charts

Inference spans multiple disciplines that live in separate teams at large clouds, which is an opening for a focused vendor.

"Inference bridges compute, networking, observability, and workflow primitives that live in four different internal teams. A customer using a hyperscaler is effectively dealing with four different organizations at once, not just one."

The episode also includes a segment titled "Why inference is hard to build in-house," reinforcing the build-vs-buy argument (though the detail is in the audio, not the text).


2. Contrarian Perspectives

Luck is ~90% of startup success, yet the 10% of skill is indispensable

An unusually candid founder take: even a $13B-valuation CEO credits luck as the dominant factor while insisting skill still matters.

"...why Tuhin believes luck accounts for roughly 90% of startup success, while insisting that the 10% skill is still indispensable."

Even a world with AGI doesn't commoditize infrastructure businesses; it makes them more important

The consensus view tends to focus on model providers as the value layer. Srivastava argues the opposite: when intelligence becomes ubiquitous, the operational layer that runs it is what's left.

"Even in a world where there is AGI, the only thing left to do will be inference: to run the intelligence itself."

The company's breakout came from abandoning years of work

Counter to "stay the course" startup wisdom, Baseten's inflection came from a radical pivot to a product it was effectively giving away.

"Baseten's breakout began when it discarded three years of other product work in roughly six weeks to focus on an inference engine it had been giving away."


3. Companies Identified

Baseten

  • Description: Infrastructure company for AI training and inference.
  • Why mentioned: The subject of the interview, led by CEO Tuhin Srivastava; a case study in a high-growth inference pure-play.
  • Quote: "...a company building infrastructure for training and inference. When an AI model runs in production (responding to a user, powering an app, making a decision), Baseten is one of the companies that makes that happen reliably, quickly, and at scale."

Abridge

  • Description: Baseten customer (AI company).
  • Why mentioned: Named as a customer.
  • Quote: "The company counts Abridge, Cursor, and OpenEvidence among its customers."

Cursor

  • Description: Baseten customer (AI coding company).
  • Why mentioned: Named as a customer.
  • Quote: "The company counts Abridge, Cursor, and OpenEvidence among its customers."

OpenEvidence

  • Description: Baseten customer (AI company).
  • Why mentioned: Named as a customer.
  • Quote: "The company counts Abridge, Cursor, and OpenEvidence among its customers."

Hyperscalers (generic)

  • Description: Large cloud providers.
  • Why mentioned: Used as the foil for Baseten's value proposition, since their organizational structure fragments inference.
  • Quote: "The hyperscalers' blind spots are organizational."

Companies in the resource list (mentioned without surrounding commentary): Sierra, Coinbase, Etched, Cerebras, NVIDIA Groq 3 LPX (inference hardware/products), and investor firms Blackbird VC, Greylock, and Conviction. These appear only as links in the Resources section, so their specific relevance to the conversation isn't stated in the text.

  • Quote: "Resources: Sierra... Coinbase... Etched... Cerebras... NVIDIA Groq 3 LPX... Blackbird VC... Greylock... Conviction"

Union Square Cafe

  • Description: Restaurant associated with Danny Meyer.
  • Why mentioned: Source of the hospitality philosophy Baseten uses as a hiring lens.
  • Quote: "Danny Meyer's book about Union Square Cafe, Setting the Table."

4. People Identified

Tuhin Srivastava

  • Description: Co-founder and CEO of Baseten.
  • Why mentioned: The interview guest and source of the "inference as final market" thesis.
  • Quote: "Tuhin Srivastava is the Co-founder and CEO of Baseten, a company building infrastructure for training and inference."

Mario Gabriele

  • Description: Author/host of The Generalist.
  • Why mentioned: Interviewer and author of the "notebook" commentary.
  • Quote: "An idea I have been having that I think I will write a piece on soon is the trend towards intensive, personal surveillance."

Danny Meyer

  • Description: Restaurateur and author (Union Square Cafe; Setting the Table).
  • Why mentioned: His full-service hospitality philosophy shapes Baseten's hiring approach.
  • Quote: "Tuhin referenced Danny Meyer's full service philosophy as a hiring lens."

5. Operating Insights

Hire for a hospitality mindset and make customer contact daily

Baseten treats customer intimacy as a cultural trait, not just a support function.

"Alongside things like intellectual curiosity, Baseten looks for a high-end hospitality mindset. Teammates should talk to customers every day and go the extra mile. That leads to deep relationships, with one Baseten employee recently attending a customer's wedding!"

Be willing to kill existing roadmap for a signal you're already seeing

Baseten's turning point came from reallocating all effort toward a product that was already pulling demand, even though it was free.

"Baseten's breakout began when it discarded three years of other product work in roughly six weeks to focus on an inference engine it had been giving away."

Plan for exponentials by assuming your forecasts are too low

Systematically underestimating growth is a trap even for experts; build capacity and capital plans with that bias in mind.

"Baseten's January 2025 forecast versus year-end actuals was off by four times, and its forecasts have been wrong every year since 2023."


6. Overlooked Insights

Maximal-context AI could drive self-surveillance behavior

Mentioned in the "Tangents" section, this is a potentially large behavioral and market shift, with implications for data capture, privacy, and sensor/recording businesses.

"AI relies on context to do its best work and as it becomes more capable, the right amount of context for it may be: everything. That might lead to some really strange behaviors that effectively amount to state-style surveillance of yourself."

Multi-cloud footprint as a competitive asset

Baseten's spread across 20 clouds and 90 regions is mentioned only in passing but is a notable operational moat given the compute shortage.

"...spans 20 clouds and 90 regions..."