The AI Product Playbook: 38… | The AI Corner Summary

1. Key Themes

Theme: Agent-first distribution (MCP servers) can beat in-app AI, but only if your users are developers

Agents now out-produce humans inside PostHog

Substantiation: "According to their October 2026 lessons list, there are 5x more people who use the MCP server on a daily basis compared to PostHog AI."
"In 30 days, agents created 252,000 charts using MCP, while people created 99,000 in the web app"
"PostHog studied 63 million MCP tool calls (90 days to September 16, 2026)" and found "Running SQL and reading the data schema made up 32% of all calls"

The agent-first playbook is audience-dependent

Substantiation: "Going with MCP worked well for PostHog because its users are mostly software teams, people who are already working inside AI coding tools."
PostHog says "a custom in-app agent is the better choice in 3 cases: 1. Your users aren't engineers. 2. Compliance blocks outside agents. 3. You need full control of the experience."
"If yes, consider building an MCP server first, which is easier to build and maintain anyway (lesson 2)."

Theme: Context, not the harness or reasoning, is the durable moat for agents

Harnesses are disposable

Substantiation: "Lesson 11 details how PostHog AI went through 3 harnesses in a year": "1. A coordinator that routed work to sub-agents 2. One agent loop with 44 tools written by hand 3. Anthropic's Claude Agent SDK, plus PostHog's own tools and skills"
"PostHog's advice is to not treat a harness as an advantage... Better to spend your effort on context."

Most agent failures are context failures

Substantiation: "Lesson 27 says that context failures are behind many agent failures."
For the install wizard: "With 20+ PostHog products, 17+ SDKs, and 25+ frameworks, the possible setups are in the thousands, and the agent kept choosing wrong." Adding "docs, source code, SDK reference, and example apps... helped more than the reasoning and planning changes it had tried."

Context delivery should be on-demand and kept current

Substantiation: "PostHog AI's system prompt included every event and property in a user's data. PostHog replaced that list with a tool that the agent can call when it needs to look up an event or property (lesson 29)."
"PostHog now has agents that look at merged pull requests and open a doc pull request when needed to keep the docs current."

Theme: AI features must clear a "beats Claude/ChatGPT" bar before you build them

The first filter: does this beat a general chatbot?

Substantiation: "Lesson 10 indicates Claude as the default go-to, and so according to PostHog, you must answer the question 'How does this beat Claude?'"
"For PostHog, the answer is context: the product's features, users' data, the tools, and even the skills that PostHog has written."
"If it can't, then you're just building a worse version."

A simple three-signal test for what to hand to AI

Substantiation: "lesson 3 gives 3 signs that a job is worth passing to AI (you only need 1): 1. Takes more than 30 seconds 2. Needs a language users don't know (like SQL) 3. Users repeat the task at least 20 times"
Evidence: PostHog's install wizard "can do 2 hours' worth of work in 8 minutes"; wizard users reached first event in "about 1.9 hours compared to 3.8 hours" and "14.2% of wizard users convert to paid, compared with 2.6% of others."

Theme: Trust and reliability beat capability

Users want reliability, not more features

Substantiation: PostHog's early complaints were "Inconsistent performance, Unexpected failures, Generic error messages, Unclear limits, No sign of uncertainty." "Users weren't asking for more features. As PostHog puts it, 'capabilities are worthless if they aren't reliable.'"

Show the work, including failures

Substantiation: "Lesson 36 contains an example of when PostHog reversed its decision to hide the agent's reasoning and failed tool calls. PostHog's reason was that it looked messy. But according to the users, that only made it hard for them to trust the AI. PostHog now streams everything to users."
"Lessons 19 and 37... more of the screens now are for approving, reviewing, and undoing the agent's work."

Theme: Agent cost economics and security are underpriced risks

Know your per-run unit cost; subagents are a hidden cost multiplier

Substantiation: "One wizard run cost PostHog an average of about $6.67 in February 2026." "Across wizard runs, cost generally rises with the number of turns, from about $2 to about $11."
"A subagent... doesn't share the main agent's cache, so the subagent has to read everything again, adding to the cost." In one extreme case, "the wizard spawned 5 subagents, then idled for 5 minutes... That whole process took about 15 minutes, and it was all a waste. The real work then took about 3 minutes."

Prompt injection is the unsolved security risk for autonomous agents

Substantiation: "OWASP ranks prompt injection first in its 2025 and 2026 LLM Top 10 lists. It also says that mitigation doesn't remove prompt injection entirely."
"PostHog's filter alone can't solve this, and self-driving can open PRs without a human approving them."

2. Contrarian Perspectives

Build for agents first, not your own in-app AI

Most companies default to embedding a chatbot or copilot in their own UI. PostHog argues the higher-leverage move may be making your product easy for other agents to use.
Evidence: "Lesson 2 says to consider whether you need to build your own AI feature inside your app. Making it easy for other agents to use your product might be more worth it." Daily MCP users outnumber PostHog AI users 5x, and agents "created 252,000 charts" versus "99,000" by people in the web app.
Caveat the author adds: "The real question is how many of those 252,000 charts were actually used?"

Sometimes the right answer is not to build the AI feature at all

Counter to the "add AI everywhere" consensus: "Lesson 1's point is you may not always need to add an AI feature in your product, especially when it could make your product worse by getting slower or unreliable."
Example: "a document chat feature, which every AI tool can already do, isn't worth adding. Not to mention, your users may already have their own AI agent or model that they trust."

Don't treat your harness as a competitive advantage

Counter to the popular view that agent infrastructure is the moat: "PostHog's advice is to not treat a harness as an advantage."
The author partially disagrees ("I wrote an earlier post where I argued that the harness decides whether an agent works at all"), reconciling it as: "The harness affects quality (my point), but it won't set you apart from the rest, which is PostHog's point."

Hiding messy reasoning to look polished backfires

Counter to the instinct to clean up AI UX: PostHog hid reasoning and failed tool calls because "it looked messy," but "that only made it hard for them to trust the AI." They now "stream everything."

Skepticism toward PostHog's own data and incentives

The author flags that the 14.2% vs 2.6% conversion stat may be selection bias: "PostHog has no control test... Those using the wizard may already be familiar with the command line terminal. They may already be more technical and may have paid anyway, which can explain part of the gap."
On incentives: "it helps if you remember what PostHog is selling. For example, lessons 22 (AI observability) and 24 (tracking agent usage) match PostHog's AI Observability and MCP Analytics products."

3. Companies Identified

PostHog

Description: Product analytics company that shipped 10 AI products.
Why mentioned: Primary case study; source of all 38 lessons on building AI products.
Quotes: "PostHog builds product analytics, and its 10 AI products reach nearly 100,000 people a month. In October 2026 it published 38 lessons from building them." Also: "PostHog's own last lesson says to expect half of them to be wrong in a year."

Anthropic (Claude, Claude Code, Claude Agent SDK)

Description: AI lab behind Claude models and agent tooling.
Why mentioned: The benchmark competitor for any AI feature and the basis of PostHog's current harness.
Quotes: "Lesson 10 indicates Claude as the default go-to, and so according to PostHog, you must answer the question 'How does this beat Claude?'" and PostHog's third harness is "Anthropic's Claude Agent SDK, plus PostHog's own tools and skills." Also, "reliable tool use (Anthropic's Claude 4) made the old setup outdated."

OpenAI (ChatGPT, Codex, o4-mini)

Description: AI lab behind ChatGPT, Codex, and o4-mini.
Why mentioned: Alternative general chatbot users may already use; its cheap reasoning model drove a harness rebuild.
Quotes: "Cheap reasoning (OpenAI's o4-mini)... made the old setup outdated." And "Note what your AI feature has that a general chatbot like Claude or ChatGPT doesn't have."

Intercom (Fin)

Description: Customer support software company with an AI support agent, Fin.
Why mentioned: Example of defending against an AI-native competitor by cannibalizing your own pricing model.
Quotes: "PostHog's example is Intercom, which it says 'probably' pivoted so hard into Fin, its AI support agent, for this reason. In Intercom's case, an AI that answers the tickets means fewer human agents are needed. Fewer human agents mean fewer Intercom seats, and Intercom charges per seat."

HubSpot for Startups

Description: Sponsor; hosting the IGNITION 2026 founder summit (Nov 10, San Francisco).
Why mentioned: Sponsored placement (not editorial endorsement).
Quotes: "IGNITION 2026 by HubSpot for Startups: the summit for AI-native founders, Nov 10 in San Francisco," described as "a curated room of 200, mostly Seed and Series A founders and GTM leaders."

OWASP

Description: Open application security organization publishing the LLM Top 10.
Why mentioned: Authority validating prompt injection as the top LLM security risk.
Quotes: "OWASP ranks prompt injection first in its 2025 and 2026 LLM Top 10 lists."

Microsoft (via related-reading link)

Description: Tech giant; referenced only in a "keep reading" headline.
Why mentioned: Peripheral pointer to related content.
Quotes: "Satya Nadella said every agent needs its own computer"

Google (via related-reading link)

Description: Tech giant; referenced only in a "keep reading" headline.
Why mentioned: Peripheral pointer to related content on agent memory.
Quotes: "Google's agent broke a 56-year math record. Yours forgets yesterday"

4. People Identified

Ruben Dominguez

Description: Author of the newsletter post at The AI Corner.
Why mentioned: Synthesized and sorted PostHog's 38 lessons by durability.
Quotes: "We sorted all 38 so you can see which ones to build on and which ones to revisit when the next model ships." Also: "Anyway, if I had to pick 1 lesson to start with, I'd implement lesson 10 and ask how the AI feature could beat Claude."

Sean Ellis

Description: Creator of the 40% "very disappointed" product-market fit benchmark.
Why mentioned: His benchmark was used to evaluate PostHog's own AI bet.
Quotes: "Only 23% said they'd be very disappointed, which was way lower than the 40% benchmark Sean Ellis set for product-market fit."

Satya Nadella

Description: CEO of Microsoft.
Why mentioned: Referenced only in a related-reading headline.
Quotes: "Satya Nadella said every agent needs its own computer"

5. Operating Insights

Run the three-sign test, then the "beats Claude" test, before building any AI feature

Only build if the job (1) takes over 30 seconds, (2) needs a language users don't know, or (3) is repeated 20+ times, and if you can name what you have that a chatbot doesn't (your data, tools, skills).
Quote: "Check the AI feature against the 3 signs in lesson 3. If it hits none of the signs, don't build it. Note what your AI feature has that a general chatbot like Claude or ChatGPT doesn't have (like your users' data or your own tools)."

When an agent fails, debug context first, and budget for harness rebuilds and cost per run

Treat the harness as replaceable, check context before touching reasoning, compute cost per run, and re-test subagents on each new model.
Quote: "Expect to rebuild your harness, and don't count on it as your advantage. When the agent fails, check its context first before working on the reasoning. Calculate your cost per run, and test the subagents again on current models."

Validate AI bets like new products and pre-fund your own disruptor

Define success criteria and runway before building; survey your own team afterward; document how an AI-native competitor would beat you and build that yourself.
Quote: "Write the criteria and runway down before building anything, then run the survey on your team. Write down how an AI-native competitor would beat you, then fund your own version. Always treat public inputs as untrusted, especially if an agent can act without approval."

6. Overlooked Insights

Product UI is shifting from "doing" screens to "approving" screens

Lessons 19 and 37 get only a sentence, but they imply a redesign of where product effort goes once agents do the work.
Quote: "Basically, more of the screens now are for approving, reviewing, and undoing the agent's work."

Docs that fall behind the product silently degrade your agents, and agents can fix it

A brief note on keeping agent knowledge fresh, including an automated loop that is cheap to replicate.
Quote: "Lesson 30 says that when the product changes faster than the docs, your agents will start working from outdated information. PostHog now has agents that look at merged pull requests and open a doc pull request when needed to keep the docs current."