SaaStr 849: How We Built Our AI VP of Customer Success with SaaStr's CEO and CAIO
- 01Custom Vibe-Coded Agents Are Replacing Off-the-Shelf SaaS for Specific Workflows
- 0270% Reduction in Human Hours Is the Real ROI Benchmark
- 03The Real Unlock Is Personalized, Real-Time Customer Engagement at Scale
Participants: Jason Lemkin (CEO, SaaStr), Amelia Lerutte (CAIO, SaaStr)
1. Key Themes
Custom Vibe-Coded Agents Are Replacing Off-the-Shelf SaaS for Specific Workflows
SaaStr built "QB" (Quby), an AI VP of Customer Success, using Replit — no engineers involved. The agent replaced a paid SaaS portal they'd used for two years and dramatically outperformed it. The core argument is that for high-touch, repetitive workflows, custom-built agents beat generic software because they can be iterated in real-time based on actual customer behavior.
"We built this with no engineers and it adds dramatic value to our little tiny team doing eight figures of revenue." — Jason Lemkin 00:03:26
"At the very beginning when I wanted him to know all the contracts, all the customers who was going to need access to the portal... our old portal didn't even tell us if they logged in or not." — Amelia Lerutte 00:13:15
70% Reduction in Human Hours Is the Real ROI Benchmark
The most concrete metric shared: QB reduced billable agency and production hours by approximately 70% over Q1 2025 vs. Q1 2024. The cost of building QB was a few thousand dollars. The savings were tens of thousands. This is a repeatable benchmark that operators should test against their own CS workflows.
"I saw something like a 70% decrease of literally billable hours... these are agencies and production teams we work with... that's literally thousands upon thousands of dollars each month for the last three months." — Amelia Lerutte 00:17:37
"It's not even comparable. It's like apples to oranges. We've spent maybe a couple thousand, but we've saved tens of thousands of dollars on human input hours." — Amelia Lerutte 00:19:24
The Real Unlock Is Personalized, Real-Time Customer Engagement at Scale — Not QBRs
QB flips the traditional QBR model. Instead of once-a-quarter generic reports, QB delivers individualized, real-time status updates to every customer. 100 sponsors got personalized emails in 10 minutes — something a human team couldn't do in a week. Customer engagement (portal logins, task completion) went up more than 10x.
"QBRs have always pissed me off... why am I waiting once a quarter for super generic stats? You're not telling me anything in real time. Everything in a QBR has already happened. Whereas QB, the agent, is in real time." — Amelia Lerutte 00:48:23
"QB was done with this in 10 minutes. 100 people got a highly customized, personalized email saying exactly what they needed to do, exactly where they are." — Amelia Lerutte 00:24:32
2. Contrarian Perspectives
Most Post-Sales Software Actively Hurts Retention Because It Burdens the Customer
The conventional wisdom is that investing in a customer portal improves the post-sales experience. Jason argues the opposite: most portals ask too much of customers and result in non-compliance, which looks like customer disengagement but is actually vendor-imposed friction.
"So many of these post-sales products, post-sales motions put too much of a burden on the customer. And so no one just does it... people didn't want to use it. They didn't want to use it. So that's another reason to do it and to vibe it yourself — you're asking too much of the customer. You're making their lives harder after they give you money." — Jason Lemkin 00:26:40
You Don't Need Prompt Engineering Skills Anymore — That Idea Is Already Dated
The prevailing belief in tech circles is that prompt engineering is a valuable, emerging skill set. Jason flatly contradicts this.
"In 2026, you don't need to be a prompt engineer. That is dated... You just need to talk with Claude or ChatGPT. We prefer Claude. And just say, I want to build the spec for an application... The agents are so good. AI is so good. That's dated." — Jason Lemkin 00:32:10
Even a 10-Minute Vibe-Coded Build Can Outperform 2 Years of Paid SaaS
The assumption is that battle-tested SaaS tools are more reliable, feature-rich, and trustworthy than quickly-built custom apps. SaaStr's experience inverts this.
"I could save time by doing this. I didn't realize until after the fact how much time it would save us, how involved Kiwi could get with our customers, how much of a better experience it would be not only for us, but for them." — Amelia Lerutte 00:09:04
Never Store Sensitive Data Directly in Your Vibe-Coded Agent — "Agent Hopping" Is the Correct Architecture
Most builders assume the agent should have access to all relevant data to function well. Amelia argues the opposite — keeping data segmented and forcing the agent to "hop" across APIs (Salesforce, Clerk, Mizabo) is both more secure and architecturally sounder.
"Never give it to your agent directly. Never, ever, ever... QB directly does not have that data. He has to call up Salesforce. So he doesn't have 100 contracts just sitting in his knowledge base, which would be scarier." — Amelia Lerutte 00:38:08
The Best AI Agents Are Never "Set and Forget" — Daily Human Maintenance Is Non-Negotiable
The popular narrative around AI agents is that they run autonomously. Jason and Amelia are emphatic that without daily human oversight, these agents will fail.
"There is no set and forget... Whether it's things that will break... Or there are regressions... You need to budget... every day you have to check in on how they're working... anything that's as good as QB — it is great — and between Amelia and me, we're checking in on it every day. Just like your humans." — Jason Lemkin 00:51:55
3. Companies Identified
Replit
- Description: Vibe-coding platform used to build and host QB end-to-end
- Why mentioned: Primary tool for building QB; their AI agent handled the full development cycle with no engineers. Chad from Replit was also specifically named as attending SaaStr Annual.
-
"We took our spec. We loaded it into our vibe coding platform. Again, you can use Replit, Lovable, Vercel V0, any of them." — Amelia Lerutte 00:34:55
Salesforce
- Description: CRM used as the secure system of record and data hub for QB's agent hopping architecture
- Why mentioned: Central to the security model; QB calls Salesforce rather than storing sensitive contract data internally. Also used as the hub across all SaaStr agents.
-
"We use Salesforce as our hub for all our agents... there is inherently an additional layer of security by having that be our system of record." — Jason Lemkin 00:39:13
Anthropic (Claude)
- Description: AI model provider; Claude used for spec writing, co-reviewing assets, and general agent reasoning
- Why mentioned: Explicitly preferred over ChatGPT for spec writing and iteration; also attending SaaStr Annual
-
"We prefer Claude. And just say, I want to build the spec for an application... iterate it." — Jason Lemkin 00:32:10
Clerk
- Description: Single sign-on (SSO) authentication tool
- Why mentioned: Used for customer login management in QB; keeps user identity data separate from the vibe-coded agent
-
"We use Clerk in this instance. You can use other things. But between how we did single sign-on and how we built it into our Vibe Coded app, we got a whole lot more granular visibility on what people were actually doing." — Amelia Lerutte 00:13:15
Resend
- Description: Email delivery infrastructure
- Why mentioned: Used by QB to send individualized sponsor emails at scale
-
"QB was done with this in 10 minutes. 100 people got a highly customized, personalized email... you can see these are all at 11:11, 11:12. He's got to go to Resend and send these." — Amelia Lerutte 00:24:32
Lovable / Vercel V0
- Description: Alternative vibe-coding platforms to Replit
- Why mentioned: Named as having built-in single sign-on features that make customer-facing deployments easier; attending SaaStr Annual
-
"I know for a fact it's easier in Lovable and V0 because they have a built-in." — Amelia Lerutte 00:41:28
Zapier
- Description: Automation and API connection tool
- Why mentioned: Used for API key management and connecting QB to external tools securely
-
"We have Zapier keys that are secretly stored, other API keys that are stored that we just load different sets of data and things from other tools." — Amelia Lerutte 00:41:04
4. People Identified
Amelia Lerutte
- Description: CAIO of SaaStr; builder of QB (AI VP of Customer Success) and the AI VP of Marketing agent
- Why mentioned: Built two production AI agents with no engineering background; achieved 70% reduction in billable hours; architected the "agent hopping" security model; driving SaaStr's entire agentic strategy
-
"I added things to him all the time, every week. Sometimes it was daily for a while in the beginning... he didn't start fully agentic. That was actually not even my end goal." — Amelia Lerutte 00:09:04
Anson from Lovable (referenced as "Obobo")
- Description: Representative from Lovable (vibe-coding platform)
- Why mentioned: Named as a speaker/attendee at SaaStr Annual 2026; part of the vibe-coding ecosystem SaaStr is building around
-
"Anson from Lovable, Chad from Replit, Jean from Vercel is going to be there on Tuesday." — Amelia Lerutte 00:04:20 (from final segment 01:04:20)
5. Operating Insights
Deploy to a Tiny Pilot Group First, Then Scale — Never Go Straight to Full Production
Amelia's rollout method is precise: pick one customer per tier, deploy, watch what breaks, fix it, then expand. This is distinct from standard software QA because real users reveal failure modes (like session timeouts) that internal testing never catches.
"I basically picked one customer at each of our levels. And I said, okay, I'm going to deploy to them. I'm going to see what breaks, what doesn't... I learned in the first week that we had a gold sponsor submitting some tasks and they're like, it's not really working... I was like, send me a screenshot. I loaded into the agent and I was like, hey, what happened?" — Amelia Lerutte 00:42:26
Write Your Spec in Claude First, Then Port to Your Vibe-Coding Platform — This Saves Time and Token Costs
Starting in a vibe-coding platform directly is more expensive and less structured. Writing the spec in Claude first produces a more complete blueprint, reducing iterative back-and-forth once you're in the builder.
"We both have found you get a double benefit if you write it in Claude and then get a spec out of it and then put that in the agent and then kind of tweak it again... If cost is one of your concerns, I would say you want to get your spec closer to this one than my original one." — Jason Lemkin 00:32:10 and Amelia Lerutte 00:34:25
Have the Agent Send You a Daily Email Status Update — This Is Your Bug Detection System
Rather than manually checking whether the agent is working, build a daily self-reporting email into the agent from day one. This catches regressions and broken features before customers do.
"My hack is make sure you have them send you an email status update every single day. Have them build that because you'll see stuff break in the email and then you'll realize things are broken that you can't see." — Jason Lemkin 00:53:20
At High Deal Sizes, Go to Where the Customer Wants to Work — AI Frees You Up to Do This
Counterintuitively, the agent doesn't standardize customer interaction — it enables more flexibility. Because QB handles the 90% routine work, the human team has capacity to meet top-tier accounts in Slack, Google Meet, or wherever they prefer.
"For our very top tier accounts... we go to where they want. Salesforce likes to work in Slack... The Google team likes to be on Google Meet and Gemini and Google chats... QB is not the end-all be-all... it's freed up a lot of our time to do that too." — Amelia Lerutte 01:00:43
6. Overlooked Insights
The "90/10 Rule" for Build vs. Buy Is a Deployable Framework Most Operators Are Ignoring
Jason briefly mentions a specific decision framework almost in passing: buy off-the-shelf for 90% of your stack, vibe-code only the 10% where generic tools create friction or gaps. This is actually a highly actionable capital allocation principle for any operator or portfolio company — it tells you exactly when custom AI tooling has positive ROI and when it doesn't. Most people in this conversation glossed over it, but it directly answers the "should I build or buy" question that paralyzes most operators.
"If you can do this all in Notion and it works great, don't build it yourself... The 90-10 rule applies. Buy 90%, vibe 10% at most. But if you decide to build something like this and you get into production... your ability to enhance it for what your customers want, it is magical." — Jason Lemkin 00:56:07
Giving a Vibe-Coding Agent a Competitor's or Reference URL Lets It Clone the UX Instantly — This Is Massively Underutilized
Amelia mentions almost offhandedly that you can paste a live URL (like sastersponsors.com) directly into a vibe-coding agent and it will functionally replicate the design and structure. This is an extraordinary shortcut that almost no one is using systematically — it means any operator can start from a production-quality reference rather than a blank slate, compressing weeks of design iteration into minutes.
"You can actually give this URL to your vibe coding agent and say, hey, this is one I saw that I liked... you can't actually give it URLs like this because it can pretty much copy it. So you can use it as kind of like a format too." — Amelia Lerutte 00:06:53