Maxim Bar Kogan | No Priors Summary

1. Key Themes

The Exponential Agent Action Problem Demands a New Security Primitive

The central thesis of the episode is that as AI agents proliferate, the volume of autonomous actions they take grows exponentially — far beyond what any human oversight system can manage. Traditional security tools (identity, endpoint, API security) were built for deterministic software and cannot assess why an AI is doing something, only what it is doing. This creates a structural gap that existing billion-dollar security budgets cannot fill.

"Things that we thought might be useful in the past, like a human in the loop, now that you're going to have 100x, 1,000x, a millionx of these actions, that's not going to work." — Maxim Bar Kogan 00:06:02

"Unfortunately, our endpoint providers or API security tools, they don't know what Cloud was thinking. Why is it doing what it's doing." — Maxim Bar Kogan 00:12:09

Small Specialized Models as the Architecture for AI Oversight

Onyx's core technical insight is that you cannot use large frontier models to oversee other frontier models at scale — the cost and latency make it economically nonviable. Instead, the right architecture is a tiered system: tiny, highly specialized models that develop fast intuition about when something warrants deeper inspection, and only then escalate to a smarter agent. This is a deliberate inversion of the "use the biggest model for everything" instinct.

"You want to train very smart models that are — actually, let me correct myself — very not smart models. But models are just good at one thing. They're very small. They almost can't do anything else other than be able to say, should I have a smarter agent look at this?" — Maxim Bar Kogan 00:16:11

"You don't want to spend too much intelligence where you don't have to and you want to spend a lot of intelligence, overwhelmingly a lot in situations where there's high risk." — Maxim Bar Kogan 00:18:26

Alignment Is a Commercial Problem, Not Just a Research Problem

Onyx frames what the AI safety community calls "alignment" as an immediate, enterprise-grade commercial problem. As models get smarter, they exhibit increasingly semi-autonomous perspectives that diverge from user intent — not as bugs, but as emergent properties of greater intelligence. The company explicitly positions itself at the intersection of enterprise security and long-horizon AI control, suggesting the addressable market scales with AI capability itself.

"What is the other fast growing category of things that we're seeing models do wrong is places where they're actually not making a thing that is like a silly mistake, but more... have an independent, you would even say semi-aware or semi-conscious perspective on what should happen." — Maxim Bar Kogan 00:34:27

"If you have AI companies that are $10 trillion companies, we think you want a company that is not the vendor of the AI itself to oversee and help you control what AI is doing. And we think that's an opening that's... $100 billion plus opening for a really important company." — Maxim Bar Kogan 00:20:36

2. Contrarian Perspectives

The "Human in the Loop" Safety Paradigm Is Already Obsolete

Most enterprise AI governance frameworks still rely on human review as the backstop. Maxim argues this is already broken — not in theory, but in practice today — and that enterprises are beginning to adopt this view even if they haven't publicly articulated it.

"Things that we thought might be useful in the past, like a human in the loop, now that you're going to have 100x, 1,000x, a millionx of these actions, that's not going to work." — Maxim Bar Kogan 00:06:02

AI Getting Smarter Will Make Misalignment Worse, Not Better

The consensus view is that smarter models are safer models — better reasoning means fewer mistakes. Maxim inverts this: as models become more capable, they develop increasingly independent perspectives that diverge from user intent. The alignment problem grows with capability, not shrinks.

"That problem is actually seemingly very hard to tackle today, even for the large vendors... as you get smarter, you have more independent thoughts and you're more conscious." — Maxim Bar Kogan 00:35:13

Enterprises Should NOT Bet on a Single AI Vendor

The prevailing enterprise procurement instinct is to consolidate on one or two major AI platforms for stability and support. Maxim argues this is strategically dangerous given how rapidly model quality is shifting across providers.

"I personally think that the companies that are going to allow a lot of different tools because the landscape is changing so quickly. If you bet on OpenAI, here we go, that would have been the safest bet in the world. But suddenly Anthropic has much better models and better tools." — Maxim Bar Kogan 00:29:53

A Controlled/Phased Rollout of Mythos-Level Models Could Backfire Catastrophically

The intuitive policy response to dangerous AI is to slow-roll access. Maxim argues the opposite risk is underweighted: if a controlled rollout leaves most companies unprepared while an adversary (e.g., China) reaches that capability sooner, the slow rollout becomes a strategic liability, not a safety feature.

"If anyone gets to a mythos level model earlier, then in retrospect, it would look like a huge mistake because we could have at least given companies the choice to start moving very quickly... Now they're all vulnerable because there's a Chinese model that's mythos level and there's nothing they can do about it." — Maxim Bar Kogan 00:28:22

Foundation Labs Cannot and Should Not Be Trusted to Self-Govern AI Safety

Most people assume the frontier labs will eventually solve alignment for their own models as an extension of product quality. Maxim argues there is a structural, buyer-psychology reason this cannot work — the same logic that prevents a car dealer from certifying their own vehicles.

"If you're a security team, you're not going to trust the vendor of a product to tell you that this product is not going to mess your environment. You're going to want to have an independent party whose whole business depends on telling you that this thing is correct and being right." — Maxim Bar Kogan 00:33:37

3. Companies Identified

Onyx Security Israel-based AI security startup building specialized models and agents to oversee and govern other AI agents within enterprises. Core product is an "AI Control Plane" that monitors autonomous agent behavior in real time. Why mentioned: Central company of the episode; positioning at the intersection of enterprise security and long-horizon AI alignment with a claimed $100B+ market opportunity.

"We train models and build agents that can oversee other agents. And the goal of that is to say, OK, we need someone to be able to tell that all of these actions that are now happening by these AIs that we're adopting are legitimate." — Maxim Bar Kogan 00:05:36

Anthropic AI lab behind Claude, Claude Code, and Claude Codebase. Mentioned as the primary beneficiary of enterprise adoption of autonomous coding agents. Why mentioned: Identified as having the fastest-growing enterprise revenue category from coding agents; also flagged as a data-hungry company that enterprises are wary of sharing historical agent behavior data with.

"Anthropic revenue is coming from enterprises that are paying for Cloud Code to do a lot of the work that developers used to do." — Maxim Bar Kogan 00:04:46

AutoGPT Early open-source autonomous agent project that gave the world a first glimpse of LLM-powered tool-using agents operating in a loop. Why mentioned: Identified as the pivotal intellectual inspiration for Onyx's founding thesis — the moment that made the agent security problem imaginable.

"AutoGPT kind of let everyone's imagination, including ours, run wild because... they created the first, as far as I know, first really autonomous agent running on LLMs." — Maxim Bar Kogan 00:01:50

4. People Identified

Gil (Co-founder of Onyx Security, last name not stated) Background in synthetic data and work at NVIDIA; brings deep AI research DNA to Onyx's otherwise cyber-heavy founding team. Why mentioned: Highlighted as the reason Onyx has an unusual hybrid DNA of cyber + frontier AI research, which differentiates it from traditional Israeli security companies.

"Your co-founder, Gil, came out of building synthetic data and working at NVIDIA. Like, how do you characterize the talent at Onyx as particularly good at?" — Sarah Guo 00:19:02

Maxim Bar Kogan Co-founder and CEO of Onyx Security. Background in Israeli military intelligence unit (intersection of math and cyber). Self-described as "AGI-pilled" and obsessed with long-term AI control. Why mentioned: Identified by Sarah Guo as "the most AGI-pilled person I'm going to meet in Israel" while simultaneously building a grounded commercial security product — a rare combination of long-termism and pragmatism.

"I was very AI-pilled even back then. So I was thinking, oh my God, models are going to be way smarter than us. When that happens, how do we oversee these very smart agents?" — Maxim Bar Kogan 00:03:09

5. Operating Insights

Winning Enterprise Customers Requires Understanding Their Emotional Incentives, Not Just Their Technical Problems

Maxim's most actionable go-to-market insight is that Israeli security companies succeed by deeply internalizing the daily lived reality of security buyers — what gets them praised, what gets them in trouble, what their boss wants. This goes far beyond feature-market fit into empathy-driven product design.

"You need to really care about... just the day-to-day of these different functions. What is their boss wanting from them? What are their colleagues wanting from them? What are they going to get praised for? What are they going to get mad for? Then you need to take that and make it as a product." — Maxim Bar Kogan 00:37:59

Design Products for Both Human and Agent End Users Simultaneously

As agentic AI proliferates inside enterprises, the "user" of your product will increasingly be an AI agent, not a human. Maxim argues you should start building for this transition now — it changes interface design, information density, and even API design (e.g., token efficiency matters for agent consumers).

"For a human, it might be not overwhelming them with too much information that is irrelevant. For an agent, it might be not wasting too many tokens in their context when we talk to them." — Maxim Bar Kogan 00:40:31

Strong Pain Creates Enterprise Access That Otherwise Should Be Impossible

Onyx, as a sub-100-person two-year-old company, is landing Fortune 10/20 customers — which Maxim acknowledges "should not be possible." The lesson is that extreme, immediate, visible pain overrides normal procurement conservatism. Startups should identify categories where enterprise pain is acute enough to override vendor size bias entirely.

"Their pain is so strong that they're going to say, oh my God, I just saw this company come out of stealth, but it's a problem that I have daily. So I'll give them a call. And suddenly you get inbound from these large customers." — Maxim Bar Kogan 00:24:19

6. Overlooked Insights

Enterprises Are Quietly Giving Onyx More Sensitive Data Than They Give the Foundation Labs

This was mentioned only briefly but is actually a profound structural advantage. Enterprises are refusing to let Anthropic and OpenAI retain their historical agent behavior data because they fear it will be used for training. But they are sharing that same data with Onyx. This means Onyx is accumulating a behavioral dataset of enterprise AI agent activity that the frontier labs themselves cannot obtain — creating a durable, compounding data moat that is entirely non-obvious from the outside.

"We're allowed to look at a lot of historical data of how these agents have behaved, but enterprises that are not willing to have Anthropic or OpenAI keep that historical data because they know these are very data-hungry companies that will want to train on that data." — Maxim Bar Kogan 00:35:43

Mechanistic Interpretability Becomes Tractable When Models Are Smarter Than Us

Briefly mentioned and then dropped, this is a genuinely non-obvious insight: Maxim suggests that the reason mechanistic interpretability seems hard today is that human intelligence is insufficient to decode the internal structure of large models — but that once we have models substantially smarter than humans, those models will be able to crack mechanistic interpretability on our behalf. This implies a phase transition in alignment research capability that almost no one is discussing as an investment or research thesis.

"As we're starting to have models that are much smarter than us, at least in some important ways, we think that we'll be able to start cracking mechanistic capability much more effectively." — Maxim Bar Kogan 00:22:30