Niko Grupen

“Niko was previously at Google Brain as well. Together, they led the open-sourcing of LAB.”

“I sat down with Harvey Co-Founder & CEO Winston Weinberg, Co-Founder & President Gabe Pereyra, & Head of Applied Research Niko Grupen at the company's San Francisco HQ”

“LAB, the Legal Agent Benchmark, the first open-source benchmark for measuring AI agent performance on real-world legal tasks. It covers 1,200+ tasks across 24 practice areas with 75,000+ rubric criteria”

“People aren't really thinking about performance in terms of just, like, quality maxing anymore. The frame has shifted to quality per dollar and quality per second.”

“Niko Grupen: Head of Applied Research at Harvey; former Google Brain researcher; joined Harvey nearly three years ago”

“I think this year is the year that we see intelligence at an individual level kind of brought into intelligence at an organizational level... How do lawyers collaborate with lawyers on our platform? How do human agent teams collaborate?”

“We launched Lab, which is our legal agent benchmark. It's a benchmark for measuring the performance of agents on real world legal tasks.”

“Harvey's response is the Legal Agent Benchmark (LAB), open-sourced in May, which tests long-horizon agentic work across 1,200+ tasks and 24 practice areas, with collaborators including NVIDIA, OpenAI, Anthropic, Mistral, and DeepMind.”

“Co-founder Gabe Pereyra & Niko Grupen, Harvey's Head of Applied Research, join us next to break down LAB, the new benchmark data, and the technical architecture behind legal agents at production scale.”