#298 Ryan Kolln: How Appen Trains the World's Most Powerful AI Models
Eye On A.I.6 Nov 2025

#298 Ryan Kolln: How Appen Trains the World's Most Powerful AI Models

This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.

Visit https://agntcy.org/ and add your support.


How do the world's most powerful AI models get trained and trusted at scale, and what does that really take from data to deployment?

In this episode, Appen CEO Ryan Kolln joins Eye on AI to unpack how rigorous human evaluation, culturally aware data, and model-based judges come together to raise real-world performance.

In this episode of Eye on AI, host Craig Smith speaks with Ryan Kolln, CEO of Appen, about building evaluation systems that go beyond static benchmarks to measure usefulness, safety, and reliability in production. They explore how human raters and AI evaluators work in tandem, why localization matters across regions and domains, and how quality controls keep feedback signals trustworthy for training and post-training.

Ryan explains how evaluation feeds reinforcement strategies, where rubric-driven human judgments inform reward models, and how enterprises can stand up secure workflows for sensitive use cases. He also discusses emerging needs around sovereign models, domain-specific testing, and the shift from general chat to agentic workflows that operate inside real business systems.

Learn how leading teams design human-in-the-loop evaluation, when to route judgments from models back to expert reviewers, how to capture cultural nuance without losing universal guardrails, and how to build an evaluation stack that scales from early prototypes to production AI.


Stay Updated:
Craig Smith on X: https://x.com/craigss
Eye on A.I. on X: https://x.com/EyeOn_AI

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(373)

From Zero to 150 Robots in Just 20 Months | Mike LeBlanc, Foundation Future Industries

From Zero to 150 Robots in Just 20 Months | Mike LeBlanc, Foundation Future Industries

Most humanoid robot companies are still running curated demos in replica environments. Foundation Future Industries is running 150 robots on real automotive production lines in Georgia, and heading to...

19 Aug 1h 6min

Why People Are Paying 10x More for AI - and What That Means for the Chip Market | Sid Sheth, d-Matrix

Why People Are Paying 10x More for AI - and What That Means for the Chip Market | Sid Sheth, d-Matrix

The AI chip market looks monolithic from the outside - NVIDIA dominates, and everyone else is fighting for scraps. But d-Matrix's CEO Sid Sheth argues that the market is quietly splitting into two dis...

17 Aug 50min

American Companies Have 36 Months to Go AI-Native or Get Left Behind | Drew Cukor, TWG AI

American Companies Have 36 Months to Go AI-Native or Get Left Behind | Drew Cukor, TWG AI

The same tools that slowed the U.S. military down in Afghanistan (PowerPoint, Excel, email, and Word) are now slowing American businesses down in the AI race. Drew Cukor spent 30 years as a Marine int...

13 Aug 58min

In 5 Years, 90% of What You Use AI For Will Run on Your Smartphone | Paolo Ardoino, Tether

In 5 Years, 90% of What You Use AI For Will Run on Your Smartphone | Paolo Ardoino, Tether

Hundreds of billions of dollars are flowing into AI data centers right now, and Paolo Ardoino, CEO of Tether - the company behind the world's most widely used stablecoin with 573 million users - think...

10 Aug 58min

AI Agents Fixing Your IT Before You Even Know Something Broke | Erhan Giral & Ryan Manning, BMC Helix

AI Agents Fixing Your IT Before You Even Know Something Broke | Erhan Giral & Ryan Manning, BMC Helix

Most enterprise IT teams spend the majority of their time fighting the same fires repeatedly. BMC Helix is building the AI system that handles those fires automatically, detecting anomalies, tracing r...

3 Aug 59min

Real AI Transformation Costs HALF of Everyone's Salary for 2 Years | Chris Blackburn, Liatrio

Real AI Transformation Costs HALF of Everyone's Salary for 2 Years | Chris Blackburn, Liatrio

Most companies think they're transforming with AI. They're not, and the gap between what they believe and what's actually happening on the ground is costing them far more than they realize. In this e...

30 Juli 1h 5min

"According to NASA's Definition of Life, I'm Not Alive" - Why Nobody Can Define Life | Dr. Kate Adamala

"According to NASA's Definition of Life, I'm Not Alive" - Why Nobody Can Define Life | Dr. Kate Adamala

Nobody has ever built a cell from scratch - assembled entirely from purified molecules on a shelf - that can feed itself, grow, and split into daughter cells through its own genetic activity. Until no...

29 Juli 46min

Video Is About to Stop Being One-Way (and That Changes Everything) | Victor Riparbelli, Synthesia

Video Is About to Stop Being One-Way (and That Changes Everything) | Victor Riparbelli, Synthesia

Every company is creating content that nobody reads, nobody watches, and nobody remembers, and the CEO of the AI platform that 90% of Fortune 100 companies use to fix that just explained what comes ne...

28 Juli 39min

Populärt inom Teknik

uppgang-och-fall
market-makers
skogsforum-podcast
rss-elektrikerpodden
rss-laddstationen-med-elbilen-i-sverige
rss-uppgang-och-fall
rss-en-ai-till-kaffet
natets-morka-sida
hej-bruksbil
bli-saker-podden
garagehang
 och-bilen-gar-bra
bosse-bildoktorn-och-hasse-p
under-femton
gubbar-som-tjotar-om-bilar
elbilsveckan
klocksnack-tillsammans-med-nymans-ur-1851
rss-milpodden
bilar-med-sladd
rss-fabriken-2