Episode 49: Rethinking AI Agent Evaluations

Episode 49: Rethinking AI Agent Evaluations

In this episode we explore how companies should evaluate AI agents across multiple dimensions — including correctness, tool selection, multi-turn reasoning, and safety . The conversation covers building reliable evaluation frameworks, balancing automated vs. human-in-the-loop testing, and leveraging observability to debug agent behavior in production.


Links from the Show

AgentCore Evaluation: https://github.com/awslabs/agentcore-samples/tree/main/01-tutorials/07-AgentCore-evaluations

Strands Evaluation: https://strandsagents.com/docs/user-guide/evals-sdk/quickstart/


AWS Hosts: Nolan Chen & Malini Chatterjee

Email Your Feedback: rethinkpodcast@amazon.com

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(53)

Episode 53: AWS re:Think - AWS Summits Recap Part 2

Episode 53: AWS re:Think - AWS Summits Recap Part 2

In Part 2 of our AWS Summits recap, Geries AbouAyash, AWS Managing Director, North America Solutions Architecture joins Malini and Amit to unpack what the shift to agentic AI really means for how we w...

23 Jul 25min

Episode 52: AWS re:Think - AWS Summits Recap Part 1

Episode 52: AWS re:Think - AWS Summits Recap Part 1

Three AWS Summits across the US — LA on June 10, NYC on June 17, and Washington DC on June 30–July 1 — brought a wave of announcements that all orbited a single thesis: the age of agentic AI is here, ...

16 Jul 25min

Episode 51: Rethinking Cloud Security in the Age of Zero-Days and AI

Episode 51: Rethinking Cloud Security in the Age of Zero-Days and AI

Modern cloud environments are evolving faster than traditional security models can keep up. In this episode, we sit down with Yarin Pinyan, VP Products at Upwind, to explore how real-time runtime visi...

10 Jun 42min

Episode 50: Rethink Agentic AI: From Shadow Risk to Strategic Value

Episode 50: Rethink Agentic AI: From Shadow Risk to Strategic Value

AI agents move at machine speed. Your security doesn't. Discover how enterprises are finding shadow agents, measuring agent ROI, and transforming agentic AI from risk to strategic value. Learn from Ar...

29 Apr 51min

Episode 48: Rethinking Software Engineering through a Spec-Driven Approach with Kiro

Episode 48: Rethinking Software Engineering through a Spec-Driven Approach with Kiro

In this episode we join AWS Senior GTM Specialists Aidin Khosrowshahi and Rakesh Kumar to unpack how Kiro is transforming software development through a spec-driven approach that goes beyond tradition...

10 Mar 38min

Episode 47: What it Takes to Win in 2026

Episode 47: What it Takes to Win in 2026

"Its amazing to see the rate and pace of advancements in technology in 2025" Todd Pond, AWS Director of Strategic Sales, is out in the field with commercial customers every day, helping them leverage ...

30 Jan 25min

Episode 46: Rethinking Bio Pharma Compliance in the Cloud with Aizon

Episode 46: Rethinking Bio Pharma Compliance in the Cloud with Aizon

Aizon.ai provides an AI software platform for the pharmaceutical and biotech industries to optimize manufacturing processes, ensure GxP compliance, and improve product quality. Their top outcomes incl...

21 Jan 26min

Populært innen Business og økonomi

stopp-verden
dine-penger-pengeradet
lydartikler-fra-aftenposten
rss-penger-polser-og-politikk
e24-podden
rss-borsmorgen-okonominyhetene
rss-skravla-gar
pengepodden-2
rss-pa-konto
utbytte
tid-er-penger-en-podcast-med-peter-warren
morgenkaffen-med-finansavisen
liberal-halvtime
livet-pa-veien-med-jan-erik-larssen
okonomiamatorene
lederpodden
rss-markedspuls-2
rss-kron-podden
stockup
boligbobla