Measuring LLMs with Jodie Burchell
.NET Rocks!3 Huhti 2025

Measuring LLMs with Jodie Burchell

How do you measure the quality of a large language model? Carl and Richard talk to Dr. Jodie Burchell about her work measuring large language models for accuracy, reliability, and consistency. Jodie talks about the variety of benchmarks that exist for LLMs and the problems they have. A broader conversation about quality digs into the idea that LLMs should be targeted to the particular topic area they are being used for - often, smaller is better! Building a good test suite for your LLM is challenging but can increase your confidence that the tool will work as expected.

Jaksot(1988)

Matt Mannion Goes to the Races!

Matt Mannion Goes to the Races!

Matt Mannion from Clarity Consulting talks about the application his company developed with and for the Indy Racing League in Indianappolis, IN. Using Visual Studio .NET 2003 and VB.NET they were able...

17 Huhti 200559min

Live from the Microsoft Platform Adoption Center

Live from the Microsoft Platform Adoption Center

Grant BlahaErath and Glenn Podany talk about the MS Platform Adoption Center: what it is and what it's used for. Sometimes called the Customer Lab, the PAC has generated some great case studies with m...

11 Huhti 20051h 16min

Scott Hanselman and Rory Blyth

Scott Hanselman and Rory Blyth

Scott Hanselman returns to DNR for the first time in several months. In this otherwise ASP.NET-centric show, Rory shows off his hands-free flush abilities and Scott rants about Cascading Style Sheets...

4 Huhti 200559min

Rocky Lhotka and Bill Vaughn Live at Dev Connections!

Rocky Lhotka and Bill Vaughn Live at Dev Connections!

Rhocky Lhotka and Bill Vaughn return in this *live* recording of Dot Net Rocks, from the March 2005 Dev Connections conferene in Orlando, FLSupport this podcast at — https://redcircle.com/net-rocks/do...

27 Maalis 20051h 10min

Ken Getz Does a Mind Meld

Ken Getz Does a Mind Meld

Ken Getz comes back for round 2 with .NET Rocks! talking about new features of VB.NET 2005, SharePoint and WSS, wireless service, and toys!Support this podcast at — https://redcircle.com/net-rocks/don...

21 Maalis 20051h 5min

Paul Sheriff on Architecture

Paul Sheriff on Architecture

World-renowned author and trainer Paul Sheriff talks with Carl and Richard about all things architecture: what, why, when, where, and how. Even if you have no plans to move beyond the cowboy way of so...

14 Maalis 20051h 8min

Chris Anderson and the Longhorn Evangelism Team

Chris Anderson and the Longhorn Evangelism Team

This is a great show on what's coming in Longhorn technology, namely Indigo, Avalon, and XAML, and compatibility. We get to the heart of what the term "Managed OS" really means. Jeff Sandquist tries t...

7 Maalis 20051h 17min

NASA World Wind Development Team

NASA World Wind Development Team

Chris Maxwell and Randy Kim from the National Aeronautics and Space Administration talk about their recent .NET project, World Wind, an educational and informational tool for viewing planet Earth (and...

28 Helmi 20051h 12min