AI Insights – EP.2: Unlocking Cost-Effective AI with Small Language Models
Cisco Podcast Network26 Helmi

AI Insights – EP.2: Unlocking Cost-Effective AI with Small Language Models

In the latest episode of the Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques welcome Cisco AI operations engineer James Tidd for a discussion on the world of small language models (SLMs) and the evolution of efficient AI inference. Together, they unravel the complexities behind “Fast Inference from Transformers via Speculative Decoding,” a groundbreaking paper from Google that explores how smaller draft models can speed up large language model predictions while maintaining accuracy. James shares his hands-on experience experimenting with the technique, leveraging knowledge distillation and speculative execution. The trio also discusses the potential of this approach to optimize AI, reduce power consumption and costs, and help businesses of all sizes get more out of existing hardware. A special thank you to Google’s AI team for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: https://research.google/blog/looking-back-at-speculative-decoding/.

Jaksot(500)

S7 E3: Talking security innovations in the age of AI, the intersection of creativity and tech, and more with Tom Gillis

S7 E3: Talking security innovations in the age of AI, the intersection of creativity and tech, and more with Tom Gillis

AB sits down with Tom Gillis, Cisco’s SVP and GM, Infrastructure & Security Group, to explore the crucial role security plays in the ever-changing AI landscape.

20 Maalis 19min

404 Script Not Found: The Car Crash

404 Script Not Found: The Car Crash

Ian and Kat are back with a "yin and yang" week of personal chaos and professional growth. Kat recounts the harrowing story of her recent hit-and-run—including a very embarrassing realization about he...

19 Maalis 16min

SHIFT HAPPENS-EP 29: The Architecture of Partner Value w/Brian Overmeyer

SHIFT HAPPENS-EP 29: The Architecture of Partner Value w/Brian Overmeyer

What does it really take to redesign one of the most influential partner programs in the tech industry? In this episode of Shift Happens, Jeff Edwards sits down with Brian Overmeyer, Director of Part...

17 Maalis 49min

Beyond the Fiber - Tech Talk on X-Haul with Waris Sagheer 02182026

Beyond the Fiber - Tech Talk on X-Haul with Waris Sagheer 02182026

Unified Networking Architecture: Find out how X-Haul consolidates fronthaul, midhaul, and backhaul into a single, software-defined transport platform, and enables seamless 5G deployment across diverse...

12 Maalis 42min

Inside Distributed Monitoring Infrastructures in Network Management

Inside Distributed Monitoring Infrastructures in Network Management

Visibility from diverse networks can help uncover blind spots to detect regional issues and invisible failures impacting end users. In this episode, you’ll hear first-hand insights on the operational ...

12 Maalis 28min

404 Script Not Found: Spring Cleaning

404 Script Not Found: Spring Cleaning

Happy spring cleaning time! We're cleaning house with this episode. Enjoy some behind-the-scenes recordings from 404 over the last few months! If you haven’t already, and you want to keep these BTS ...

12 Maalis 10min

SHIFT HAPPENS EP.28: Talking Shift: Who are you without the title?

SHIFT HAPPENS EP.28: Talking Shift: Who are you without the title?

Jeff Edwards sits down with Rebecca Leach—former Cisco and AppDynamics leader turned executive coach and founder of Growth Road, an executive coaching practice focused on helping leaders navigate chan...

10 Maalis 45min

S7 E2: Talking the role of partnerships in the AI era, the Cisco 360 Partner Program, and how tech alliances drive business success, with Tim Coogan

S7 E2: Talking the role of partnerships in the AI era, the Cisco 360 Partner Program, and how tech alliances drive business success, with Tim Coogan

AB sits down with Tim Coogan, Cisco’s SVP, Global Partner Sales, for a fun chat about the important role partners play to drive positive business outcomes. From partnering in the AI era to the link be...

5 Maalis 13min