Catalyst with Shayle Kann18 Dec 2025

Will inference move to the edge?

Today virtually all AI compute takes place in centralized data centers, driving the demand for massive power infrastructure. But as workloads shift from training to inference, and AI applications become more latency-sensitive (autonomous vehicles, anyone?), there‘s another pathway: migrating a portion of inference from centralized computing to the edge. Instead of a gigawatt-scale data center in a remote location, we might see a fleet of smaller data centers clustered around an urban core. Some inference might even shift to our devices. So how likely is a shift like this, and what would need to happen for it to substantially reshape AI power? In this episode, Shayle talks to Dr. Ben Lee, a professor of electrical engineering and computer science at the University of Pennsylvania, as well as a visiting researcher at Google. Shayle and Ben cover topics like: The three main categories of compute: hyperscale, edge, and on-device Why training is unlikely to move from hyperscale The low latency demands of new applications like autonomous vehicles How generative AI is training us to tolerate longer latencies Why distributed inference doesn‘t face the same technical challenges as distributed training Why consumer devices may limit model capability Resources: ACM SIGMETRICS Performance Evaluation Review: A Case Study of Environmental Footprints for Generative AI Inference: Cloud versus Edge Internet of Things and Cyber-Physical Systems: Edge AI: A survey Credits: Hosted by Shayle Kann. Produced and edited by Daniel Woldorff. Original music and engineering by Sean Marquand. Stephen Lacey is our executive editor. Catalyst is brought to you by EnergyHub. EnergyHub helps utilities build next-generation virtual power plants that unlock reliable flexibility at every level of the grid. See how EnergyHub helps unlock the power of flexibility at scale, and deliver more value through cross-DER dispatch with their leading Edge DERMS platform, by visiting energyhub.com. Catalyst is brought to you by Bloom Energy. AI data centers can’t wait years for grid power—and with Bloom Energy’s fuel cells, they don’t have to. Bloom Energy delivers affordable, always-on, ultra-reliable onsite power, built for chipmakers, hyperscalers, and data center leaders looking to power their operations at AI speed. Learn more by visiting⁠ ⁠⁠BloomEnergy.com⁠. Catalyst is supported by Third Way. Third Way’s new PACE study surveyed over 200 clean energy professionals to pinpoint the non-cost barriers delaying clean energy deployment today and offers practical solutions to help get projects over the finish line. Read Third Way's full report, and learn more about their PACE initiative, at www.thirdway.org/pace.

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(269)

Is now the time for DERs to scale?

A decade ago, DERs were hot. The hype was that things like batteries, smart devices, and other distributed energy technologies would offset the need for expanding traditional grid infrastructure. But...

13 Aug 42min

Critical Capital: Senator Tillis' case for "all-of-the-above" energy

This week, we’re presenting an episode of Critical Capital, a new podcast from Crux and Latitude Studios. Energy policy has become increasingly polarized just as the U.S. faces record electricity dem...

11 Aug 31min

The electrician shortage slowing the AI boom

The race to build AI infrastructure is often framed around the availability of power, chips, and capital. But another critical constraint is becoming impossible to ignore: skilled labor. In this epis...

6 Aug 34min

The commercial battery comeback

Commercial and industrial (C&I) battery storage has long been the missing middle of the energy storage market. It’s been too complex and expensive to scale like residential systems, but too small to ...

30 Juli 44min

The multi-billion dollar promise of vehicle-to-grid integration

Most electric vehicles spend nearly all of their lives parked, doing nothing. But under the hood of those idle cars, a massive store of energy sits untapped. The Kia EV9, for instance, holds a 100-kil...

23 Juli 34min

When will quantum computing have its breakout moment?

Over the course of the past decade, quantum computing (or the concept thereof) has ridden hype cycles, just like AI or the internet writ large. In the past year, however, as governments across the g...

16 Juli 40min

Inside the AI power wars

For those steeped in the world of AI development, the major differences between tech giants’ chip strategies are easy enough to see: the speed of Google’s TPUs, the affordability of Amazon’s Trainium,...

9 Juli 43min

Inside the most sophisticated plan for solar geoengineering

One of the biggest challenges with technologies that reduce greenhouse gas emissions is that they require mass adoption. Solar radiation management (SRM) has the opposite problem. By Stardust Solutio...

1 Juli 37min