Lessons from Transcribing and Indexing 3.5 Million Podcasts with Arvid Kahl
Software Huddle8 Heinä 2025

Lessons from Transcribing and Indexing 3.5 Million Podcasts with Arvid Kahl

Big time guest today as Arvid Kahl joins us. Arvid is my favorite type of guest -- a deeply technical founder that can talk about both the technical and business challenges of a startup. Lots to enjoy from this episode. Arvid is known as the Bootstrapped Founder and has documented his path to selling Feedback Panda back in 2019. He's now building Podscan and sharing his journey as he goes. Podscan is a fascinating project. It's making the content of *every* podcast episode around the world fully searchable. He currently has 3.5 million episodes transcribed and adds another 30,000 - 50,000 episodes every day. This involves a ton of technical challenges, including how to get the best transcription results from the latest LLMs, whether you should use APIs from public providers or run your own LLMs, and how to efficiently provide full-text search across terabytes of transcription data. Arvid shares the lessons he's learned and the various strategies he's tried over the years. But there are also unique business challenges. For most technical businesses, your infrastructure costs grow in line with your customers. More customers == more data == more servers. With Podscan, Arvid has to index the entire podcast ecosystem regardless of his customers. This means a lot of upfront investment as he looks to grow his customer base. Arvid tells us how he's optimized his infrastructure to account for this unique challenge.

Tämä jakso on lisätty Podme-palveluun avoimen RSS-syötteen kautta eikä se ole Podmen omaa tuotantoa. Siksi jakso saattaa sisältää mainontaa.

Jaksot(79)

AI and Proactive Reliability with Kolton Andrus

AI and Proactive Reliability with Kolton Andrus

Today we're talking with Kolton Andrus, the Founder and CEO of Gremlin, about what happens to reliability when AI is writing most of the code. Kolton helped build the Chaos Engineering practice of bot...

8 Huhti 55min

Making Data Agent Ready with Andre Elizondo

Making Data Agent Ready with Andre Elizondo

Today we are talking with Andre Elizondo, the Director of Innovation at Mezmo about their open source agentic harness for SREs called AURA. Mezmo got their start handling observability data at scale. ...

27 Maalis 51min

Exponential Engineers with Ashmeet Sidana

Exponential Engineers with Ashmeet Sidana

Today on the show, we have a special guest — Ashmeet Sidana, the founder of Engineering Capital. Ashmeet started his career as an engineer at some great companies like Hewlett-Packard and Silicon Grap...

9 Tammi 53min

Powered by Neurons with Ewelina Kurtys

Powered by Neurons with Ewelina Kurtys

Today we have Dr. Ewelina Kurtys on the show. Ewelina has a background in Neuroscience and is currently working at FinalSpark. FinalSpark is using live Neurons for computations instead of traditiona...

16 Syys 202542min

Lessons from Building AI Agents with Rafal Wilinski

Lessons from Building AI Agents with Rafal Wilinski

Today we're talking with one of our favorite engineers, Rafal Wilinski. Rafal has been on the cutting edge of AI development in the last few years as he has led AI teams at Zapier and Vendr. Rafal wa...

12 Elo 20251h 8min

Building a High-Ownership Engineering Culture with Matt Watson

Building a High-Ownership Engineering Culture with Matt Watson

If you’ve ever felt like engineering teams are stuck in execution mode—heads down, building what they’re told—then today’s episode is for you. We're talking about what it really takes to build high ow...

5 Elo 202551min

Building CI for the age of AI Agents with Aayush Shah

Building CI for the age of AI Agents with Aayush Shah

Today's episode is with Aayush Shah. Aayush is one of the co-founders of Blacksmith, which is a CI compute platform. Basically, Blacksmith will run your GitHub Actions jobs faster and with more visibi...

22 Heinä 20251h 4min

Valkey After the Fork: A Conversation with Madelyn Olson

Valkey After the Fork: A Conversation with Madelyn Olson

Today, we're talking Valkey, Redis, and all things caching. Our guest is Madelyn Olson, who is a principal engineer at AWS working on Elasticache and is one of the most well-known people in the cachin...

16 Heinä 20251h 21min