Token Cost Optimization
cloud20304 Sep

Token Cost Optimization

In this episode, we focus on token cost and token optimization in generative AI, and we discuss why token use needs budgets and controls rather than being treated as unlimited. We also talk about how repeated prompting, testing, and downstream work can quickly increase token consumption. We also talk about where AI agents make sense and where cheaper models or existing data should be used instead, including a bug triage example where systems should gather history and reproduce issues before escalating. We close by discussing responsibility, accuracy, and governance, including marking unverified claims clearly and using strict prompts and review passes. We also note the need for better expectations around safety and liability in AI systems. Transcript here: https://otter.ai/u/0Y-EBKoKsN7UWxC6YzMkb44ZN6w?utm_source=copy_url

Det här avsnittet är hämtat från ett öppet RSS-flöde och publiceras inte av Podme. Det kan innehålla reklam.

Avsnitt(500)

Harness AI Governance

Harness AI Governance

In this episode, we continue our discussion of governed execution for agentic AI and how to prevent agents from taking actions they should not take. We use a maintenance example to examine access, app...

25 Sep 48min

Harnesses Pt 1

Harnesses Pt 1

This week we discuss the role of a harness in AI systems and why it matters for infrastructure, tool use, rules, and operational constraints when an AI acts on behalf of a person or organization. We a...

18 Sep 37min

Whats Up with VMware?

Whats Up with VMware?

In this episode, we are joined by guest Byron Dill from Summit for a great deep dive into virtualization. We talk about virtualization from an enterprise and infrastructure perspective, with a focus o...

11 Sep 58min

Vibing vs Buying

Vibing vs Buying

In this episode, we get into current market news, including pending IPOs, hardware investment, and IBM’s AI-related announcement with Red Hat, and what this may mean for open weights models and hybrid...

28 Aug 55min

AI Harnesses and Hosting

AI Harnesses and Hosting

This week we have an exciting conversation about harnesses, neoclouds, open weights models, and the current AI infrastructure landscape. We discuss bare metal automation, agent-friendly APIs, CPU comp...

14 Aug 57min

Innovations in Agentic Pipelines

Innovations in Agentic Pipelines

In this episode, we discuss Red Hat’s enterprise AI work, including OpenShift AI, safety guardrails, and router-based model selection. We also talk about smaller AI systems, long-term memory, and the ...

14 Aug 52min

Mirantis IREN Acquisition

Mirantis IREN Acquisition

We discuss the acquisition of Mirantis by the neocloud company IREN, and use it to examine the current market for infrastructure and AI deployment. We look at Mirantis’ consulting business, Kubernetes...

7 Aug 1h 4min

Populärt inom Teknik

natets-morka-sida
uppgang-och-fall
elbilsveckan
bilar-med-sladd
market-makers
rss-laddstationen-med-elbilen-i-sverige
rss-en-ai-till-kaffet
rss-elektrikerpodden
skogsforum-podcast
rss-technokratin
rss-ai-med-jonas-benjamin
rss-sakerhetspodcasten
rss-uppgang-och-fall
rss-veckans-ai
rss-snacka-om-ai
developers-mer-an-bara-kod
rss-powerboat-sverige-podcast
bli-saker-podden
rss-it-sakerhetspodden
rss-fabriken-2