Building and deploying production grade AI agents with Microsoft Foundry with Edgar McOchieng [MVP]

In this deep-dive episode of the M365 FM podcast, Mirko Peters welcomes Edgar McOchieng for an extensive conversation about enterprise AI architecture, Microsoft Foundry, scalable AI agents, and the real-world challenges organizations face when deploying production-grade AI systems. Edgar shares his journey from discovering Microsoft Azure during university in Kenya to becoming a Microsoft MVP focused on Microsoft Foundry, Business Applications, data engineering, and AI-driven enterprise solutions. He also talks about his passion for mentorship and community building through “Ochieng Labs,” where students and early-career developers gain hands-on experience with Power Platform, Microsoft Fabric, Copilot Studio, and modern AI engineering practices.

BUILDING REAL-WORLD ENTERPRISE AI APPLICATIONS

The conversation explores how organizations can move beyond AI experimentation and start building reliable, secure, and scalable AI applications that deliver measurable business value. Edgar explains how his team created an enterprise AI platform capable of connecting to SharePoint, OneDrive, Outlook, Microsoft Graph, AWS, and Google Cloud environments to help employees retrieve organizational knowledge faster and reduce data silos across departments. Listeners will learn how Retrieval-Augmented Generation (RAG), vector search, semantic indexing, embeddings, and enterprise search architectures play a critical role in modern AI systems. Edgar breaks down how AI applications can access live organizational knowledge instead of relying solely on static training data, helping businesses build more accurate and context-aware AI assistants. HYBRID AI ARCHITECTURES AND AI COST OPTIMIZATION A major focus of this episode is enterprise AI cost management and hybrid AI infrastructure design. Edgar openly discusses the challenges organizations face with rising AI costs caused by heavy usage of premium cloud-based large language models such as Anthropic Claude and GPT services. He explains how his team introduced a hybrid orchestration model that intelligently switches between local small language models and cloud-hosted LLMs depending on the complexity of the task. This hybrid AI approach dramatically reduced operational expenses while maintaining scalability and performance. The discussion also covers rate limiting, token management, AI workload monitoring, hosted agents, orchestration layers, and why enterprises increasingly need ownership and control over their AI infrastructure.

MICROSOFT FOUNDRY, COPILOT STUDIO, AND AI DEVELOPMENT WORKFLOWS

Edgar describes Microsoft Foundry as a powerful “model playground” where developers can experiment with multiple AI models, create hosted agents, build orchestration pipelines, evaluate model safety, apply guardrails, and integrate enterprise systems using MCP connectors. He also explains the differences between Microsoft 365 Copilot, Copilot Studio, and Microsoft Foundry — helping listeners understand when each platform is the right choice depending on customization requirements and technical maturity. The episode also dives into prompt engineering, AI workflows, GitHub Copilot, VS Code integrations, CI/CD pipelines with GitHub Actions, evaluation pipelines, hallucination testing, and the growing importance of developer tooling in AI application development. Edgar shares practical insights into how AI engineering teams structure, test, deploy, and continuously improve enterprise AI systems in production environments.

AI GOVERNANCE, SECURITY, AND ENTERPRISE MONITORING

Another key topic throughout the conversation is AI governance, observability, security, and responsible AI implementation. Edgar explains why governance and monitoring are becoming more important than simply selecting the “best” AI model. Organizations need visibility into user behavior, AI usage patterns, permissions, hallucination risks, security controls, and compliance requirements. The discussion also covers multi-tenant enterprise AI architectures, tenant isolation, data partitioning, hosted AI agents, containerization, Kubernetes integrations, Power Platform connectivity, Logic Apps orchestration, and enterprise-grade monitoring systems designed to support scalable AI workloads.

THE FUTURE OF ENTERPRISE AI

Toward the end of the episode, Mirko and Edgar discuss several hot topics shaping the future of enterprise AI, including small language models (SLMs), prompt engineering, orchestration-driven AI workflows, fine-tuning versus data grounding, and the long-term sustainability of relying entirely on external AI providers. Edgar argues that organizations increasingly need flexibility, transparency, governance, and infrastructure ownership to remain competitive as AI adoption continues to accelerate. This episode is packed with practical insights for enterprise architects, AI engineers, cloud developers, CTOs, IT leaders, Microsoft professionals, startup founders, and anyone interested in understanding how Microsoft Foundry and Azure AI technologies are reshaping modern enterprise software development and intelligent automation.

IN THIS EPISODE

• Building production-grade AI agents with Microsoft Foundry
• Designing scalable hybrid AI architectures for enterprises
• Implementing AI governance, observability, and monitoring
• Reducing enterprise AI costs using local and hosted models
• Retrieval-Augmented Generation (RAG) and vector search
• Hosted AI agents, orchestration layers, and prompt flows
• Enterprise integrations with Microsoft Graph, SharePoint, and Power Platform
• Multi-tenant AI architectures and secure data isolation
• AI evaluation pipelines, guardrails, and hallucination prevention
• CI/CD strategies for enterprise AI deployments

KEY TECHNOLOGIES DISCUSSED

• Microsoft Foundry
• Azure AI Services
• Microsoft 365 Copilot
• Copilot Studio
• Microsoft Fabric
• Power Platform
• GitHub Copilot
• MCP Connectors
• Vector Databases
• Retrieval-Augmented Generation (RAG)
• Kubernetes
• Logic Apps
• Azure Hosted Agents

WHO SHOULD LISTEN

This episode is highly recommended for enterprise architects, AI engineers, Microsoft consultants, cloud developers, CTOs, CIOs, IT decision-makers, Power Platform professionals, startup founders, security teams, and technology leaders looking to understand how enterprise AI systems can be designed, governed, scaled, and optimized using Microsoft’s modern AI ecosystem.

Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.