Stop Wasting Money: The 3 Architectures for Fabric Data Flows Gen 2

Stop Wasting Money: The 3 Architectures for Fabric Data Flows Gen 2

Fabric Dataflows Gen2 architectures: in this episode of M365.fm, Mirko Peters explains why most Microsoft Fabric Dataflows Gen2 deployments quietly burn far too much compute—and how three clear architectures for staging, transforming, and serving data can cut your capacity bill while improving governance and performance. He shows how treating Dataflows Gen2 like “Power BI dataflows 2.0” leads to duplicated ingestion, repeated refreshes, and multiple workspaces pulling the same source data over and over again.

Mirko starts with the core misunderstanding: in Fabric, compute—not storage—is what you pay for. Every refresh spins up distributed compute, lands delta files, and tears clusters down again, so copying the same data into multiple workspaces multiplies your costs without adding value. He explains why Fabric assumes a shared lakehouse model—data lands once in OneLake and is reused many times—and how Dataflows Gen2 were redesigned as pipelines in Power Query clothing to support that pattern with lineage and reuse instead of one‑off imports.

The first architecture he introduces is the Staging (Bronze) Dataflow. Here, each external system—CRM, ERP, HR, line‑of‑business SQL—lands once into standardized delta tables in a shared lakehouse. Mirko shows how to keep logic minimal at this layer (types, basic cleanup, incremental refresh), so refresh jobs are cheap, repeatable, and reusable for every downstream team. This “ingest once, share everywhere” pattern stops five departments from hammering the same API with five near‑identical dataflows.

The second architecture is the Transform (Silver) Dataflow, where business logic, joins, and normalization happen on top of the bronze layer instead of directly against external sources. Mirko explains how to centralize entity logic (customer, product, calendar) into curated silver tables that multiple domains share, avoiding each workspace inventing its own slightly different version. He shows why running transformations against delta data instead of external systems is cheaper, more reliable, and easier to govern.

The third architecture is the Serve (Gold) pattern, where lightweight, consumption‑ready Dataflows or shortcuts feed semantic models, Direct Lake datasets, and downstream tools. Mirko explains how this layer should be thin—final shaping, field naming, and aggregations instead of heavy ETL—so refreshes stay fast and compute stays low. He walks through how Staging–Transform–Serve fits together as a reusable blueprint you can replicate across domains, instead of reinventing pipelines for every new project.

WHAT YOU WILL LEARN
  • Why treating Fabric Dataflows Gen2 like old Power BI dataflows explodes compute and refresh costs.
  • How a Staging (Bronze) Dataflow layer lands each external source once into reusable delta tables.
  • How a Transform (Silver) layer centralizes business logic and joins on top of shared lakehouse data.
  • How a Serve (Gold) layer delivers thin, consumption‑ready outputs for Direct Lake and semantic models.
  • How to design lineage, workspaces, and refresh patterns so one ingestion serves many consumers without duplication.
THE CORE INSIGHT

Fabric Dataflows Gen2 are not just a nicer way to import—they are your front door for lakehouse architectures. Once you adopt a Staging–Transform–Serve pattern, each source lands once, transformations become reusable assets, and your capacity spend reflects business value instead of duplicated refresh cycles.

WHO THIS EPISODE IS FOR

This episode is ideal for Fabric architects, data engineers, BI leads, and Power BI professionals who are moving from classic Power BI to Fabric and want to avoid building a sprawling, expensive tangle of Gen2 dataflows. It is especially valuable if you are seeing rising capacity costs, duplicated ingestion across workspaces, or unclear lineage and want a simple, three‑architecture blueprint to standardize new projects.

ABOUT THE HOST

Mirko Peters is a Microsoft 365 and data platform consultant focused on building governed, scalable analytics architectures with Microsoft Fabric, Power BI, the Power Platform, and OneLake. Through M365.fm, he shares practical lakehouse patterns, cost‑control strategies, and real‑world Fabric migration stories that help organizations turn Dataflows Gen2 into an efficient backbone instead of an expensive ETL tangle.

Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(814)

From AI Hype to AI Harness Engineering – Building AI That People Can Actually Trust with Alan Buscaglia [MVP] from Gentleman Programming

From AI Hype to AI Harness Engineering – Building AI That People Can Actually Trust with Alan Buscaglia [MVP] from Gentleman Programming

Artificial Intelligence is evolving rapidly, but building AI that organizations can actually trust requires far more than choosing the latest language model. In this episode of the M365.fm podcast, Mi...

20 Jul 0s

Agent-to-Agent (A2A) Communication - Simply Explained

Agent-to-Agent (A2A) Communication - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Agent-to-Agent (A2A) Communication, the open protocol that allows AI agents to discover one another,...

20 Jul 18min

Microsoft Entra External ID - Simply Explained

Microsoft Entra External ID - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Microsoft Entra External ID, Microsoft's modern Customer Identity and Access Management (CIAM) platf...

20 Jul 14min

Microsoft Graph Connectors - Simply Explained

Microsoft Graph Connectors - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Microsoft Graph Connectors, now increasingly referred to as Microsoft Copilot Connectors. While Micr...

20 Jul 15min

Azure Advisor - Simply Explained

Azure Advisor - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Azure Advisor, Microsoft's free optimization service that continuously analyzes your Azure environme...

20 Jul 15min

Azure Cost Management - Simply Explained

Azure Cost Management - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Azure Cost Management, Microsoft's built-in solution for monitoring, analyzing, controlling, and opt...

20 Jul 18min

Azure Automation - Simply Explained

Azure Automation - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Azure Automation, Microsoft's cloud service for automating repetitive IT tasks across Azure and hybr...

20 Jul 15min

Azure Managed Disks - Simply Explained

Azure Managed Disks - Simply Explained

Welcome to another episode of Knowledge Nuggets with Mirko Peters. In this episode, we're exploring Azure Managed Disks, the storage technology that powers every Azure Virtual Machine. While many peop...

20 Jul 15min

Populært innen Politikk og nyheter

giver-og-gjengen-vg
aftenpodden
aftenpodden-usa
fotballpodden-2
forklart
stopp-verden
popradet
rss-gukild-johaug
det-store-bildet
hanna-de-heldige
aftenbla-bla
rss-ness
dine-penger-pengeradet
nokon-ma-ga
bt-dokumentar-2
unitedno
e24-podden
lydartikler-fra-aftenposten
oppdatert
rss-borsmorgen-okonominyhetene