Data Match, AgileData Engineering Pattern #7 - Episode #75

Data Match, AgileData Engineering Pattern #7 - Episode #75

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the Data Match.

The Data Match pattern provides an automated, granular comparison capability to efficiently identify and report discrepancies between two datasets, moving from row counts to specific data values.

This 'data diff' solution transforms hours of manual data reconciliation into minutes by optimising comparisons for cloud analytics databases like BigQuery, serving as a support feature for on-demand exception handling rather than a continuous trust rule.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/172820886/pattern-name

Discover more AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |

Buy the Green Book now!

Simply Magical Data Ways of Working

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(93)

Listen now | Recording of AURA - Anna Bergevin & Shane Gibson 21st September 2026

Listen now | Recording of AURA - Anna Bergevin & Shane Gibson 21st September 2026

Raw and unedited   Top 3 things: Personal OS / “Watson OS” a privacy-conscious household management system that filters emails, checks shared calendars, routes responsibilities, and sends daily/week...

21 Sep 56min

Listen Now | Recording of AURA - Chris Gambill & Shane Gibson on 11th September 2026

Listen Now | Recording of AURA - Chris Gambill & Shane Gibson on 11th September 2026

Raw and unedited Top 3 things: How to migrate legacy data platforms safely audit first, choose greenfield vs. lift-and-shift early, define the target architecture, and validate with a thin-slice POC...

11 Sep 1h 6min

Listen now | Recording of AURA - Tim Frazer & Shane Gibson on 2nd September 2026

Listen now | Recording of AURA - Tim Frazer & Shane Gibson on 2nd September 2026

Raw and unedited Top 3 things: Data platform architecture and orchestration whether agent meshes can replace traditional DAG-based orchestration, and when platform-native tools are sufficient. Cho...

4 Sep 1h 4min

Listen now | Recording of AURA - Juha Korpela & Shane Gibson on 28 August 2026

Listen now | Recording of AURA - Juha Korpela & Shane Gibson on 28 August 2026

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-juha-a97?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you ask...

28 Aug 1h

Listen now | Recording of AURA - Anna Bergevin & Shane Gibson on 21st August 2026

Listen now | Recording of AURA - Anna Bergevin & Shane Gibson on 21st August 2026

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-anna-ad5?utm_source=youtube   <-OO-> Information Product Canvas Workshop “That’s not what I wanted.” “But that’s what you ask...

21 Aug 1h 7min

Listen now | Recording of AURA - Chris Gambill & Shane Gibson on 14th August 2026

Listen now | Recording of AURA - Chris Gambill & Shane Gibson on 14th August 2026

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-chris-0ce?utm_source=youtube   <-OO-> Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata....

14 Aug 55min

Listen now | Recording of AURA - Tim Frazer & Shane Gibson on 5th August 2026

Listen now | Recording of AURA - Tim Frazer & Shane Gibson on 5th August 2026

Raw and unedited Top 3 things: Architecture and tests matter more than code generation. AI can build quickly, but sustainable work depends on clear outcomes, strong test coverage, and system-level ...

5 Aug 1h 2min

Listen now | Recording of AURA - Juha Korpela & Shane Gibson on 31st July 2026

Listen now | Recording of AURA - Juha Korpela & Shane Gibson on 31st July 2026

Raw and unedited   https://agiledata.info/p/watch-now-recording-of-aura-juha?utm_source=youtube     <-OO-> Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.i...

31 Jul 1h 1min

Populært innen Teknologi

teknisk-sett
lydartikler-fra-aftenposten
energi-og-klima
rss-ki-praten
elektropodden
hans-petter-og-co
smart-forklart
rss-alt-som-gar-pa-strom
rss-snakk-om-sikkerhet
fornybaren
shifter
rss-ai-forklart
tomprat-med-gunnar-tjomlid
rss-teknologioptimistene-en-podkast-om-teknologi-og-mennesker
teknologi-og-mennesker
pedagogisk-intelligens
nasjonal-sikkerhetsmyndighet-nsm
rss-alt-vi-kan
rss-ki-til-kaffen
plattformpodden