Data Match, AgileData Engineering Pattern #7 - Episode #75
Agile Data N’ Info4 Syys 2025

Data Match, AgileData Engineering Pattern #7 - Episode #75

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the Data Match.

The Data Match pattern provides an automated, granular comparison capability to efficiently identify and report discrepancies between two datasets, moving from row counts to specific data values.

This 'data diff' solution transforms hours of manual data reconciliation into minutes by optimising comparisons for cloud analytics databases like BigQuery, serving as a support feature for on-demand exception handling rather than a continuous trust rule.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/172820886/pattern-name

Discover more AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |

Buy the Green Book now!

Simply Magical Data Ways of Working

Jaksot(81)

Fact-based modelling patterns with Marco Wobben - Episode #81

Fact-based modelling patterns with Marco Wobben - Episode #81

Join Shane Gibson as he chats with Marco Wobben about the patterns within Fact-based modeling.   You can get in touch with Marco via LinkedIn or over at Casetalk.com If you want want to read a summar...

14 Helmi 1h 9min

Redesigning traditional data systems to support Large Language Models and AI agents with Mayowa Oludoyi - Episode #80

Redesigning traditional data systems to support Large Language Models and AI agents with Mayowa Oludoyi - Episode #80

Join Shane Gibson as he chats with Mayowa Oludoyi about redesigning traditional data systems to better support LLM and AI Agents   You can get in touch with Mayowa via LinkedIn  If you want want to r...

18 Tammi 52min

Building Data Services with AI with Jason Taylor - Episode #79

Building Data Services with AI with Jason Taylor - Episode #79

Join Shane Gibson as he chats with Jason Taylor a former quant researcher who turned towards the light (or dark) side of data, to explore the practicalities and pitfalls of building data services usin...

9 Tammi 1h 13min

Can AI tools bring back data modeling with Andy Cutler - Episode #78

Can AI tools bring back data modeling with Andy Cutler - Episode #78

Join Shane Gibson as he chats with Andy Cutler about the art of data modeling and the potential of AI tools to improve the art. You can get in touch with Andy via LinkedIn or over at https://linktr.e...

23 Joulu 20251h 13min

The pattern of Metric Trees with Timo Dechau - Episode #77

The pattern of Metric Trees with Timo Dechau - Episode #77

Join Shane Gibson as he chats with Timo Dechau about the pattern of Metric Trees. You can get in touch with Timo via LinkedIn or over at https://timodechau.com If you want want to read a summary gen...

21 Marras 20251h 6min

Diagramming to  Understanding Your Data Estate with Rob Long - Episode #76

Diagramming to  Understanding Your Data Estate with Rob Long - Episode #76

Join Shane Gibson as he chats with Rob Long about patterns that you can use to document and understand your data estate. You can read the article we discuss in the podcast episode here: https://atlo...

13 Loka 202559min

Patterns to define the ROI of a data product with Nick Zervoudis - Episode #74

Patterns to define the ROI of a data product with Nick Zervoudis - Episode #74

Join Shane Gibson as he chats with Nick Zervoudis about patterns that you can use to quickly and easily define the ROI of your data products, before you build them. You can get in touch with Nick via...

4 Syys 20251h 7min