DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

DoK Talks#103 -Performant and Version-Aware Analytics With Spark & lakeFS on K8s // Itai Admi

https://go.dok.community/slack
https://dok.community/

ABSTRACT OF THE TALK

Spark and lakeFS are revolutionizing large scale data processing that is version-aware. Is it possible to run this architecture over Kubernetes? We’ll cover the fastest way to get this environment up and running, and the benefits you get with it. Finally we’ll show how horizontal scaling and the lakeFS Hadoop Filesystem avoid processing bottlenecks as workloads increase.

BIO

Itai is a R&D team leader at Treeverse, the company behind open-source lakeFS. He thrives on finding creative solutions for complex problems, especially if it involves code. Previously, Itai worked at Microsoft and Ridge on data infrastructure, tooling, and performance. Itai received his B.Sc degree in Computer Science and an MBA from Tel Aviv University.

KEY TAKE-AWAYS FROM THE TALK

- Importance of building reproducible data pipelines.
- Managing your data the same way you're managing your code.

Avsnitt(243)

Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall

Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall

Implementing Data & Databases on K8s within the Dutch GovernmentPresented by Sebastiaan Mannem, Director at Mannem Solutions A small walkthrough of projects within the Dutch government running databas...

13 Feb 202444min

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and HeraPresented by Matt Menzenski, Senior Software Engineering Manager, Payitgov At PayIt, we’ve been deploying app...

6 Feb 202423min

Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall

Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall

Repel Boarders! How to find a Kubernetes operator that really protects your dataPresented by Robert Hodges, AltinityOperators are a godsend for managing data in Kubernetes. But how about protecting it...

30 Jan 202419min

DoK + Apache Spark | DoKC Town Hall

DoK + Apache Spark | DoKC Town Hall

DoK + Apache SparkPresented by Holden Karau, Spark Committer and Open Source Engineer at NetflixIn this brief talk, Holden will cover some of the best practices from trying to deploy both small and la...

23 Jan 202419min

DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall

DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall

DoK @ Comcast: Delivering Business Outcomes & Improved DevX with Data Services Running on Kubernetes Presented by Greg Otto, Executor Director, DevX Platforms & Charles Ju, Principal Engineer Transfor...

3 Jan 202416min

DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell

DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell

Abbey Russell, PM at Cockroach Labs, shared the backstory on how and why Kafka was created. Along the way, you'll learn about - Who Franz Kafka was - Kafka's earliest use at Linkedin in 2010 -...

9 Mars 202315min

DoK Talks - (almost)Everything you need to know about stateful cloud native network applications // W Watson

DoK Talks - (almost)Everything you need to know about stateful cloud native network applications // W Watson

https://go.dok.community/slack https://dok.community/ https://youtu.be/KjiK6eXYO34 DoK Talk with W Watson, Founder at Vulk Co-op

2 Mars 202343min

The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen

The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen

https://dokcommunity.slack.com/ https://dok.community/ ABSTRACT OF THE TALK Fabian, Chris and Abhi will discuss their passion for roleplaying games, and what they can teach us about the power of ...

13 Dec 202258min

Populärt inom Teknik

uppgang-och-fall
elbilsveckan
bilar-med-sladd
market-makers
skogsforum-podcast
rss-elektrikerpodden
rss-technokratin
rss-laddstationen-med-elbilen-i-sverige
bli-saker-podden
rss-uppgang-och-fall
developers-mer-an-bara-kod
rss-en-ai-till-kaffet
rss-veckans-ai
natets-morka-sida
rss-fabriken-2
rss-powerboat-sverige-podcast
har-vi-akt-till-mars-an
hej-bruksbil
dom-kallar-oss-krypto
rss-snacka-om-ai