#19 DoK community: Towards a K8s Native Streaming Application // Francesco Nobilia & Jeremy Frenay
Data on Kubernetes Community30 Marras 2020

#19 DoK community: Towards a K8s Native Streaming Application // Francesco Nobilia & Jeremy Frenay

Starting from a simple application which can be deployed in every machine running Docker, we will go through all steps required to transform the simple app into a Kubernetes native streaming application. We will explain the theory and then exemplify the learnt concepts to define a recipe for running streaming applications on Kubernetes. We will focus both on cultural and technical tricks to help you successfully adopt streaming applications at scale.

At the end of the talk, you will have a comprehensive view regarding all platform building blocks and application requirements needed to successfully run a streaming application on Kubernetes.

Spoiler: you will hear several times the words Apache Kafka, Kafka Streams and Strimzi.


Francesco Nobilia is an enthusiastic engineer focused on building the next generation of a self-service and cost-effective streaming data platform. Event-Drive addicted. Apache Kafka fun. Kafka Summit and Meetup speaker. Currently, he is Principal Engineer at Nutmeg.

Jeremy Frenay is a platform engineer at Babylon Health.
He has been leading the Babylon's Data Operations efforts since late 2017, building the Kafka based data infrastructure, the automation and the tooling required to support teams of software and data engineers working on data integration pipelines in a cloud-native environment.

Jaksot(243)

The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022)

The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) ABSTRACT This talk will go through both the improvements that have been made in Kubernetes for batch analytic workloads as well as ...

31 Loka 202220min

Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022)

Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) ABSTRACT Sourcegraph is a code intelligence platform that helps our customers to understand their code better. As we have scaled up...

29 Loka 202221min

Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022)

Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract We at OpsVerse provide a DevOps tools platform with fully-managed open source-based tools. One of our key offerings is a ...

28 Loka 202213min

Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022)

Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract We develop systems to digitize the sheet metal industry with the belief that they should cooperate with each other in an ...

27 Loka 202220min

How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY) Abstract Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent ...

26 Loka 202216min

Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández

Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández

https://go.dok.community/slack https://dok.community We are going to speak about CRDs, and discuss considering them as higher level entities that we normally consider them. CRDs normally are kind of a...

14 Loka 202258min

Dok #152-Running PostgreSQL in Kubernetes:from day 0 to day 2 with CloudNativePG // Gabriele Bartolini

Dok #152-Running PostgreSQL in Kubernetes:from day 0 to day 2 with CloudNativePG // Gabriele Bartolini

https://go.dok.community/slack https://dok.community With: Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB Bart Farrell - Head of Community, Data on Kubernetes Community...

28 Syys 20221h 3min

Dok Talks #148 - Cost and Kubernetes // Chris Love

Dok Talks #148 - Cost and Kubernetes // Chris Love

https://go.dok.community/slack https://dok.community With: Chris Love - Managing Partner, LionKube Bart Farrell - Head of Community, Data on Kubernetes Community ABSTRACT OF THE TALK Using Kuberne...

27 Syys 202245min