26. What can you do with Kafka on AWS?
AWS Bites4 Maalis 2022

26. What can you do with Kafka on AWS?

Luciano and Eoin explore the wonderful world of data streaming using Kafka on AWS. In this episode we focus mainly on Managed Streaming for Kafka (or MSK) and discuss what are the main differences between MSK and Kinesis. We also explore the main features that MSK provides, its scaling characteristics, pricing and, finally, how MSK works in conjunction with other AWS services.

We conclude the episode by providing a decision tree that should help you to decide whether you should use Kinesis or MSK or avoid streaming services entirely in favor of something like SNS or SQS.

In this episode, we mentioned the following resources:

- Our previous episode on Kinesis data streams: https://www.youtube.com/watch?v=u_nR6up4Kvs

- Our series of Event services: https://www.youtube.com/watch?v=CG7uhkKftoY&list=PLAWXFhe0N1vLHkGO1ZIWW_SZpturHBiE_

- AWS MSK sizing spreadsheet: https://dy7oqpxkwhskb.cloudfront.net/MSK_Sizing_Pricing.xlsx

- Should My Startup use Kinesis or MSK? - https://www.youtube.com/watch?v=TJS19EuzH2k

- Intro to MSK (reinvent talk from 2018) - https://www.youtube.com/watch?v=9nKswHsLseY

- Running Apache Kafka on AWS (by Frank Munz) - https://www.youtube.com/watch?v=HtU9pb18g5Q

- Cloudonaut - Kinesis versus MSK - https://www.youtube.com/watch?v=kcBAKz0MPf8

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:
- https://twitter.com/eoins
- https://twitter.com/loige

Jaksot(157)

129. Lambda Provisioned Concurrency

129. Lambda Provisioned Concurrency

In this episode, we discuss AWS Lambda provisioned concurrency. We start with a recap of Lambda cold starts and the different concurrency control options. We then explain how provisioned concurrency w...

22 Elo 202422min

128. Writing a book about Rust & Lambda

128. Writing a book about Rust & Lambda

In this episode, we discuss Luciano's new book project on using Rust to write AWS Lambda functions. We start with a recap on why Rust is a good fit for Lambda, including performance, efficiency, safet...

25 Heinä 202426min

127. Which Load Balancer should you use?

127. Which Load Balancer should you use?

An overview of load balancers, explaining how they distribute traffic across multiple servers and provide high availability. We discuss layer 4 and layer 7 load balancers, detailing their pros and con...

11 Heinä 202424min

126. Bastion Containers

126. Bastion Containers

This episode discusses solutions for securely accessing private VPC resources for debugging and troubleshooting. We cover traditional approaches like bastion hosts and VPNs and newer solutions using c...

27 Kesä 202413min

125. A first look at CloudFront Hosting Toolkit

125. A first look at CloudFront Hosting Toolkit

In this episode, we discuss the newly announced CloudFront Hosting Toolkit from AWS. We provide an overview of the tool, which aims to simplify deploying modern front-end applications to AWS while ret...

13 Kesä 202433min

124. S3 Performance

124. S3 Performance

In this episode, we discuss some tips and tricks for optimizing performance when working with Amazon S3 at scale. We start by giving an overview of how S3 works, highlighting the distributed nature of...

30 Touko 202420min

123. What do you need to know about DynamoDB?

123. What do you need to know about DynamoDB?

In this episode, we provide a comprehensive overview of DynamoDB, including how it compares to relational databases, when to use it, how to get started, writing and querying data, secondary indexes, a...

16 Touko 202435min

122. Amazing Databases with Aurora

122. Amazing Databases with Aurora

In this episode, we provide an overview of Amazon Aurora, a relational database solution on AWS. We discuss its unique capabilities like distinct storage architecture for better performance and faster...

2 Touko 202428min