26. What can you do with Kafka on AWS?
AWS Bites4 Maalis 2022

26. What can you do with Kafka on AWS?

Luciano and Eoin explore the wonderful world of data streaming using Kafka on AWS. In this episode we focus mainly on Managed Streaming for Kafka (or MSK) and discuss what are the main differences between MSK and Kinesis. We also explore the main features that MSK provides, its scaling characteristics, pricing and, finally, how MSK works in conjunction with other AWS services.

We conclude the episode by providing a decision tree that should help you to decide whether you should use Kinesis or MSK or avoid streaming services entirely in favor of something like SNS or SQS.

In this episode, we mentioned the following resources:

- Our previous episode on Kinesis data streams: https://www.youtube.com/watch?v=u_nR6up4Kvs

- Our series of Event services: https://www.youtube.com/watch?v=CG7uhkKftoY&list=PLAWXFhe0N1vLHkGO1ZIWW_SZpturHBiE_

- AWS MSK sizing spreadsheet: https://dy7oqpxkwhskb.cloudfront.net/MSK_Sizing_Pricing.xlsx

- Should My Startup use Kinesis or MSK? - https://www.youtube.com/watch?v=TJS19EuzH2k

- Intro to MSK (reinvent talk from 2018) - https://www.youtube.com/watch?v=9nKswHsLseY

- Running Apache Kafka on AWS (by Frank Munz) - https://www.youtube.com/watch?v=HtU9pb18g5Q

- Cloudonaut - Kinesis versus MSK - https://www.youtube.com/watch?v=kcBAKz0MPf8

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:
- https://twitter.com/eoins
- https://twitter.com/loige

Jaksot(156)

144. Lambda Billing Changes, Cold Start Costs, and Log Savings: What You Need to Know

144. Lambda Billing Changes, Cold Start Costs, and Log Savings: What You Need to Know

Cost is always top of mind when building in the cloud, and recently AWS has introduced some changes worth paying attention to. In this episode of AWS Bites, we explore a shift that caught many by surp...

22 Touko 202513min

143. Is App Runner better than Fargate?

143. Is App Runner better than Fargate?

Picture this. You’ve got a web app built with Rust and Solid.js. It started life running on a dusty on-prem server, but now it's time to move it to the cloud. The clock is ticking. You could take the ...

8 Touko 202542min

142. Escape from S3

142. Escape from S3

We discuss common use cases and challenges for copying data between S3 buckets and S3-compatible object storage services. We share our experience building an open source Node.js CLI tool called S3-Mig...

3 Huhti 202530min

141. Step Functions with JSONata and Variables

141. Step Functions with JSONata and Variables

In this episode, we provide an overview of AWS Step Functions and dive deep into the powerful new JSONata and variables features. We explain how JSONata allows complex JSON transformations without cus...

21 Maalis 202515min

140. DuckDB Meets AWS: A Match Made in Cloud

140. DuckDB Meets AWS: A Match Made in Cloud

In this episode, we explore DuckDB, an open-source analytical database known for its speed and simplicity. Discover how DuckDB stands out in various applications and compare it to other tools like SQL...

21 Helmi 202517min

139. Building Great APIs with Powertools

139. Building Great APIs with Powertools

In this episode, we discuss using AWS Lambda Powertools for Python to build serverless REST APIs with AWS Lambda. We cover the benefits of using Powertools for routing, validation, OpenAPI support, an...

19 Helmi 202524min

138. How Do You Become A Cloud Architect?

138. How Do You Become A Cloud Architect?

Ready to take your tech career to the cloud and build those awe-inspiring systems you see? Then you're in the right place. This episode of AWS Bites is your blueprint for becoming a successful cloud a...

10 Tammi 202539min

137. Transit Gateway Explained

137. Transit Gateway Explained

In this episode, David Lynam provides an overview of AWS Transit Gateway, which aims to simplify complex network connectivity between VPCs, VPNs, and on-premises networks. We discuss the limitations o...

13 Joulu 202418min