26. What can you do with Kafka on AWS?
AWS Bites4 Maalis 2022

26. What can you do with Kafka on AWS?

Luciano and Eoin explore the wonderful world of data streaming using Kafka on AWS. In this episode we focus mainly on Managed Streaming for Kafka (or MSK) and discuss what are the main differences between MSK and Kinesis. We also explore the main features that MSK provides, its scaling characteristics, pricing and, finally, how MSK works in conjunction with other AWS services.

We conclude the episode by providing a decision tree that should help you to decide whether you should use Kinesis or MSK or avoid streaming services entirely in favor of something like SNS or SQS.

In this episode, we mentioned the following resources:

- Our previous episode on Kinesis data streams: https://www.youtube.com/watch?v=u_nR6up4Kvs

- Our series of Event services: https://www.youtube.com/watch?v=CG7uhkKftoY&list=PLAWXFhe0N1vLHkGO1ZIWW_SZpturHBiE_

- AWS MSK sizing spreadsheet: https://dy7oqpxkwhskb.cloudfront.net/MSK_Sizing_Pricing.xlsx

- Should My Startup use Kinesis or MSK? - https://www.youtube.com/watch?v=TJS19EuzH2k

- Intro to MSK (reinvent talk from 2018) - https://www.youtube.com/watch?v=9nKswHsLseY

- Running Apache Kafka on AWS (by Frank Munz) - https://www.youtube.com/watch?v=HtU9pb18g5Q

- Cloudonaut - Kinesis versus MSK - https://www.youtube.com/watch?v=kcBAKz0MPf8

This episode is also available on YouTube: https://www.youtube.com/AWSBites

You can listen to AWS Bites wherever you get your podcasts:

- Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017

- Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q

- Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==

- Breaker: https://www.breaker.audio/aws-bites

- RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:
- https://twitter.com/eoins
- https://twitter.com/loige

Jaksot(157)

121. 5 Ways to extend CloudFormation

121. 5 Ways to extend CloudFormation

In this episode, we discuss 5 different ways to extend CloudFormation capabilities beyond what it natively supports. We started with a quick recap of what CloudFormation is and why we might need to ex...

18 Huhti 202430min

120. Lambda Best Practices

120. Lambda Best Practices

In this episode, we discuss best practices for working with AWS Lambda. We cover how Lambda functions work under the hood, including cold starts and warm starts. We then explore different invocation t...

4 Huhti 202426min

119. The state of AWS 2024 (AnsWeRS community survey commentary)

119. The state of AWS 2024 (AnsWeRS community survey commentary)

In this episode, we provide commentary and analysis on the 2024 AWS Community Survey results. We go through the key findings for each area including infrastructure as code, CI/CD, serverless, containe...

22 Maalis 202439min

118. The landing zone: Managing multiple AWS accounts

118. The landing zone: Managing multiple AWS accounts

In this episode, we provide an introductory overview of AWS's best practices for managing infrastructure using multiple accounts under an organization. We discuss the advantages of this approach and h...

15 Maalis 202425min

117. What do EBS and a jellyfish have in common?

117. What do EBS and a jellyfish have in common?

In this episode, we provide an overview of Amazon EBS, which stands for Elastic Block Storage. We explain what block storage is and how EBS provides highly available and high-performance storage volum...

8 Maalis 202421min

116. What is RAM (Resource Access Manager)?

116. What is RAM (Resource Access Manager)?

In this episode, we discuss AWS Resource Access Manager (RAM) and how it can be used to securely share AWS resources like VPC subnets, databases, and SSM parameters across accounts. We explain the ben...

1 Maalis 202413min

115. What can you do with Permissions Boundaries?

115. What can you do with Permissions Boundaries?

In this episode, we discuss Permission Boundary policies in AWS IAM. A permissions boundary is an advanced feature in which you set the maximum permissions that an identity-based policy can grant to a...

23 Helmi 202413min

114. What's up with LLRT, AWS' new Lambda Runtime?

114. What's up with LLRT, AWS' new Lambda Runtime?

In this episode, we discuss the new experimental AWS Lambda LLRT Low Latency runtime for JavaScript. We provide an overview of what a Lambda runtime is and how LLRT aims to optimize cold starts and pe...

16 Helmi 202430min