25. What can you do with Kinesis Data Streams?
AWS Bites25 Feb 2022

25. What can you do with Kinesis Data Streams?

Eoin and Luciano are back with some more AWS messaging services material! This time we talk about Kinesis Data Streams and everything there’s to know about them! In this episode of AWS Bites, we cover the difference between data streaming and queue or message buses. How Kinesis can let you process large batches of messages in near real time. Why you might use Kinesis as a queue or pub/sub bus in some cases. A few different ways to use Kinesis with the focus on avoiding too much complexity. We also provide some insights based on our experience with Kinesis Data Streams including real use cases, monitoring tips and some gotchas to be aware of.

In this episode, we mentioned the following resources:

- Our introductory episode about what services you should use for events: https://www.youtube.com/watch?v=CG7uhkKftoY

- Amazon Kinesis Producer Library (KPL): https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html

- Amazon Kinesis Consumer Library (KCL): https://docs.aws.amazon.com/streams/latest/dev/shared-throughput-kcl-consumers.html

- Kinesis Library wrapper for Node.js: https://github.com/awslabs/amazon-kinesis-client-nodejs

- Kinesis Library wrapper for Python: https://github.com/awslabs/amazon-kinesis-client-python

- Kinesis Data Stream with captures from DynamoDB (for CDC): https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/kds.html

- Kinesis Data Stream with captures from Aurora: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.Overview.html

- Kinesis Data Analytics: https://aws.amazon.com/kinesis/data-analytics/

- Kinesis Firehose: https://aws.amazon.com/kinesis/data-firehose/

- Must-read on Kinesis: Anahit Pogosova’s two-part blog series: https://dev.solita.fi/2020/05/28/kinesis-streams-part-1.html & https://dev.solita.fi/2020/12/21/kinesis-streams-part-2.html

- Cloudonaut Kinesis vs MSK: https://cloudonaut.io/versus/messaging/kinesis-data-streams-vs-msk/

- Deep Dive on Lambda Consumers for Kinesis / Heeki Park - re:invent 2020 https://www.youtube.com/watch?v=tCYwc7-wwsU

You can listen to AWS Bites wherever you get your podcasts. Find all the links on https://awsbites.com

Do you have any AWS questions you would like us to address?

Leave a comment here or connect with us on Twitter:
- https://twitter.com/eoins
- https://twitter.com/loige

Episoder(157)

145. We Tried Amazon DSQL So You Don’t Have To (But You Might Want To)

145. We Tried Amazon DSQL So You Don’t Have To (But You Might Want To)

Amazon Aurora DSQL promises to bring a truly serverless experience to SQL databases. But does it actually deliver? In this episode of AWS Bites, we put Aurora DSQL to the test. We explore what makes i...

26 Jun 202528min

144. Lambda Billing Changes, Cold Start Costs, and Log Savings: What You Need to Know

144. Lambda Billing Changes, Cold Start Costs, and Log Savings: What You Need to Know

Cost is always top of mind when building in the cloud, and recently AWS has introduced some changes worth paying attention to. In this episode of AWS Bites, we explore a shift that caught many by surp...

22 Mai 202513min

143. Is App Runner better than Fargate?

143. Is App Runner better than Fargate?

Picture this. You’ve got a web app built with Rust and Solid.js. It started life running on a dusty on-prem server, but now it's time to move it to the cloud. The clock is ticking. You could take the ...

8 Mai 202542min

142. Escape from S3

142. Escape from S3

We discuss common use cases and challenges for copying data between S3 buckets and S3-compatible object storage services. We share our experience building an open source Node.js CLI tool called S3-Mig...

3 Apr 202530min

141. Step Functions with JSONata and Variables

141. Step Functions with JSONata and Variables

In this episode, we provide an overview of AWS Step Functions and dive deep into the powerful new JSONata and variables features. We explain how JSONata allows complex JSON transformations without cus...

21 Mar 202515min

140. DuckDB Meets AWS: A Match Made in Cloud

140. DuckDB Meets AWS: A Match Made in Cloud

In this episode, we explore DuckDB, an open-source analytical database known for its speed and simplicity. Discover how DuckDB stands out in various applications and compare it to other tools like SQL...

21 Feb 202517min

139. Building Great APIs with Powertools

139. Building Great APIs with Powertools

In this episode, we discuss using AWS Lambda Powertools for Python to build serverless REST APIs with AWS Lambda. We cover the benefits of using Powertools for routing, validation, OpenAPI support, an...

19 Feb 202524min

138. How Do You Become A Cloud Architect?

138. How Do You Become A Cloud Architect?

Ready to take your tech career to the cloud and build those awe-inspiring systems you see? Then you're in the right place. This episode of AWS Bites is your blueprint for becoming a successful cloud a...

10 Jan 202539min

Populært innen Teknologi

lydartikler-fra-aftenposten
romkapsel
smart-forklart
teknisk-sett
tomprat-med-gunnar-tjomlid
energi-og-klima
nasjonal-sikkerhetsmyndighet-nsm
rss-impressions-2
elektropodden
rss-ki-praten
shifter
rss-praktisk-proptech
pedagogisk-intelligens
hans-petter-og-co
rss-ki-til-kaffen
i-loopen
kunstig-intelligens-med-morten-goodwin
rss-heis
teknologi-og-mennesker
rss-nerding-med-netlife