The New Stack Podcast5 Sep 2023

Apache Flink for Real Time Data Analysis

This episode delves into Apache Flink, a versatile platform for executing both batch and real-time streaming data analysis tasks. This session marks the beginning of a three-part series unveiling Amazon Web Services' (AWS) new managed service built on Flink. Future episodes will explore this service in detail and examine customer experiences.

The podcast features insights from Danny Cranmer, a principal engineer at AWS and an Apache Flink PMC and Committer, along with Hong Teoh, a software development engineer at AWS.

Flink stands out as a high-level framework for defining data analytics jobs, accommodating both batch and streaming data sets. It offers APIs for building analysis jobs in various languages, including Java, Python, and SQL. Flink also provides a distributed job execution engine with fault tolerance and horizontal scaling capabilities.

One prominent use case is Extract-Transform-Load (ETL), where raw data is swiftly processed for specific workloads. Flink excels in delivering low-latency transformations for unbounded data streams. Additionally, Flink supports event-driven applications, responding immediately to triggers such as user requests for weather data.

Flink ensures exactly-once processing, critical for scenarios like financial transactions. It employs checkpoints to maintain data integrity in case of node failures.

The podcast also touches on AWS's role in supporting the open-source Flink project and the future outlook for this powerful data processing framework.

Learn more from The New Stack about Apache Flink:

3 Reasons Why You Need Apache Flink for Stream Processing

Apache Flink for Unbounded Data Streams

8 Real-Time Data Best Practices

Denne episoden er hentet fra en åpen RSS-feed og er ikke publisert av Podme. Den kan derfor inneholde annonser.

Episoder(300)

Dynatrace's $915M Arize deal bets AI agents are just another app to monitor

Dynatrace completed its $915 million acquisition of Arize on Oct. 1, combining Dynatrace’s application and infrastructure monitoring with Arize’s AI agent tracing and evaluation capabilities. The deal...

5 Okt 20min

Bit Cloud’s next chapter starts after the AI builds your app

For many developers, turning an AI-generated prototype into maintainable software requires more than generating code—it requires infrastructure, collaboration, testing and review. In this episode of T...

1 Okt 31min

CloudBees just committed to an AI-first pivot. Here's why it matters for enterprise DevOps teams

CloudBees CEO Mo Plassnig is leading the CI/CD company through a major transformation as generative AI reshapes software development. Returning to CloudBees eight years after joining through its acqui...

30 Sep 23min

A third option is emerging in the fight over AI and your data

The AI industry has faced a growing enterprise dilemma: companies want access to powerful proprietary AI models without risking sensitive data or intellectual property, while AI labs want to protect t...

23 Sep 30min

Drowning in AI pull requests: Harness's field CTO on code review and a Git repo built for agents

Harness Field CTO Martin Reynolds joins The New Stack to talk about what happens after coding agents start opening pull requests faster than anyone can review them. He explains how he first saw the bo...

7 Sep 26min

How to find failures without drowning in tracing data

Traces provide a detailed view of a request’s journey through data, microservices and applications, helping SREs pinpoint where failures occur and resolve issues faster. But while tracing can reduce d...

3 Sep 31min

Why CPUs still matter in the age of AI agents

As AI evolves from conversational chatbots to autonomous agents, CPUs are becoming an increasingly important part of the infrastructure equation. In this episode, The New Stack speaks with Bhumik Pate...

11 Aug 26min

Why Doist Says Less AI Can Deliver More

Doist CTO Gonçalo Silva says AI is reshaping software development, but success depends on restraint rather than rapid feature expansion. Instead of chasing every AI capability, Doist prioritizes “subt...

31 Jul 35min