(Part 4/4) Confluent Cloud (Managed Kafka as a Service) - What is a connector & How to create Custom Connectors

(Part 4/4) Confluent Cloud (Managed Kafka as a Service) - What is a connector & How to create Custom Connectors

In this podcast, Krish explores the various connectors available in Confluent Cloud. He starts by recapping the previous podcasts and the basics of Confluent Cloud. Krish then focuses on connectors, explaining their value and why they can reduce the need for writing code. He explores different connectors, such as the data gen source connector and the MongoDB Atlas connectors. Krish also discusses different data formats, including Avro, Protobuf, and JSON. He briefly touches on implementing custom connectors. Krish explores the topic of connectors in Confluent Cloud. He discusses the process of creating connectors and the different types of connectors available. Krish also delves into configuring connectors and defining configuration parameters. He explores the concept of custom connector configuration and the use of connector properties files. Krish then explores existing connectors, such as the HTTP source and sync connectors, and discusses the process of publishing custom connectors. He concludes by mentioning the Confluent CLI for managing connectors.

Takeaways

  • Connectors in Confluent Cloud provide value by reducing the need for writing code.
  • Different connectors are available for various data sources and destinations, such as MongoDB, Amazon S3, and Elasticsearch.
  • Data formats like Avro, Protobuf, and JSON can be used with connectors.
  • Implementing custom connectors allows for more flexibility and integration with specific systems.
  • Connectors enable seamless data integration and propagation between different systems. Connectors in Confluent Cloud allow for seamless integration with various systems and services.
  • Custom connectors can be created and published to Confluent Cloud.
  • Configuration parameters for connectors can be defined and managed.
  • The Confluent CLI provides a command-line interface for managing connectors.

Chapters

00:00 Introduction

00:35 Recap of Previous Podcasts

01:05 Focus on Connectors in Confluent Cloud

02:16 Exploring Data Gen Source Connector

03:43 Different Formats: Avro, Protobuf, JSON

08:07 Differences Between Avro and Protobuf

10:03 Exploring Other Connectors

11:14 Using MongoDB Atlas Connectors

12:08 Testing Different Formats with Connectors

13:36 Handling Avro Format with Consumer

16:58 Exploring More Connectors: Snowflake, Amazon S3, Elasticsearch

20:33 Implementing Custom Connectors

27:31 Exploring More Connectors: Salesforce, Oracle, Jira

35:16 Exploring More Connectors: SQL Server, MySQL

38:43 Implementing Custom Connectors

43:24 Exploring More Connectors: Kafka, File

46:20 Understanding Connector Implementation

49:06 Creating Custom Connectors

50:00 Summary and Conclusion

50:59 Creating Connectors

52:04 Configuring Connectors

54:00 Custom Connector Configuration

56:08 Defining Configuration Parameters

57:38 Configuration Properties

59:49 Self-Managed Connectors

01:00: 27 Connector Properties File

01:01:28 Creating Custom Connectors

01:02: 09 Publishing Custom Connectors

01:03: 37Existing Connectors

01:04: 14HTTP Source Connector

01:06:40 HTTP Sync Connector

01:08:34 Other Connectors

01:10:34 Managing Connectors

01:12:14 Confluent CLI

Snowpal Products

Avsnitt(411)

(Feat. Special Guest!) Pros & Cons of Technology Choices (Part 2/3)

(Feat. Special Guest!) Pros & Cons of Technology Choices (Part 2/3)

(Part 2/3) When you are building a new app, you are bound to assess a variety of options available to you. Some of these choices are going to stick around with you for a while so it is imperative to make the right choices (with the information available at that point).   Manage your projects on https://snowpal.com.

26 Mars 20219min

(Feat. Special Guest!) Pros & Cons of Technology Choices (Part 1/3)

(Feat. Special Guest!) Pros & Cons of Technology Choices (Part 1/3)

(Part 1/3) When you are building a new app, you are bound to assess a variety of options available to you. Some of these choices are going to stick around with you for a while so it is imperative to make the right choices (with the information available at that point).   Manage your projects on https://snowpal.com.

25 Mars 20219min

Upgrade Dart version to 2.12.0 to leverage null safety

Upgrade Dart version to 2.12.0 to leverage null safety

Starting dart version 2.12.0, there is support for null safety. Added to type safety, this will make flutter even more compelling. Having said that, it will take some work for an existing app to get there.   Manage your projects on https://snowpal.com.

22 Mars 20218min

Polyglot Development is the best way to go today

Polyglot Development is the best way to go today

Polyglot Programming can be loosely defined as a programming methodology where you pick the right language, library or framework that best suits the needs of the given problem (and in particular, not let your "current" stack drive that decision).   Manage your projects on https://snowpal.com.

21 Mars 20218min

(Part 1/N) Upgrading to Flutter 2.0.x?

(Part 1/N) Upgrading to Flutter 2.0.x?

Every time I run "flutter upgrade", I tend to have my fingers crossed. The most recent upgrade to 2.0.x was quite a challenge. I'll share my experience (both problems and solutions) in a series of podcasts.   Manage your projects on https://snowpal.com.

16 Mars 20219min

Asynchronous Processing: Don't keep your users waiting. Accept request, process later.

Asynchronous Processing: Don't keep your users waiting. Accept request, process later.

Is there ever a good reason to do something synchronously when there is a way for you to do it asynchronously? Let's take a look at how we can implement asynchronous services using RabbitMQ.   Manage your projects on https://snowpal.com.

5 Mars 20219min

Do you get anxious about LIVE Coding Interviews? Don't be.

Do you get anxious about LIVE Coding Interviews? Don't be.

It is quite natural to get anxious when you know people are going to be judging you. After all, isn't that one of the primary purposes of an interview? I may have a few tips to help you be at ease.   Manage your projects on https://snowpal.com.

21 Feb 20219min

User Experience (UX) is not just the designer's responsibility

User Experience (UX) is not just the designer's responsibility

How many times have we used an app (on the web or mobile) and thought to ourselves, "Why in the world would someone have designed it this way?". That's plenty of times for me and here I talk about one of those experiences.   Manage your projects on https://snowpal.com.

19 Feb 20219min

Populärt inom Teknik

uppgang-och-fall
elbilsveckan
market-makers
natets-morka-sida
skogsforum-podcast
rss-uppgang-och-fall
rss-elektrikerpodden
bli-saker-podden
rss-veckans-ai
har-vi-akt-till-mars-an
hej-bruksbil
rss-upplyst-entreprenordirektor
allt-du-behover-veta-om-ny-teknik
bilar-med-sladd
gubbar-som-tjotar-om-bilar
developers-mer-an-bara-kod
rss-fabriken-2
rss-digitala-influencer-podden
rss-badfluence
solcellskollens-podcast