(Part 4/4) Confluent Cloud (Managed Kafka as a Service) - What is a connector & How to create Custom Connectors

(Part 4/4) Confluent Cloud (Managed Kafka as a Service) - What is a connector & How to create Custom Connectors

In this podcast, Krish explores the various connectors available in Confluent Cloud. He starts by recapping the previous podcasts and the basics of Confluent Cloud. Krish then focuses on connectors, explaining their value and why they can reduce the need for writing code. He explores different connectors, such as the data gen source connector and the MongoDB Atlas connectors. Krish also discusses different data formats, including Avro, Protobuf, and JSON. He briefly touches on implementing custom connectors. Krish explores the topic of connectors in Confluent Cloud. He discusses the process of creating connectors and the different types of connectors available. Krish also delves into configuring connectors and defining configuration parameters. He explores the concept of custom connector configuration and the use of connector properties files. Krish then explores existing connectors, such as the HTTP source and sync connectors, and discusses the process of publishing custom connectors. He concludes by mentioning the Confluent CLI for managing connectors.

Takeaways

  • Connectors in Confluent Cloud provide value by reducing the need for writing code.
  • Different connectors are available for various data sources and destinations, such as MongoDB, Amazon S3, and Elasticsearch.
  • Data formats like Avro, Protobuf, and JSON can be used with connectors.
  • Implementing custom connectors allows for more flexibility and integration with specific systems.
  • Connectors enable seamless data integration and propagation between different systems. Connectors in Confluent Cloud allow for seamless integration with various systems and services.
  • Custom connectors can be created and published to Confluent Cloud.
  • Configuration parameters for connectors can be defined and managed.
  • The Confluent CLI provides a command-line interface for managing connectors.

Chapters

00:00 Introduction

00:35 Recap of Previous Podcasts

01:05 Focus on Connectors in Confluent Cloud

02:16 Exploring Data Gen Source Connector

03:43 Different Formats: Avro, Protobuf, JSON

08:07 Differences Between Avro and Protobuf

10:03 Exploring Other Connectors

11:14 Using MongoDB Atlas Connectors

12:08 Testing Different Formats with Connectors

13:36 Handling Avro Format with Consumer

16:58 Exploring More Connectors: Snowflake, Amazon S3, Elasticsearch

20:33 Implementing Custom Connectors

27:31 Exploring More Connectors: Salesforce, Oracle, Jira

35:16 Exploring More Connectors: SQL Server, MySQL

38:43 Implementing Custom Connectors

43:24 Exploring More Connectors: Kafka, File

46:20 Understanding Connector Implementation

49:06 Creating Custom Connectors

50:00 Summary and Conclusion

50:59 Creating Connectors

52:04 Configuring Connectors

54:00 Custom Connector Configuration

56:08 Defining Configuration Parameters

57:38 Configuration Properties

59:49 Self-Managed Connectors

01:00: 27 Connector Properties File

01:01:28 Creating Custom Connectors

01:02: 09 Publishing Custom Connectors

01:03: 37Existing Connectors

01:04: 14HTTP Source Connector

01:06:40 HTTP Sync Connector

01:08:34 Other Connectors

01:10:34 Managing Connectors

01:12:14 Confluent CLI

Snowpal Products

Jaksot(410)

The Social Contract of Immigration: Why Assimilation Still Matters (feat. Nikki Barua)

The Social Contract of Immigration: Why Assimilation Still Matters (feat. Nikki Barua)

Welcome to the Snowpal Podcast, where we explore technology, leadership, and the ideas shaping the future. I’m your host, Krish Palaniappan, founder of Snowpal and a long-time software builder and entrepreneur, and on this show we go beyond surface-level trends to unpack real experiences and hard-earned insights. Today’s conversation is both timely and deeply personal. Our guest is Nikki Barua (LinkedIn), a serial entrepreneur, keynote speaker, bestselling author, and the CEO of FlipWork, where she helps people and organizations prepare for the AI age. Nikki’s work sits at the intersection of technology, culture, and human potential, but her story goes far beyond the boardroom. As an immigrant who has navigated visas, uncertainty, and reinvention over decades, she brings a deeply lived perspective to today’s discussion. In this episode, we explore what immigration really means beyond headlines and politics, diving into themes of belonging, assimilation, and cultural responsibility. We examine the difference between entitlement and privilege, why immigrants often become entrepreneurs and leaders, and how identity, contribution, and shared values shape acceptance. This is a conversation about people, not policies—about responsibility, opportunity, and the social contracts that bind us together in a rapidly changing world.

15 Tammi 36min

AI Demands Reinvention, Not Optimization: The Operating System of Future-Ready Companies (feat. Nikki Barua)

AI Demands Reinvention, Not Optimization: The Operating System of Future-Ready Companies (feat. Nikki Barua)

In this conversation, Nikki Barua (LinkedIn), CEO of FlipWork, discusses the critical need for companies to fundamentally reinvent themselves rather than merely transform. She emphasizes the difference between evolutionary and revolutionary change, highlighting the importance of leadership, cultural shifts, and the iterative process of change. Nikki shares insights on how organizations can identify focus areas for change, engage teams in the process, and navigate the complexities of implementing new strategies in a rapidly evolving business landscape. In this conversation, Nikki Barua and Krish Palaniappan explore the complexities of high-stakes decision-making, the future of software development, and the impact of AI on careers. They discuss personal reflections on identity and belonging, particularly in the context of immigration and cultural assimilation. The dialogue emphasizes the importance of adapting to new environments while maintaining one’s cultural roots, and the role of language in fostering connections. Ultimately, they highlight the shared human experience of seeking belonging and the potential for innovation in a rapidly changing world.

15 Tammi 1h 17min

From Product Design to Healthcare Lending: The Entrepreneurial Journey (feat. Sharmeen Aqeel)

From Product Design to Healthcare Lending: The Entrepreneurial Journey (feat. Sharmeen Aqeel)

In the fast-evolving world of medical spas and aesthetic clinics, securing funding is a critical step for growth and sustainability. In this episode of the Snowpal podcast, host Krish Palaniappan speaks with Sharmeen Aqeel, founder of Lyyvora, a lending platform tailored for healthcare businesses. Sharmeen shares her journey, the challenges faced by clinic owners, and how her platform is revolutionizing the lending process for medical spas across North America.

13 Tammi 1h

Oracle, Snowflake, and Datadog: Three Cloud Giants, Three Very Different Stories

Oracle, Snowflake, and Datadog: Three Cloud Giants, Three Very Different Stories

In this finance-focused discussion, we examine three companies that often appear together in conversations about cloud computing and data—but operate in meaningfully different segments of the market: Oracle, Snowflake, and Datadog.

9 Tammi 17min

When User Experience Goes Wrong: A Cautionary Tale for Product Builders

When User Experience Goes Wrong: A Cautionary Tale for Product Builders

In today’s conversations about software, AI, and innovation, it’s easy to forget one critical truth: humans still use the products we build. No matter how powerful the backend or how advanced the algorithms, a poor user experience can render a product nearly unusable.

9 Tammi 23min

Early 2026 Stock Market: Momentum, Volatility, and What Traders Are Watching

Early 2026 Stock Market: Momentum, Volatility, and What Traders Are Watching

The first four trading days of 2026 have delivered exactly what active market participants look for: movement, volume, and clear intraday patterns. While the year is still young, early signals across indices, semiconductors, hyperscalers, and financials are already shaping trader sentiment and short-term strategies .

8 Tammi 18min

What AI Is Forcing Every Founder to Rethink (feat. Suri Nagarajan)

What AI Is Forcing Every Founder to Rethink (feat. Suri Nagarajan)

In this engaging conversation, Krish Palaniappan and Suri Nagarajan delve into the intricacies of investing, particularly in the tech sector versus non-tech industries. They discuss the importance of financial literacy, the risks associated with high-risk investments, and the significance of diversification for new investors. Suri shares insights on industry allocation for a hypothetical $100 investment, emphasizing stable sectors like healthcare and energy. The discussion also touches on the cyclical nature of markets, the risks of entrepreneurship versus employment, and personal reflections on career choices and cultural opportunities in the US. Ultimately, the conversation highlights the importance of pursuing one’s passions and the unique educational experiences available in America.

4 Tammi 1h 57min

Happy New Year 2026: The Boring Path to an Interesting Life

Happy New Year 2026: The Boring Path to an Interesting Life

Only take vacations you can afford—debt and relaxation should never meet. If it needs installments and isn’t food or medicine, you don’t need it. Credit card debt is just regret charging interest—kill it first. Don’t enroll in college by muscle memory; 2026 doesn’t care about tradition. If you switch careers, pick a role that actually matters—vibes don’t pay rent. Skipping AI in 2026 is like skipping email in 1999—bold, but stupid. Invest real money if you can, fake money if you can’t, ignorance is not an option. Park far away—walking is free cardio and cheaper than therapy. Eat because you’re hungry, not because the fridge made eye contact. There is no perfect time—there’s only now and a lot of excuses. Build a side hustle so layoffs don’t decide your personality. If every year feels the same, congratulations—you’re on career autopilot. Social media isn’t real life—it’s a heavily filtered lie with Wi-Fi. Comparison kills joy, and titles won’t fix insecurity. Everything important takes longer than promised—welcome to reality. Ignore naysayers—they’re deeply committed to being comfortable. One focused person with AI can now replace a small team—choose wisely. If your car works, keep it—nobody’s impressed by your loan balance. Be selfish enough to protect your time, energy, and sanity. Nothing is saturated—most people just quit early.

2 Tammi 31min