
(Part 4/4) Confluent Cloud (Managed Kafka as a Service) - What is a connector & How to create Custom Connectors
In this podcast, Krish explores the various connectors available in Confluent Cloud. He starts by recapping the previous podcasts and the basics of Confluent Cloud. Krish then focuses on connectors, explaining their value and why they can reduce the need for writing code. He explores different connectors, such as the data gen source connector and the MongoDB Atlas connectors. Krish also discusses different data formats, including Avro, Protobuf, and JSON. He briefly touches on implementing custom connectors. Krish explores the topic of connectors in Confluent Cloud. He discusses the process of creating connectors and the different types of connectors available. Krish also delves into configuring connectors and defining configuration parameters. He explores the concept of custom connector configuration and the use of connector properties files. Krish then explores existing connectors, such as the HTTP source and sync connectors, and discusses the process of publishing custom connectors. He concludes by mentioning the Confluent CLI for managing connectors. Takeaways Connectors in Confluent Cloud provide value by reducing the need for writing code. Different connectors are available for various data sources and destinations, such as MongoDB, Amazon S3, and Elasticsearch. Data formats like Avro, Protobuf, and JSON can be used with connectors. Implementing custom connectors allows for more flexibility and integration with specific systems. Connectors enable seamless data integration and propagation between different systems. Connectors in Confluent Cloud allow for seamless integration with various systems and services. Custom connectors can be created and published to Confluent Cloud. Configuration parameters for connectors can be defined and managed. The Confluent CLI provides a command-line interface for managing connectors. Chapters 00:00 Introduction 00:35 Recap of Previous Podcasts 01:05 Focus on Connectors in Confluent Cloud 02:16 Exploring Data Gen Source Connector 03:43 Different Formats: Avro, Protobuf, JSON 08:07 Differences Between Avro and Protobuf 10:03 Exploring Other Connectors 11:14 Using MongoDB Atlas Connectors 12:08 Testing Different Formats with Connectors 13:36 Handling Avro Format with Consumer 16:58 Exploring More Connectors: Snowflake, Amazon S3, Elasticsearch 20:33 Implementing Custom Connectors 27:31 Exploring More Connectors: Salesforce, Oracle, Jira 35:16 Exploring More Connectors: SQL Server, MySQL 38:43 Implementing Custom Connectors 43:24 Exploring More Connectors: Kafka, File 46:20 Understanding Connector Implementation 49:06 Creating Custom Connectors 50:00 Summary and Conclusion 50:59 Creating Connectors 52:04 Configuring Connectors 54:00 Custom Connector Configuration 56:08 Defining Configuration Parameters 57:38 Configuration Properties 59:49 Self-Managed Connectors 01:00: 27 Connector Properties File 01:01:28 Creating Custom Connectors 01:02: 09 Publishing Custom Connectors 01:03: 37Existing Connectors 01:04: 14HTTP Source Connector 01:06:40 HTTP Sync Connector 01:08:34 Other Connectors 01:10:34 Managing Connectors 01:12:14 Confluent CLI Snowpal Products Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
18 Tammi 20241h 13min

(Part 2/2) Remote Work - An Engineer’s Perspective (feat. Jason Crum)
In this podcast, Krish and Jason discuss the cultural aspects of remote work and the importance of knowing each other in a remote setting. They explore whether it is necessary for engineers to know each other and the responsibility of individuals and organizations in fostering relationships. They also touch on the impact of engagement length on building relationships and the importance of treating software development as an art. The conversation concludes with a discussion on the challenges of working with different cultures and the need to embrace diversity and learn from different cultures. Takeaways Building relationships and getting to know each other is important in a remote work setting. Both individuals and organizations have a responsibility in fostering relationships and creating a positive work culture. Engagement length can impact the depth of relationships and the quality of deliverables. Treating software development as an art and giving 100% effort leads to better results. Working with different cultures requires understanding and embracing diversity. Chapters 00:00 Introduction and Recap 00:46 The Importance of Knowing Each Other 03:48 The Individual vs. Organization Responsibility 06:12 Building Relationships in a Remote Setting 11:11 The Impact of Engagement Length 12:32 Treating Software Development as an Art 17:50 Giving 100% and Treating Software Development as an Art 22:23 Challenges of Working with Different Cultures 24:51 Embracing Diversity and Learning from Different Cultures 26:56 Closing Remarks Snowpal Products Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
18 Tammi 202428min

(Part 1/2) Remote Work - An Engineer’s Perspective (feat. Jason Crum)
In this podcast episode, Krish interviews Jason Crum, a Senior Enterprise Architect, about remote work from an engineer's perspective. They discuss various aspects of remote work, including pre-pandemic, during pandemic, and post-pandemic experiences, preferences for remote or in-office work, benefits and challenges of remote work, limitations of remote work, social interaction, pairing in remote work, implementing remote work in startups, team dynamics, and challenges of non-engineers in remote teams. Takeaways Remote work offers flexibility and the ability to manage time effectively. Remote work can limit social interaction and the ability to collaborate in person. Pairing in remote work can be challenging but can also be beneficial for certain tasks. Implementing remote work in startups requires considering team dynamics and ensuring everyone is either remote or in-office. Non-engineers in remote teams may face unique challenges. Chapters 00:00 Introduction and Welcome 01:10 Favorite Food Items 04:06 Remote Work Experience 06:23 Pre-Pandemic, During Pandemic, and Post-Pandemic Remote Work 10:33 Preference for Remote or In-Office Work 13:07 Benefits and Challenges of Remote Work 15:28 Limitations of Remote Work 19:16 Social Interaction and Remote Work 20:39 Pairing in Remote Work 24:46 Implementing Remote Work in Startups 27:50 Team Dynamics in Remote Work 29:43 Challenges of Non-Engineers in Remote Teams 32:21 Closing Comments Snowpal Products Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
18 Tammi 202435min

(Part 3/4) Confluent Cloud (Managed Kafka as a Service) - Create a Go client to consume messages
In this podcast episode, the host continues from the previous episodes in the Confluent Cloud series and focuses on consuming messages. The consumer code is reviewed, and the concept of group ID is explained. The process of initializing the consumer and subscribing to topics is demonstrated. The use of channels in Go for message consumption is explored. The host troubleshoots issues related to group ID and discusses the inconsistencies observed. The allocation of messages among multiple consumers with different group IDs is examined. The episode concludes with a summary of the learnings and an invitation to explore Snowpals' products and APIs. Takeaways Consuming messages in Confluent Cloud involves initializing the consumer and subscribing to topics. The group ID determines which consumers belong to which group and affects work sharing and fault tolerance. Messages can be consumed using channels in Go, allowing for continuous listening and processing. Troubleshooting issues with group ID and understanding message allocation among multiple consumers is important. Snowpal offers a range of products and APIs for software development and architecture. Chapters 00:00 Introduction and Recap 00:31 Starting to Consume Messages 01:44 Consumer Code Overview 03:08 Consumer Group ID04:04Initializing the Consumer 04:32 Consuming Messages with Channels 05:00 Publishing Messages and Consuming 06:25 Group ID and Offset 07:07 Troubleshooting Group ID08:29Group ID and Producer 09:12 Inconsistencies and Debugging 10:24 Cluster Settings and Group ID 11:08 Cluster and Topic Configuration 12:24 Consumer Group ID and Partitions 13:22 Consumer Group ID and Message Filtering 16:29 Consumer Group ID and Work Sharing 17:49 Producer and Consumer Group ID 18:35 Multiple Consumers and Group ID 19:54 Multiple Consumers with Different Group IDs 20:50 Consuming Messages with Multiple Consumers 22:07 Parallelism and Message Distribution 23:52 Consumer Group ID and Message Allocation 26:30 Consumer Group ID and Message Allocation (Continued) 28:30 Consumer Group ID and Message Allocation (Continued) 30:03 Consumer Group ID and Message Allocation (Continued) 35:45 Conclusion Snowpal Products Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
17 Tammi 202442min

(Part 2/4) Confluent Cloud (Managed Kafka as a Service) - Create a Go client to publish messages
In this podcast episode, the host continues the discussion on Confluent Cloud and focuses on adding a consumer and creating a Go client. The process of building a producer and troubleshooting and debugging common issues is also covered. The host explores topics such as topic creation, error handling, and configuration. Known issues and workarounds are discussed, along with cluster settings and security protocols. The episode concludes with final debugging and error handling techniques. In this conversation, Krish explores the process of publishing messages to a Kafka topic using a Go client. He encounters some issues along the way, such as delivery failures and SSL connection problems. However, after making some code changes and switching back and forth, the publishing starts working unexpectedly. Krish also discusses the use of Go channels in the producer and the importance of reading config and initializing the producer correctly. He concludes by mentioning the next steps, which involve consuming the messages from the topic. Takeaways Adding a consumer and creating a Go client are important steps in working with Confluent Cloud. Troubleshooting and debugging are essential skills when working with messaging systems like Kafka. Understanding topic creation, error handling, and configuration is crucial for successful message production. Being aware of known issues and their workarounds can save time and effort in troubleshooting. Configuring cluster settings and security protocols correctly is essential for smooth operation. Publishing messages to a Kafka topic using a Go client involves initializing the producer and ensuring the correct configuration. Go channels can be used in the producer to handle message production. Reading the config and initializing the producer correctly is crucial for successful message publishing. Issues such as delivery failures and SSL connection problems can be resolved by making code changes and switching back and forth. Chapters 00:00 Introduction and Recap 02:30 Adding a Consumer 03:44 Creating a Go Client 08:08 Building the Producer 10:55 Creating a Consumer 17:30 Troubleshooting and Debugging 21:02 Topic Creation and Message Production 25:48 Error Handling and Configuration 33:27 Continued Troubleshooting 46:20 Correcting Configuration Issues 55:41 Known Issues and Workarounds 59:12 Cluster Settings and Security Protocols 01:01:07 Final Debugging and Error Handling 01:02:19 Connecting to the Bootstrap Server 01:03:47 Using Channels 01:04:48 Replacing Code and Expecting a Broker and Topic 01:05:21 Building and Running with Broker and Topic 01:06:36 Using Go Channels in the Producer 01:07:16 Reading Config and Initializing the Producer 01:08:43 Delivery Failed and SSL Connection 01:10:13 Sending Messages via Postman and Code 01:11:02 Switching Code and Unexpected Working 01:11:39 Messages Sent and Refreshing Stand 01:12:55 Publishing to Different Topics 01:13:32 Publishing Messages and Minor Changes 01:14:00 Initializing the Producer and Randomizing Messages 01:15:09 Failed to Deliver Message and Event Types 01:17:00 Producing Messages with Go Routine 01:18:13 Producing Messages and Business Functionality 01:19:21 Producing Messages and Printing Output 01:21:48 Subscription to the Topic 01:22:37 Go Routine and Message Type 01:23:56 Event Types and Handling 01:30:07 Error Handling and Non-Existent Topic 01:32:12 Next Steps: Consuming Messages Snowpal Products: Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
13 Tammi 20241h 33min

(Part 1/4) Confluent Cloud (Managed Kafka as a Service) - Create a cluster, generate API keys, create topics, publish messages
In this podcast, the host explores Confluent Cloud, a fully managed Kafka service. The host shares their experience with RabbitMQ and Kafka and explains the value of using a managed service like Confluent Cloud. They walk through the process of signing up for an account, creating a cluster, generating API keys, and creating topics. The host also discusses the use of connectors and introduces ksqlDB and Apache Flink. They explore cluster settings, message consumption, and additional features of Confluent Cloud. The podcast concludes with a summary of the topics covered. Takeaways Confluent Cloud is a fully managed Kafka service that provides added value through pre-built connectors and ease of use. Creating a cluster, generating API keys, and creating topics are essential steps in getting started with Confluent Cloud. ksqlDB and Apache Flink offer stream processing capabilities and can be integrated with Confluent Cloud. Cluster settings, message consumption, and additional features like stream lineage and stream designer enhance the functionality of Confluent Cloud. Using a managed service like Confluent Cloud allows developers to focus on solving customer problems rather than managing infrastructure. Chapters 00:00 Introduction 02:25 Exploring Confluent Cloud 09:14 Creating a Cluster and API Keys 11:00 Creating Topics 13:20 Sending Messages to Topics 15:12 Introduction to ksqlDB and Apache Flink 17:03 Exploring Connectors 25:44 Cluster Settings and Configuration 28:05 Consuming Messages 35:20 Stream Lineage and Stream Designer 38:44 Exploring Additional Features 44:21 Summary and Conclusion Snowpal Products: Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
11 Tammi 202445min

2 engineers discussing many things about Software Development (feat. Serkan Durusoy)
In this conversation, Serkan Durusoy, the head of engineering at Okra, discusses his background, living on a sailboat, and his passion for engineering. He emphasizes the importance of creativity and art in engineering and how personal background and upbringing can influence problem-solving approaches. Serkan also explores the balance between individual approaches and the need for alignment on outcomes in engineering teams. He highlights the role of personal relationships and the challenges of making tough decisions as a manager. In this conversation, Krish and Serkan discuss various topics related to software engineering and career development. They explore managing differences as a manager, the challenges of remote work, choosing a software engineering career, applying opinions to circumstances, and the impact of software engineering on the world. Takeaways Engineering is a creative and artistic field that allows for the creation of solutions and the impact on users. Personal background and upbringing can shape problem-solving approaches in engineering. Balancing individual approaches and aligning on outcomes is crucial for effective teamwork in engineering. Personal relationships and emotions should be considered, but the focus should be on the problem and desired outcome. As a manager, it is important to approach conflicts with the goal of amicably solving the problem and moving things forward. Remote work requires intentional communication and over-communication to compensate for the lack of social cues and body language. Choosing a software engineering career can be a shortcut to making a visible impact in the world, but it requires a lifelong commitment to learning and dealing with frustrations. Opinions and learnings should be applied to specific circumstances rather than taken as absolute truths. Software engineering has the potential to make a significant impact on the world and offers opportunities for personal and professional growth. Chapters 00:00 Introduction and Background 01:37 Living on a Sailboat and Engineering 03:25 Favorite Food and Cultural Differences 05:29 Engineering as Art and Creativity 09:42 The Influence of Upbringing on Engineering 11:14 Differences in Approaching Engineering Problems 16:36 The Role of Personal Background in Engineering 19:22 Balancing Individual Approaches in Engineering 25:14 Navigating Differences in Engineering Approaches 28:59 The Importance of Alignment on Outcomes 31:45 Balancing Personal Relationships and Engineering Decisions 36:43 Prioritizing the Problem and Outcome 40:12 Making Tough Decisions as a Manager 40:40 Managing Differences as a Manager 49:16 Challenges of Remote Work 59:13 Choosing a Software Engineering Career 01:01:07 Applying Opinions to Circumstances 01:03:21 Software Engineering as a Career Snowpal Products: Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
10 Tammi 20241h 7min

World Traveler Series (1/N): Learnings from Global Travels about gaps in App Localizations (feat. Prasanna Veeraswamy)
Prasanna Veeraswamy shares his unique experience of traveling to every country in the world and the personal growth and education he gained from his travels. He emphasizes the importance of living in a place to truly understand its culture and build relationships. Prasanna also discusses the impact of travel on food perceptions and the accessibility of travel and learning opportunities for everyone. In this conversation, Prasanna Veeraswamy shares his experiences and insights from traveling to different countries. He discusses the number of countries and cities he has visited, the unplanned nature of his travels, and the sacrifices he has made for his nomadic lifestyle. Prasanna also reflects on the impact of traveling on his appreciation of different cultures and the expansion of his food palette. He emphasizes the importance of exploring the world and understanding different markets for product development. The conversation concludes with a discussion on the evolution of personal identity through travel and the potential for future conversations on overlapping topics. Takeaways Traveling is a valuable form of education that provides personal growth and learning opportunities. Living in a place, even for a short period, allows for a deeper understanding of the culture and the ability to build meaningful relationships. Food perceptions can vary greatly between different countries and regions, and experiencing local cuisine is an important part of travel. Travel and learning opportunities are accessible to everyone, and it is possible to balance travel with professional life. Traveling to different countries allows for a deeper understanding and appreciation of different cultures. The number of countries and cities visited can vary based on personal circumstances and opportunities. Unplanned travel can lead to unexpected adventures and opportunities to explore new places. Traveling often requires sacrifices, such as stability and financial considerations. Experiencing different cuisines can expand one's food palette and provide a deeper connection to different cultures. Understanding local markets and cultures is crucial for effective product development. Traveling can contribute to personal growth and the evolution of one's identity. Future conversations can explore overlapping topics related to travel, culture, and product development. Chapters 00:00 Introduction to Prasanna Veeraswamy 03:06 The Unique Experience of Traveling to Every Country 06:02 The Education and Personal Growth of Travel 09:29 Balancing Travel and Professional Life 10:53 The Topic: Learnings from Global Travels about Gaps in App Localizations 12:14 Factors that Shape a Person 15:31 The Impact of Living in Different Countries 19:14 The Difference Between Traveling and Living in a Place 23:17 Understanding Culture and Building Relationships 26:55 The Influence of Travel on Food Perceptions 30:38 The Importance of Being on the Ground 36:53 The Accessibility of Travel and Learning Opportunities 38:41 Traveling to Different Countries 40:08 The Number of Countries and Cities Visited 41:05 Unplanned Traveling 43:41 Appreciating and Remembering the Travel Experiences 45:55 Sacrifices Made for Traveling 47:24 The Impact of Traveling on Appreciating Different Cultures 49:22 Completing the Loop of Traveling to All Countries 52:19 Expanding the Circle of Food Experiences 56:14 Favorite Food Item 57:20 Consistency in Food Preferences 58:24 Evolution of Personal Identity 01:02:27 Understanding Different Markets and Cultures 01:05:38 The Importance of Traveling for Product Development 01:08:34 Future Conversations on Overlapping Topics 01:12:07 Closing Thoughts on Traveling and Exploration Prasanna Veeraswamy's Links: LinkedIn Profile Book: One for the Road Snowpal Products: Backends as Services on AWS Marketplace Mobile Apps on App Store and Play Store Web App Education Platform for Learners and Course Creators
9 Tammi 20241h 15min

