Kafka: How to import/export CSV/TXT data
Updated: Jan 30, 2024
Overview Working with Apache Kafka, one of the most popular distributed streaming platforms, often requires the ability to import and export data in common file formats like CSV and TXT. This tutorial will walk you through the basics......
Kafka: How to read events from a topic (with examples)
Updated: Jan 30, 2024
Introduction Apache Kafka is a popular open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. It is designed to handle data feeds with high throughput and low latency.......
Kafka: How to add events to a topic with a timestamp
Updated: Jan 30, 2024
Introduction Apache Kafka has become the backbone of many modern data-driven applications because of its capability to handle high-throughput, resilient distributed streaming. Timestamps play a crucial role in Kafka messages, serving......
Kafka: Limit the memory used for buffering across all threads
Updated: Jan 30, 2024
Introduction Apache Kafka is a distributed streaming platform that is widely used for building real-time data pipelines and streaming applications. One of the key aspects of operating a Kafka cluster is managing the memory used for......
Kafka: How to limit the number of messages per partition
Updated: Jan 30, 2024
Introduction Apache Kafka is a distributed streaming platform enabling thousands of companies worldwide to process and analyze streams of data in real-time. One of Kafka’s key features is the ability to organize messages into......
ZooKeeper-less Kafka: Using Kraft and Raft Metadata Mode
Updated: Jan 30, 2024
Introduction Apache Kafka has been traditionally known to rely on ZooKeeper for distributed coordination and metadata storage. However, with the introduction of KIP-500, Kafka aims to replace ZooKeeper with a self-managed metadata......
How to Implement Complex Event Processing in Kafka
Updated: Jan 30, 2024
Introduction Apache Kafka is a powerful distributed streaming platform that allows you to process a large stream of data in real-time. Complex Event Processing (CEP) is a technique to process and analyze patterns of events in such data......
Getting started with KSQL for Kafka stream processing
Updated: Jan 30, 2024
Introduction Kafka Streams is a powerful tool for building real-time data processing pipelines and applications. With the rise of streaming data and the need to process it on the fly, Kafka and its stream processing capabilities have......
How to Design Kafka-Based Event-Driven Microservices (with Examples)
Updated: Jan 30, 2024
Understanding Kafka and Event-Driven Architecture Apache Kafka is an open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. Kafka aims to provide a unified,......
How to Configure Kafka for Exactly-Once Processing
Updated: Jan 30, 2024
Introduction Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation which is used to build real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, and......
How to Integrate Kafka with Databases
Updated: Jan 30, 2024
Introduction Apache Kafka has become a pivotal component in data pipelines and is heavily utilized for building real-time streaming data pipelines and applications. Kafka, being a distributed streaming platform, enables you to publish......
How to Set Up Source and Sink Connectors in Kafka
Updated: Jan 30, 2024
Introduction Apache Kafka has become the go-to ecosystem for real-time event streaming. But its true potential is unlocked when connected with diverse data systems for importing and exporting data seamlessly. This process is......