Apache Kafka is an open-source distributed data streaming platform that is scalable and fault tolerant and able to process large amounts of data quickly. It can serve as an interim staging area for data that will be consumed by various types of downstream consumer applications.
Producer applications, such as the PowerExchange CDC Publisher, can publish streams of change records as messages to Kafka topics. Kafka makes messages available with low latency to consumer applications.
A topic is a collection of messages to which consumer applications subscribe based on the topic name. A topic is divided into multiple partitions, each containing a set of messages in an unchangeable sequence. The use of partitions enables parallel processing of topic data by splitting the data across multiple Kafka broker servers. Partitions also allow consumer applications to read data from the same topic in parallel.
For information about Kafka use cases, configuration, and implementation, see the Apache Kafka documentation at
https://kafka.apache.org/.