User Guide

1.3
- 1.2
- 1.1
- 1.0

Back Next

Preparing the Apache Kafka Environment for Use with CDC Publisher

You must perform a few tasks on the PowerExchange CDC Publisher system and on the Kafka target system to prepare for using Apache Kafka with PowerExchange CDC Publisher.

Download the Kafka client libraries to the PowerExchange CDC Publisher system.

The version of the Kafka client libraries must match the version of the Kafka target. The Kafka client libraries do not need to be copied under the PowerExchange CDC Publisher root installation directory.

On the PowerExchange CDC Publisher system, define the KAFKA_CLIENT_LIBS environment variable.

This environment variable points to the libs
subdirectory of the local Kafka client installation. The libs directory contains Kafka .jar files.

Ensure that the CDC Publisher can access the Kafka producer.properties file. The file can be on the local CDC Publisher host or a shared device.

This file contains connectivity information that CDC Publisher Connector will use to communicate with the Kafka target.

You can optionally rename the file name. When you configure the PowerExchange CDC Publisher configuration files later, you will set the Connector.kafkaProducerPropertiesFile property to point to this producer.properties file.

On the Kafka target system, if Kafka is not configured to automatically generate Kafka topics, create the Kafka topic or topics to which the PowerExchange CDC Publisher will publish the messages that contain change data. Ensure that the topics are created before you run the PowerExchange CDC Publisher process.

You can create a single topic for all source tables or create a separate topic for each source table. When you configure the PowerExchange CDC Publisher configuration files later, set the Connector.kafkaTopic property to specify either the name of single topic to which all messages will be published or the option USE_TABLE_NAME to publish messages to a separate topic for each source table.

By default, the auto.create.topics.enable parameter in the Kafka server.properties file is set to true. With this setting, the Kafka broker automatically generates the topic or topics if they do not exist the first time messages are sent to the topics. You need to manually create the topics only if the auto.create.topics.enable parameter has been set to false.

For more information about configuring a Apache Kafka system, see the Apache Kafka documentation.

Rename Saved Search

Table of Contents

User Guide

User Guide

Preparing the Apache Kafka Environment for Use with CDC Publisher

Preparing the Apache Kafka Environment for Use with CDC Publisher