User Guide

Back Next

Kafka Connector Configuration Properties

The sample cdcPublisherKafka.cfg file contains configuration properties related to the Apache Kafka target.

The following properties are in the sample cdcPublisherKafka.cfg configuration file:

Connector.queueType: The type of target messaging queue to which PowerExchange CDC Publisher streams change data. The only valid value is Kafka.
Connector.kafkaTopic: Required. The Kafka topic or topics to which PowerExchange CDC Publisher sends messages that contain extracted change data. Enter a specific Kafka topic name if you want to send all messages to a single topic that you created, or enter USE_TABLE_NAME to direct Kafka to generate a separate topic for each source table using the mapname_tablename portion of the full extraction map name as the topic name. The full extraction map name has the format schema.
mapname_tablename
.
This parameter is optional. By default, the auto.create.topics.enable parameter in the Kafka server.properties file is set to true. With this setting, the Kafka broker automatically generates the topic or topics if they do not exist the first time messages are sent to the topics. If you set the auto.create.topics.enable parameter to false, Kafka does not generate the topics. You must set this property to determine how the topics will be created.
No default value is provided.
Connector.kafkaProducerPropertiesFile: Required. The path and file name of the Kafka producer.properties file that PowerExchange CDC Publisher uses to communicate with Kafka. This file is typically in the
/Kafka_installation/config
directory.
No default value is provided.

The following additional properties can also be included in the cdcPublisherKafka.cfg file at your discretion:

Connector.kafkaCheckpointFileDirectory: A directory name for the instance subdirectory to which the CDC Publisher writes checkpoint files. The default value is "checkpoint." You can use this property to override the default directory name.
Connector.kafkaFileCheckpointFileName: Controls the name of the file to which the CDC Publisher writes checkpoints. This file is in the directory that is specified by the
Connector.kafkaCheckpointFileDirectory
property. Enter a specific file name or DEFAULT. If you enter DEFAULT, the CDC Publisher uses the default file name of "checkpoint." The default property value is DEFAULT.
Connector.kafkaProducerGuaranteeDelivery: Controls whether the CDC Publisher uses guaranteed delivery to write a checkpoint after each change operation is acknowledged as successfully received by Kafka. This delivery mode avoids message loss and duplication but slows apply processing. Set to property to false if you do not want to use guaranteed delivery. With the setting of false, if Kafka terminates while changes are in flight, the CDC Publisher might apply duplicate messages to the target messaging system after the change stream is restarted. The default value is true.