User Guide

1.3
- 1.2
- 1.1
- 1.0

Back Next

If You Use Confluent Schema Registry on a Kafka Target

If you use Confluent Schema Registry in a Kafka target environment, when PowerExchange CDC Publisher publishes its formatted messages, it allows Confluent Schema Registry to intercept the messages and validate them against the registered Avro schema.

To enable PowerExchange CDC Publisher communication with Confluent Schema Registry, perform the following tasks:

On the CDC Publisher system, configure the KAFKA_CLIENT_LIBS environment variable to point to a local directory that contains the Kafka client .jar file and the contents of the following Confluent serialization libraries:

/confluent-v.r.m/share/java/confluent-common
/confluent-v.r.m/share/java/kafka-serde-tools

Ensure that the Kafka producer.properties file contains the following entries for the Confluent implementation of the key and value serializers:

key.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer
value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer
schema.registry.url=http://host:port

The host and port variables represent the host name and port number on which Confluent Schema Registry listens.

In the CDC Publisher cdcPublisherKafka.cfg configuration file, ensure that the Connector.kafkaProducerPropertiesFile property points to the customized Kafka producer.properties file that contains the Confluent implementation of the key and value serializers.

In the CDC Publisher cdcPublisherAvro.cfg configuration file, ensure that the Formatter.avroEncodingType property is set to none
. This value allows the Avro records to be passed to Confluent Schema Registry without encoding, as required.

By default, when CDC Publisher sends the first message to a Kafka topic, Confluent Schema Registry checks whether the Avro schema for the message is already in its registry. If the schema is not registered, Confluent Schema Registry automatically adds the schema to its registry.

Alternatively, you can manually add Avro schema to the Confluent Schema Registry before you start publishing messages with CDC Publisher. To get the Avro schema definitions for your source tables, run the PwxCDCAdmin utility with the REPORT=FORMAT parameter. Then use the resulting schema definitions to register the schema in Confluent Schema Registry. In some situations, Confluent Schema Registry requires the entire schema to be entered as a single string delimited by double-quotation marks. In this case, you must use an escape character before the quotation marks. To do so, perform the following steps:

In the cdcPublisherAvro.cfg configuration file, set the Formatter.avroDisplaySchemaWithEscapedQuotes property to true.

Run the PwxCDCAdmin utility with the REPORT=FORMAT parameter to generate the schema definitions. The reported schema definitions then include the \
escape character before the double-quotation marks, for example, /"schema_string/"
.

PowerExchange CDC Publisher does not support any schema registries other than Confluent Schema Registry.

Rename Saved Search

Table of Contents

User Guide

User Guide

If You Use Confluent Schema Registry on a Kafka Target

If You Use Confluent Schema Registry on a Kafka Target