User Guide

1.3
- 1.2
- 1.1
- 1.0

Back Next

Preparing the MapR Environment for Use with CDC Publisher

On the PowerExchange CDC Publisher system, you must perform a few tasks to configure the MapR target for use with CDC Publisher.

Download the MapR client libraries.

The version of the MapR client libraries must match the version of the MapR target.

Define the MAPR_HOME environment variable, if it does not already exist.

This environment variable points to the MapR client installation on the local machine. For example:

C:/opt/mapr

Download the MapR kafka-client .jar file to the PowerExchange CDC Publisher system, if the file is not present in the local MapR lib subdirectory.

The MapR client installation does not include the kafka-clients-version-mapr-nnnn.jar file. If the CDC Publisher runs on a MapR client machine, you must copy the kafka-clients-version-mapr-nnnn.jar file from the MapR server installation to the local $MAPR_HOME/lib directory. If the CDC Publisher runs on the MapR server, the MapR lib directory includes the MapR Kafka client libraries, which makes this step unnecessary.

For MapR release 6.0.1, the Kafka client library jar file name is kafka-clients-1.0.1-mapr-1803.jar. For MapR release 6.1.0, the jar file names is kafka-clients-1.1.1-mapr-1808.jar.

Define the KAFKA_CLIENT_LIBS environment variable to point to the lib
subdirectory of the local MapR client installation that contains the MapR client and MapR Kafka client .jar files.

For example:

KAFKA_CLIENT_LIBS =$MAPR_HOME/lib

Ensure that the CDC Publisher can access the Kafka-formatted producer.properties file.

This file can be on the local CDC Publisher host or on a shared device. The file contains information that the CDC Publisher Connector uses with the Kafka API to communicate with the MapR target. The file must contain the following minimum entries:

value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer

For more information about options supported in the MapR implementation of the Kafka producer properties file, see https://maprdocs.mapr.com/52/MapR_Streams/configuration_parameters_for_producers.html.

Create or identify the MapR stream that includes the topics to which the CDC Publisher will publish messages.

The CDC Publisher supports publishing messages to topics within a single MapR stream. You must specify the stream name in the Connector.kafkaMaprStreamName property in the CDC Publisher cdcPublisherKafka.cfg file.

On the MapR target system, if MapR is not configured to automatically generate topics, create the MapR topic or topics to which the PowerExchange CDC Publisher will publish messages. Complete this step before you run the PowerExchange CDC Publisher process.

You can create a single topic for all source tables or create a separate topic for each source table. When you configure the PowerExchange CDC Publisher configuration files later, set the Connector.kafkaTopic property to specify either the name of the single topic or the option USE_TABLE_NAME to use a separate topic for each source table.

By default, a topic is created automatically when a producer creates the first message for it and the topic does not already exist. If you disabled automatic creation of topics by setting the autocreate parameter to false when creating the stream, you must manually create the topics.

For more information about configuring a MapR system, see the MapR documentation.

Rename Saved Search

Table of Contents

User Guide

User Guide

Preparing the MapR Environment for Use with CDC Publisher

Preparing the MapR Environment for Use with CDC Publisher