PowerExchange for Kafka User Guide for PowerCenter

10.5.5
- 10.5.6
- 10.5.4
- 10.5.3
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0

Back Next

Import Kafka Source and Target Definitions

Use the

Create PowerExchange for Kafka Source

Create PowerExchange for Kafka Target

wizard to import Kafka source and target definitions into the PowerCenter repository.

You must import Kafka source and target objects before you create a mapping.

Start PowerCenter Designer, and connect to a PowerCenter repository configured with a Kafka instance.

Open a source or target folder.

Select

Source Analyzer

Target Designer

In the Source Analyzer, click

Sources

Create PowerExchange for Kafka Source

as shown in the following image:

The

Kafka Connection

wizard appears.

In the Target Analyzer, click

Targets

Create PowerExchange for Kafka Target

as shown in the following image:

The

Kafka Connection

dialog box appears.

Configure the following connection parameters:

Connection Parameter	Description
Kafka Broker List	The IP address and port combinations of the Kafka messaging system broker list. The IP address and port combination has the following format: <IP Address>:<port> You can enter multiple comma-separated IP address and port combinations.
Retry Timeout in seconds	Number of seconds the Integration Service attempts to reconnect to the Kafka broker to write data. If the source or target is not available for the time you specify, the mapping execution stops to avoid any data loss. Default is 180 seconds.
Kafka Broker Version	Select Apache 0.10.1.1 & above as the Kafka messaging broker version.
Additional Connection Properties	Comma-separated list of connection properties to connect to the Kafka broker. To connect to a Kafka broker over SSL, ensure that you set the following properties: security.protocol ssl.truststore.location ssl.truststore.password If the ssl.client.auth parameter is set to required or requested , ensure that you set the following properties: ssl.keystore.location ssl.keystore.password ssl.key.password For example, security.protocol=SSL,ssl.truststore.location=<complete path and name of the client.truststore.jks file>,ssl.truststore.password=<client truststore password>,ssl.keystore.location=<complete path and name of the client.keystore.jks file>,ssl.keystore.password=<client keystore password>,ssl.key.password=<password for the individual key> You can also specify the configuration properties required to connect to a Kerberised Kafka cluster. Specify the following property: security.protocol=SASL_PLAINTEXT,sasl.kerberos.service.name=kafka,sasl.mechanism=GSSAPI
Confluent Schema Registry URL	Address that points to your Schema Registry instance to manage schemas within the Confluent platform. For example, https://<host name>:8081 .
SSL Mode	Specifies whether the PowerCenter Integration Service establishes a secure connection to the Kafka broker. You can select one of the following options: disabled. The PowerCenter Integration Service establishes an unencrypted connection to the Kafka broker. One-Way. The PowerCenter Integration Service establishes an encrypted connection to the Kafka broker using truststore file and truststore password. Two-Way. The PowerCenter Integration Service establishes an encrypted connection to the Kafka broker using truststore file and truststore password.
SSL TrustStore File Path	Applicable only if you select One-Way or Two-Way as the SSL mode. The complete path and file name of the truststore file. The truststore file contains the SSL certificate that the Kafka cluster validates against the Kafka broker certificate.
SSL TrustStore Password	Applicable only if you select One-Way or Two-Way as the SSL mode. The password for the truststore file.
SSL KeyStore File Path	Applicable only if you select Two-Way as the SSL mode. The complete path and name of the Java keystore file. The keystore file contains the certificate that the Kafka broker validates against the Kafka cluster certificate.
SSL KeyStore Password	Applicable only if you select Two-Way as the SSL mode. The password for the keystore file.
Additional Security Properties	Not applicable for PowerExchange for Kafka.
SSL Mode Schema Registry	Applicable for schema registry. Specifies whether the PowerCenter Integration Service establishes a secure connection to the Kafka broker. You can select one of the following options: disabled. The PowerCenter Integration Service establishes an unencrypted connection to the Kafka broker. One-Way. The PowerCenter Integration Service establishes an encrypted connection to the Kafka broker using truststore file and truststore password. Two-Way. The PowerCenter Integration Service establishes an encrypted connection to the Kafka broker using truststore file and truststore password.
SSL TrustStore File Path Schema Registry	Applicable only if you select One-Way or Two-Way as the SSL mode for schema registry. The complete path and file name of the truststore file. The truststore file contains the SSL certificate that the Kafka cluster validates against the Kafka broker certificate.
SSL TrustStore Password Schema Registry	Applicable only if you select One-Way or Two-Way as the SSL mode for schema registry. The password for the truststore file.
SSL KeyStore File Path Schema Registry	Applicable only if you select Two-Way as the SSL mode for schema registry. The complete path and name of the Java keystore file. The keystore file contains the certificate that the Kafka broker validates against the Kafka cluster certificate.
SSL KeyStore Password Schema Registry	Applicable only if you select Two-Way as the SSL mode for schema registry. The password for the keystore file.
Additional Security Properties Schema Registry	Applicable for schema registry. For Avro schema registry, enter the following security property: basic.auth.credentials.source=USER_INFO, basic.auth.user.info=user:password

Click

Test

to test the connection.

Click

Finish

to add the connection.

The

Select Objects from

tab appears.

Select the

Topics

package in

Package Explorer

A list of topic appears as shown in the following image:

Select the topic that you want to import, and then click

You cannot select topics from the

Topics Pattern

package.

The

Select Format Type for

dialog box appears.

The following image shows the

Select Format Type for

dialog box:

The image shows the Package Explorer tab where a list of table appears after you select the dataset.

In the

Format Type

list, select

Json

Avro

, or

Binary

to import data from Kafka topics in JSON format or Avro format, or Binary format.

You cannot import a PowerExchange for Kafka source or target definition in Flat or XML format.

By default, the system sets

Binary

as the format type. The

Avro

file type supports binary encoding.

If you select

Json

Avro

, or

Binary

as the

Format Type

, you must configure the following format properties:

Property	Description
Schema Source	Specifies the mode to import schema for the Kafka topic. Select Import Schema from File to import schema from a schema definition file in your local machine. You cannot import a schema for a Kafka topic using the Read Data from File option.
Schema	When you select the Import Schema from File option in the Schema Source drop-down list, you must choose a schema definition file on your local machine.

Optionally, click

Data Preview

to preview data.

To view the topic metadata, select the topic, and double-click the topic name.

You cannot select multiple topics in a Kafka source or target definition.

Click

Import

If you click

Import

in the

Select Objects from

dialog box, PowerExchange for Kafka imports the data from the Kafka topic in binary format.

Rename Saved Search

Table of Contents

PowerExchange for Kafka User Guide for PowerCenter

PowerExchange for Kafka User Guide for PowerCenter

Import Kafka Source and Target Definitions

Import Kafka Source and Target Definitions