PowerExchange for Kafka User Guide for PowerCenter

PowerExchange for Kafka User Guide for PowerCenter

Configure Kafka Source Session Properties

Configure Kafka Source Session Properties

You can configure the session properties for a Kafka source on the
Mapping
tab. Define the properties for the source instance in the session.
The following table describes the session properties that you can configure for a Kafka source session:
Property
Description
Start position offset
The time from which the Kafka source starts reading Kafka messages from a Kafka topic.
You can select one of the following options:
  • Custom. Read messages from a specific time.
  • Earliest. Read the earliest messages available on the Kafka topic.
  • Latest. Read messages received by the Kafka topic after the mapping has been deployed.
Connector Mode
Specifies the mode to read data from the Kafka source.
You can select one of the following modes:
  • Batch. The PowerCenter Integration Service reads the messages available in a Kafka topic based on the specified position offset. After the PowerCenter Integration Service reads the data, the session terminates. When you configure a session for recovery, the PowerCenter Integration Service reads data from the last checkpoint and stops reading data till the offset when the session recovery started.
  • Streaming. The PowerCenter Integration Service reads the messages available in a Kafka topic in real time. The PowerCenter Integration Service does not terminate the session and continues to read messages from the Kafka topic. You must manually terminate the workflow.
Custom Start Position Timestamp
The time in GMT from which the Kafka source starts reading Kafka messages from a Kafka topic.
Specify a time in the following format:
yyyy-mm-dd HH24:mm:ss[.fffffffff]
The milliseconds are optional.
This property is applicable only when you select
Custom
as the Start position offset.
Consumer Configuration Properties
The configuration properties for the Kafka consumer.
If you read data from a Kafka cluster that is configured for Kerberos authentication, specify the following property to override the
Additional Connection Properties
specified in the Kafka connection:
security.protocol=SASL_PLAINTEXT,sasl.kerberos.service.name=kafka,sasl.mechanism=GSSAPI
For more information about the Kafka consumer configuration properties, see the Kafka documentation.
Recover from last state
If you select this option, the PowerCenter Integration Service recovers a stopped, aborted, or terminated session from the last checkpoint.
Default is selected.
Topic Pattern
Specify a regular expression pattern for the topic name that you want to read from. Use the regular expression syntax guidelines to specify the pattern.
For more information about how to use topic patterns to read data from a Kafka source, see the Knowledge Base article 565201.
Tracing Level
Amount of detail displayed in the session log for the transformation. You can choose Normal, Verbose Initialization, or Verbose Data.
Default is Normal.

0 COMMENTS

We’d like to hear from you!