How to Use Topic Patterns in Kafka Data Objects

How to Use Topic Patterns in Kafka Data Objects

Overview

Overview

You can read from and write to Kafka in streaming mappings that run on the Spark engine. Create a Kafka data object to read from or write to Kafka topics.
Kafka runs as a cluster comprised of one or more servers. Each server is called a broker. Kafka brokers stream data in the form of messages. These messages are published to a topic.
To read topic data from Kafka with Intelligent Streaming, you create a Kafka data object in Informatica Developer (the Developer tool). When you configure the Kafka data object, specify the topic name that you read from or write to. You can specify the topic name or use a regular expression for the topic name pattern only when you read from Kafka.
To subscribe to multiple topics that match a pattern, you can specify a regular expression. When you run the application on the cluster, the pattern matching is done against topics before the application runs. If you add a topic with a similar pattern when the application is already running, the application will not read from the topic.

0 COMMENTS

We’d like to hear from you!