Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Kafka Broker Properties

Kafka Broker Properties

The following table describes the Kafka broker properties for the Kafka connection:
Property
Description
Kafka Broker List
Comma-separated list of Kafka brokers which maintain the configuration of the Kafka messaging broker.
To specify a Kafka broker, use the following format:
<IP Address>:<port>
Retry Timeout
Number of seconds after which the Integration Service attempts to reconnect to the Kafka broker to read or write data. If the source or target is not available for the time you specify, the mapping execution stops to avoid any data loss.
Kafka Broker Version
Configure the Kafka messaging broker version to Apache 0.10.1.1 & above.
Additional Connection Properties
Optional. Comma-separated list of connection properties to connect to the Kafka broker.
For example, you can use the following syntax:
request.timeout.ms=<value>,session.timeout.ms=<value>, fetch.max.wait.ms=<value>,heartbeat.interval.ms=<value>, security.protocol=SASL_PLAINTEXT,sasl.kerberos. service.name=<kerberos name>,sasl.mechanism=GSSAPI, sasl.jaas.config=com.sun.security.auth.module. Krb5Login Modulerequired useKeyTab=true doNotPrompt=true storeKey=true client=true keyTab="<Keytab Location>" principal="<principal>";
To reduce the time taken to connect to the Kafka broker, ensure that you set the following properties:
  • request.timeout.ms
  • session.timeout.ms
  • fetch.max.wait.ms
  • heartbeat.interval.ms
To connect to the Kafka broker in a secured way, ensure that you set one of the following values for the
security.protocol
property:
  • SASL_SSL
  • SSL
The default value of
security.protocol
property is SASL_PLAINTEXT.
The Additional Connection Properties is available for technical preview. Technical preview functionality is supported but is unwarranted and is not production-ready. Informatica recommends that you use in non-production environments only.
For more information about the connection properties, see https://kafka.apache.org/documentation/.