Big Data Streaming User Guide

10.2
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1
- 10.2 HotFix 2
- 10.2 HotFix 1

Back Next

Prerequisites to Read From or Write to a Kerberised Kafka Cluster

Kafka uses the Java Authentication and Authorization Service (JAAS) for SASL configuration. You can use Kafka data objects to read from or write to a Kerberised Kafka cluster.

Before you read from or write to a Kerberized Kafka cluster, perform the following steps:

Ensure that you have JAAS configuration file and the krb5.conf file for the Kerberised Kafka server.

For information about creating JAAS configuration and configuring Keytab for Kafka clients, see the Apache Kafka documentation at https://kafka.apache.org/0101/documentation/#security

For example, your JAAS configuration file can contain the following lines of configuration:

//Kafka Client Authentication. Used for client to kafka broker connection
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useKeyTab=true
storeKey=true
keyTab="<path to keytab  file>/krb5.keytab"
principal="cloudqa@INFORMATICA.COM"
client=true
};

Place the JAAS config file and keytab file in the same location in all the nodes of the Hadoop cluster.

Informatica recommends that you place the files in the

/etc

location.

Configure the default realm and KDC. If the default

/etc/krb5.conf

file is not configured or you want to change the configuration, add the following lines to the

/etc/krb5.conf

file:

[libdefaults]
default_realm = <REALM NAME>
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true

[realms]
<REALM NAME> = {
kdc = <Location where KDC is installed> 
admin_server = <Location where KDC is installed>
                                }
 [domain_realm]
.<domain name or hostname> = <KERBEROS DOMAIN NAME>
<domain name or hostname> = <KERBEROS DOMAIN NAME>

Update the extra java options of the executor and the driver in the

hadoopenv.properties

file on the Data Integration Service machine with the path of the JAAS configuration and krb5 configuration files. The

hadoopEnv.properties

file is located in the following path:

<InformaticaInstallationDir>/services/shared/hadoop/<Hadoop_distribution_name>/infaConf

For example, you can include the following lines of code in the

hadoopenv.properties

file:

infaspark.executor.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom 
-XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true 
-Djava.security.krb5.conf=/<path to krb5.conf file>/krb5.conf 
-Djava.security.auth.login.config=/<path to jAAS config>/kafka_client_jaas.config 
                                
infaspark.driver.cluster.mode.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom 
-XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true 
-Djava.security.krb5.conf=/<path to keytab file>/krb5.conf 
-Djava.security.auth.login.config=<path to jaas config>/kafka_client_jaas.config

Configure the data object read or write operation.

For example, add the

security.protocol=SASL_PLAINTEXT,sasl.kerberos.service.name=kafka,sasl.mechanism=GSSAPI

to the following properties:

Consumer Configuration Properties

. Configure this property in the advanced properties of the data object read operation.

Producer Configuration Properties

. Configure this property in the advanced properties of the data object write operation.

Intelligent Streaming Configuration