Use a Static JAAS Configuration File

Ensure that you have JAAS configuration file.

For information about creating JAAS configuration and configuring Keytab for Kafka clients, see the Apache Kafka documentation at https://kafka.apache.org/0101/documentation/#security

For example, the JAAS configuration file can contain the following lines of configuration:

//Kafka Client Authentication. Used for client to kafka broker connection
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
doNotPrompt=true
useKeyTab=true
storeKey=true
keyTab="<path to keytab  file>/<keytab file name>"
principal="<principal name>"
client=true
};

Place the JAAS config file and keytab file in the same location on all the nodes of the Hadoop cluster.

Put the files in a location that is accessible to all nodes on the cluster, such as

/etc

or

/temp

.

On the

Spark Engine

tab of the Hadoop connection properties, update the

extraJavaOptions

property of the executor and the driver in the

Advanced Properties

property. Click

Edit

and update the properties in the following format:

spark.executor.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom 
-XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true 
-Djava.security.krb5.conf=/<path to krb5.conf file>/krb5.conf 
-Djava.security.auth.login.config=/<path to jAAS config>/<kafka_client_jaas>.config 
                                
spark.driver.cluster.mode.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom 
-XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true 
-Djava.security.krb5.conf=/<path to krb5.conf file>/krb5.conf 
-Djava.security.auth.login.config=<path to jaas config>/<kafka_client_jaas>.config

Configure the following properties in the data object read or write operation:

Data object read operation. Configure the
Consumer Configuration Properties
property in the advanced properties.
Data object write operation. Configure the
Producer Configuration Properties
property in the advanced properties.

Specify the following value:

security.protocol=SASL_PLAINTEXT,sasl.kerberos.service.name=kafka,sasl.mechanism=GSSAPI

Embed the JAAS Configuration