Table of Contents

Search

  1. Preface
  2. Introduction to Intelligent Streaming
  3. Intelligent Streaming Configuration
  4. Connections
  5. Sources and Targets in a Streaming Mapping
  6. Intelligent Streaming Mappings
  7. Window Transformation
  8. Data Type Reference

Prerequisites to Read From or Write to a Kerberised Kafka Cluster

Prerequisites to Read From or Write to a Kerberised Kafka Cluster

Kafka uses the Java Authentication and Authorization Service (JAAS) for SASL configuration. You can use Kafka data objects to read from or write to a Kerberised Kafka cluster.
Before you read from or write to a Kerberized Kafka cluster, perform the following steps:
  1. Ensure that you have JAAS configuration file and the krb5.conf file for the Kerberised Kafka server.
    For information about creating JAAS configuration and configuring Keytab for Kafka clients, see the Apache Kafka documentation at https://kafka.apache.org/0101/documentation/#security
    For example, your JAAS configuration file can contain the following lines of configuration:
    //Kafka Client Authentication. Used for client to kafka broker connection KafkaClient { com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true useKeyTab=true storeKey=true keyTab="<path to keytab file>/krb5.keytab" principal="cloudqa@INFORMATICA.COM" client=true };
  2. Place the JAAS config file and keytab file in the same location in all the nodes of the Hadoop cluster.
    Informatica recommends that you place the files in the
    /etc
    location.
  3. Configure the default realm and KDC. If the default
    /etc/krb5.conf
    file is not configured or you want to change the configuration, add the following lines to the
    /etc/krb5.conf
    file:
    [libdefaults] default_realm = <REALM NAME> dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true [realms] <REALM NAME> = { kdc = <Location where KDC is installed> admin_server = <Location where KDC is installed> } [domain_realm] .<domain name or hostname> = <KERBEROS DOMAIN NAME> <domain name or hostname> = <KERBEROS DOMAIN NAME>
  4. Update the extra java options of the executor and the driver in the
    hadoopenv.properties
    file on the Data Integration Service machine with the path of the JAAS configuration and krb5 configuration files. The
    hadoopEnv.properties
    file is located in the following path:
    <InformaticaInstallationDir>/services/shared/hadoop/<Hadoop_distribution_name>/infaConf
    For example, you can include the following lines of code in the
    hadoopenv.properties
    file:
    infaspark.executor.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom -XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.krb5.conf=/<path to krb5.conf file>/krb5.conf -Djava.security.auth.login.config=/<path to jAAS config>/kafka_client_jaas.config infaspark.driver.cluster.mode.extraJavaOptions=-Djava.security.egd=file:/dev/./urandom -XX:MaxMetaspaceSize=256M -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.krb5.conf=/<path to keytab file>/krb5.conf -Djava.security.auth.login.config=<path to jaas config>/kafka_client_jaas.config
  5. Configure the data object read or write operation.
    For example, add the
    security.protocol=SASL_PLAINTEXT,sasl.kerberos.service.name=kafka,sasl.mechanism=GSSAPI
    to the following properties:
    • Consumer Configuration Properties
      . Configure this property in the advanced properties of the data object read operation.
    • Producer Configuration Properties
      . Configure this property in the advanced properties of the data object write operation.