Installation and Configuration Guide

Back Next

Step 2. Configure the Environment Variables

Before you set up Spark, you must configure the following environment variables on the machine that has

Relate 360

installed:

KAFKA_HOME: Directory in which Kafka is installed.; For example,
export KAFKA_HOME=/opt/cloudera/parcels/KAFKA/lib/kafka
HBASE_HOME: Directory in which HBase is installed.; For example,
export HBASE_HOME=/usr/hdp/<version>/hbase/
SPARK_KAFKA_VERSION: Version number of the Kafka consumer API that integrates Spark and Kafka. The supported version number is
0.10
.
SPARK_MAJOR_VERSION: Required only if you use Hortonworks HDP. Major version number of Spark. The supported version number is
2
.
KAFKA_OPTS: Required only if you use Kerberos for authentication. Absolute path to the
jaas.conf
and
krb5.conf
files. In the
jaas.conf
file, ensure that you configure the Kerberos principal name of the Kafka broker and the absolute path to the keytab file.; For example,
export KAFKA_OPTS="-Djava.security.auth.login.config=/data/r360/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf"; The directory that contains the
jaas.conf
and
krb5.conf
files must not be SELinux enabled. To remove the SELinux permissions from a directory, use the
setfattr
command.

After you configure the environment variables, ensure that the Kafka broker runs.