Use the Hive Warehouse Connector and Hive LLAP for faster execution of Hive queries when you read and write to Hive tables with the Spark engine. You can use the Hive Warehouse Connector and Hive LLAP with Hortonworks HDP 3.x and Microsoft Azure HDInsight 4.x clusters on the Spark engine.

To enable the Hive Warehouse Connector and Hive LLAP, configure the following properties in the Spark advanced properties for the Hadoop connection:

infaspark.useHiveWarehouseAPI=true
spark.datasource.hive.warehouse.load.staging.dir=<HDFS temporary directory for writing to Hive tables>
spark.datasource.hive.warehouse.metastoreUri=<URI for the Hive metastore>
Use the value for
hive.metastore.uris
from the hive_site_xml cluster configuration properties.
spark.hadoop.hive.llap.daemon.service.hosts=<Application name for the LLAP service>
Use the value for
hive.llap.daemon.service.hosts
from the hive_site_xml cluster configuration properties.
spark.hadoop.hive.zookeeper.quorum=<Zookeeper hosts used by Hive LLAP>
Use the value for
hive.zookeeper.quorum
from the hive_site_xml cluster configuration properties.
spark.sql.hive.hiversever2.jdbc.url=<URL for HiveServer2 Interactive.
Use the value in Ambari for HiveServer2 JDBC URL.

For more information, refer to the

Data Engineering Integration 10.4.1 User Guide

.