Hive Connector

Back Next

Rules and guidelines for Hadoop distributions

Consider the following rules and guidelines in mappings in advanced mode for Hadoop distributions:

When you configure a Target transformation to write to Hive on the Cloudera CDH 6.1 distribution, operations such as delete, update, upsert, and data driven are not applicable. You can use only the insert operation.

Mappings that run on the

advanced cluster

to access Hive on the Amazon EMR, Azure HDI, or Cloudera CDH 6.1 distributions cannot use the HDFS staging directory.

To run mappings on the

advanced cluster

to read from or write data to Hive on the Cloudera 6.1 distribution, you must get the

hive-exec-2.1.1-cdh6.1.0.jar

file from Cloudera.

For more information about the steps to run the script and use the Cloudera 6.1 distribution, see Configure Hive Connector to download the distribution-specific Hive libraries.

After you download the required Cloudera CDH 6.1 jars, add

hive-exec-2.1.1-cdh6.1.0.jar

to the same directory as the downloaded files.

For example, copy the jar to the following directory:

<Secure Agent installation directory>/apps/Data_Integration_Server/ext/deploy_to_main/distros/Parsers/ CDH_6.1/

Do not use the Amazon S3 staging directory for mappings that run on an

advanced cluster

to access Hive on the Azure HDI distribution. The test connection fails with the following error:

java.lang.reflect.InvocationTargetException

You must instead use Azure Data Lake Storage Gen2 as the staging location for Azure HDI.

Rename Saved Search

Table of Contents

Hive Connector

Hive Connector

Rules and guidelines for Hadoop distributions

Rules and guidelines for Hadoop distributions