Hive Connector

Back Next

Step 1. Run the script on a Linux system

The Hive Connector package that contains the Informatica Hive third-party script and the Informatica Hive third-party property files is part of the Secure Agent installation. When you run the Hive third-party script, you can specify the distribution that you want to use.

Go to the following Secure Agent installation directory where the Informatica Hive third-party script is located:

<Secure Agent installation directory>/downloads/package-hiveadapter.<version>/package/hive/thirdparty/informatica.hiveadapter/scripts/

Copy the

scripts

folder outside the Secure Agent installation directory.

You can do this if the Secure Agent does not have internet access to download the third-party libraries or due to other network restrictions.

If you want to run the script in the same machine where the Secure Agent is installed, perform the following tasks:

From the terminal, run the following command from the

scripts

folder:

sh downloadHiveLibs.sh

When prompted, select

Data Integration

or CDI Advanced Mode for which you want to run the script.

Enter 1 to select CDI
.

Enter 2 to select CDI Advanced Mode
.

When prompted, specify the value of the Hadoop distribution that you want to use.

The third-party libraries are copied to the following directory based on the option you selected in step 3b:

For CDI
:

<Secure Agent installation directory>/apps/Data_Integration_Server/ext/deploy_to_main/distros/Parsers/<Hadoop distribution version>/lib

For CDI Advanced Mode
:

<Secure Agent installation directory>/ext/connectors/thirdparty/informaticallc.hiveadapter/spark/lib

<Secure Agent installation directory>/apps/Data_Integration_Server/ext/deploy_to_main/distros/Parsers/<Hadoop distribution version>/lib

where the value of the Hadoop distribution version is based on the Hadoop distribution you specified.

If you copy the

scripts

folder to a machine where the Secure Agent is not installed, perform the following tasks:

Perform steps 3a and 3b.

The third-party libraries are copied to the following directories based on the option you selected in step 3b:

For CDI
:

<CurrentDirectory>/deploy_to_main/distros/Parsers/<Hadoop distribution version>/lib

Manually copy the

deploy_to_main

directory to the following Secure Agent location:

<Secure Agent installation directory>/apps/Data_Integration_Server/ext

, or replace the directory if it is already present.

For CDI Advanced Mode
:

<CurrentDirectory>/informaticallc.hiveadapter/spark/lib

Manually perform the following tasks:

Copy the

informaticallc.hiveadapter

directory to the following Secure Agent location:

<Secure Agent installation directory>/ext/connectors/thirdparty/

Copy the

deploy_to_main

directory to the following Secure Agent location:

<Secure Agent installation directory>/apps/Data_Integration_Server/ext

, or replace the directory if it is already present.

where the value of the Hadoop distribution version is based on the Hadoop distribution you specified.

CDH_6.1 option is applicable for Cloudera CDH 6.1, Cloudera CDP 7.1 private cloud, and Cloudera CDW 7.2 public cloud in mappings. For mappings in advanced mode, CDH_6.1 is applicable only for Cloudera CDH 6.1. EMR_5.20 is applicable for EMR_6.1, EMR_6.2, and EMR_6.3 for Hive mappings in advanced mode, whereas EMR_5.20 is applicable only for Amazon EMR 5.20, EMR 6.3, and EMR 6.4 in mappings.

The Hadoop distribution directory created under

deploy_to_main/distros/Parsers/

changes based on the distribution you select:

If you select CDH_6.1, CDP_7.1, or CDW_7.2, the Hadoop distribution directory created is CDH_6.1.

If you select EMR_5.20, EMR_6.1, EMR_6.2, or EMR_6.3, the Hadoop distribution directory created is EMR_5.20.

If you select HDInsight_4.0, the Hadoop distribution directory created is HDInsight_4.0.

Rename Saved Search

Table of Contents

Hive Connector

Hive Connector

Step 1. Run the script on a Linux system

Step 1. Run the script on a Linux system