Integration Guide

10.4.1
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.0
- 10.2.2 HotFix 1
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1

Back Next

Configure the Files to Use S3

To run mappings using S3 resources, you must configure the files from the master node to the Data Integration Service machine.

Perform this task in the following situations:
You are integrating for the first time. You upgraded from any Informatica version and changed the distribution version.

Perform the following steps to configure master node files to integrate with EMR:

Copy the hadoop-assemby .JAR file.

Create an AWS configuration file on the Data Integration Service machine.

Create and configure an environment variable.

Copy and replace athe hadoop-common .JAR file.

Recycle the Data Integration Service.

Perform these steps regardless of the version of EMR you are integrating with, but note that the version number parts of filen ames may vary depending on the version.

Example

This example contains file names that support integration with EMR 5.20.

Copy the .jar file: To integrate with EMR 5.20, get
emrfs-hadoop-assembly-2.29.0.jar
from the Hadoop administrator. Copy
the file to the following locations on each Data Integration Service machine:
/<Informatica installation directory>/services/shared/hadoop/EMR_5.16/lib; Required when you run mapping on the Spark engine.
Create a file: Create a
~/.aws/config
on the Data Integration Service machine. The file must contain the AWS location.; For example,
[default] region=us-west-2; Required when you run mapping on the Spark engine.
Create an environment variable: Create the
AWS_CONFIG_FILE
environment variable on the Data Integration Service machine. Set the value to
<EMR_5.16>/conf/aws.default; Required when you run mapping on the Spark and Blaze engines.
Copy and replace a file: Copy hadoop-common-2.8.5-amzn-1.jar from the following location in the EMR 5.20 cluster:; /usr/lib/hadoop; Replace the file in the following location:; <Informatica installation directory>/services/shared/hadoop/EMR_5.16/lib; Required when you run mapping on the Spark engine.
Recycle the Data Integration Service: You must recycle the Data Integration Service to reflect the changes.

Amazon EMR Integration Tasks

Rename Saved Search

Table of Contents

Integration Guide

Integration Guide

Configure the Files to Use S3

Configure the Files to Use S3

Example