Big Data Management User Guide

Back Next

Overview of Mapping Transformations in the Hadoop Environment

Due to the differences between native environment and Hadoop environment, only certain transformations are valid or are valid with restrictions in the Hadoop environment. Some functions, expressions, data types, and variable fields are not valid in the Hadoop environment.

Consider the following processing differences that can affect whether transformations and transformation behavior are valid or are valid with restrictions in the Hadoop environment:

Hadoop uses distributed processing and processes data on different nodes. Each node does not have access to the data that is being processed on other nodes. As a result, the Hadoop execution engine might not be able to determine the order in which the data originated.

Each of the run-time engines in the Hadoop environment can process mapping logic differently.

The following table lists transformations and support for different engines in a Hadoop environment:

Transformation	Supported Engines
Transformations not listed in this table are not supported in the Hadoop environment.
Address Validator	Blaze Spark* Hive
Aggregator	Blaze Spark Hive
Case Converter	Blaze Spark* Hive
Classifier	Blaze Spark* Hive
Comparison	Blaze Spark* Hive
Consolidation	Blaze Spark* Hive
Data Masking	Blaze Spark Hive
Data Processor	Blaze Hive
Decision	Blaze Spark* Hive
Expression	Blaze Spark Hive
Filter	Blaze Spark Hive
Java	Blaze Spark Hive
Joiner	Blaze Spark Hive
Key Generator	Blaze Spark* Hive
Labeler	Blaze Spark* Hive
Lookup	Blaze Spark Hive
Match	Blaze Spark* Hive
Merge	Blaze Spark* Hive
Normalizer	Blaze Spark Hive
Parser	Blaze Spark* Hive
Python	Spark
Rank	Blaze Spark Hive
Router	Blaze Spark Hive
Sequence Generator	Blaze Spark*
Sorter	Blaze Spark Hive
Standardizer	Blaze Spark* Hive
Union	Blaze Spark Hive
Update Strategy	Blaze Spark* Hive
Weighted Average	Blaze Spark* Hive
Not supported for Big Data Streaming on the Spark engine. For more information about Big Data Streaming transformations, see the Informatica Big Data Streaming User Guide* .

Rename Saved Search

Table of Contents

Big Data Management User Guide

Big Data Management User Guide

Overview of Mapping Transformations in the Hadoop Environment

Overview of Mapping Transformations in the Hadoop Environment