Big Data Management User Guide

10.1.1
- 10.4.1
- 10.4.0
- 10.2.2 HotFix 1
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1
- 10.2 HotFix 1
- 10.2
- 10.1.1 Update 2
- 10.1.1 HotFix 1
- 10.1
- 10.0

Back Next

Transformations in a Hadoop Environment

Due to the differences between native environment and Hadoop environment only certain transformations are valid or valid with restrictions in the Hadoop environment. The Data Integration Service does not process transformations that contain functions, expressions, data types, and variable fields that are not valid in a Hadoop environment.

The following table lists transformations and levels of support for different engines in a Hadoop environment:

Transformation	Blaze Engine	Spark Engine	Hive Engine
Address Validator	Supported with restrictions	Not supported	Supported with restrictions
Aggregator	Supported with restrictions	Supported with restrictions	Supported with restrictions
Case Converter	Supported	Not supported	Supported
Comparison	Supported	Not supported	Supported
Consolidation	Supported with restrictions	Not supported	Supported with restrictions
Data Masking	Supported with restrictions	Not supported	Supported with restrictions
Data Processor	Supported with restrictions	Not supported	Supported with restrictions
Decision	Supported	Not supported	Supported
Expression	Supported with restrictions	Supported with restrictions	Supported with restrictions
Filter	Supported	Supported	Supported
Java	Supported	Supported with restrictions	Supported with restrictions
Joiner	Supported with restrictions	Supported with restrictions	Supported with restrictions
Key Generator	Supported	Not supported	Not supported
Labeler	Supported	Not supported	Supported
Lookup	Supported with restrictions	Supported with restrictions	Supported with restrictions
Match	Supported with restrictions	Not supported	Supported with restrictions
Merge	Supported	Not supported	Supported
Normalizer	Supported	Not supported	Supported
Parser	Supported	Not supported	Supported
Rank	Supported	Not supported	Supported with restrictions
Router	Supported	Supported	Supported
Sequence Generator	Supported with restrictions	Not supported	Not supported
Sorter	Supported	Supported with restrictions	Supported with restrictions
SQL	Not supported	Not supported	Supported with restrictions
Standardizer	Supported	Not supported	Supported
Union	Supported	Supported	Supported
Update Strategy	Supported with restrictions	Not supported	Supported with restrictions
Weighted Average	Supported	Not supported	Supported