User Guide

10.2.2 HotFix 1
- 10.5.7
- 10.5.6
- 10.5.5
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1

Back Next

Overview of Transformations

Due to the differences between native environments and non-native environments, only certain transformations are valid or are valid with restrictions in a non-native environment. Some functions, expressions, data types, and variable fields are not valid in a non-native environment.

Consider the following processing differences that can affect whether transformations and transformation behavior are valid or are valid with restrictions in a non-native environment:

Compute clusters use distributed processing and process data on different nodes. Each node does not have access to the data that is being processed on other nodes. As a result, the run-time engine might not be able to determine the order in which the data originated.

Each of the run-time engines in the non-native environment can process mapping logic differently.

Much of the processing behavior for batch mappings on the Spark engine also apply to streaming mappings.

The following table lists transformations and support for different engines in a non-native environment:

Transformation	Supported Engines
Transformations not listed in this table are not supported in a non-native environment.
Address Validator	Blaze Spark
Aggregator	Blaze Spark* Databricks Spark
Case Converter	Blaze Spark
Classifier	Blaze Spark
Comparison	Blaze Spark
Consolidation	Blaze Spark
Data Masking	Blaze Spark*
Data Processor	Blaze
Decision	Blaze Spark
Expression	Blaze Spark* Databricks Spark
Filter	Blaze Spark* Databricks Spark
Java	Blaze Spark*
Joiner	Blaze Spark* Databricks Spark
Key Generator	Blaze Spark
Labeler	Blaze Spark
Lookup	Blaze Spark* Databricks Spark
Match	Blaze Spark
Merge	Blaze Spark
Normalizer	Blaze Spark* Databricks Spark
Parser	Blaze Spark
Python	Spark*
Rank	Blaze Spark* Databricks Spark
Router	Blaze Spark* Databricks Spark
Sequence Generator	Blaze Spark
Sorter	Blaze Spark* Databricks Spark
Standardizer	Blaze Spark
Union	Blaze Spark* Databricks Spark
Update Strategy	Blaze Spark
Weighted Average	Blaze Spark
Window	Spark**
* Supported for both batch and streaming mappings. ** Supported for streaming mappings only. For more information, see the Big Data Streaming User Guide.

Rename Saved Search

Table of Contents

User Guide

User Guide

Overview of Transformations

Overview of Transformations