Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Connections
  4. Mappings in the Hadoop Environment
  5. Mapping Objects in the Hadoop Environment
  6. Monitoring Mappings in the Hadoop Environment
  7. Mappings in the Native Environment
  8. Profiles
  9. Native Environment Optimization
  10. Data Type Reference
  11. Function Reference
  12. Parameter Reference

Transformation Support on the Blaze Engine

Transformation Support on the Blaze Engine

Some restrictions and guidelines apply to processing transformations on the Blaze engine.
The following table describes rules and guidelines for processing transformations on the Blaze engine:
Transformation
Rules and Guidelines
Address Validator
The Address Validator transformation cannot generate a certification report.
Aggregator
Mapping validation fails in the following situations:
  • The transformation contains stateful variable ports.
  • The transformation contains unsupported functions in an expression.
Case Converter
Supported without restrictions.
Comparison
Supported without restrictions.
Data Processor
Mapping validation fails in the following situations.
  • The transformation
    Data processor mode
    is set to
    Input Mapping
    or
    Service and Input Mapping
    .
Decision
Supported without restrictions.
Expression
Mapping validation fails in the following situations:
  • The transformation contains stateful variable ports.
  • The transformation contains unsupported functions in an expression.
An Expression transformation with a user-defined function returns a null value for rows that have an exception error in the function.
Filter
Supported without restrictions.
Java
Supported without restrictions.
Joiner
Mapping validation fails in the following situations:
  • The transformation contains an inequality join.
Key Generator
Supported without restrictions.
Labeler
Supported without restrictions.
Lookup
Mapping validation fails in the following situations:
  • The cache is configured to be shared, named, persistent, dynamic, or uncached. The cache must be a static cache.
If you add a data object that uses Sqoop as a Lookup transformation in a mapping, the Data Integration Service does not run the mapping through Sqoop. It runs the mapping through JDBC.
Match
Mapping validation fails in the following situations:
  • The mapping specifies an identity match type.
A Match transformation generates cluster ID values differently in native and Hadoop environments. In a Hadoop environment, the transformation appends a group ID value to the cluster ID.
Merge
Supported without restrictions.
Normalizer
Supported without restrictions.
Parser
Supported without restrictions.
Rank
Supported without restrictions.
Router
Supported without restrictions.
Sequence Generator
Mapping validation fails in the following situations:
  • The reset property for the transformation is enabled.
If you want the Data Integration Service to reset the sequence data object to the start value for each mapping run, then you must run the mapping in the native environment.
Sorter
The Blaze engine can perform global sorts when the following conditions are true:
  • The Sorter transformation is connected directly to flat file targets.
  • The target is configured to maintain row order.
  • The sort key is not a binary data type.
If any of the conditions are not true, the Blaze engine performs a local sort.
Standardizer
Supported without restrictions.
Union
Supported without restrictions.
Update Strategy
Mapping validation fails in the following situations:
  • The Update Strategy transformation is connected to more than one target.
  • The Update Strategy transformation is not connected directly to a target.
  • The Update Strategy transformation target is an external Hive table.
  • The target is not a Hive target on the same cluster as the Update Strategy transformation.
  • The target does not contain a primary key.
  • The Hive target property to truncate the target table at runtime is enabled.
  • The Hive target property to create or replace the target table at runtime is enabled.
The mapping fails in the following situations:
  • The target is not ORC bucketed.
  • The target is ORC bucketed on all columns.
Compile validation errors occur and the mapping execution stops in the following situations:
  • The Hive version is earlier than 0.14.
  • The target table is not enabled for transactions.
To use a Hive target table with an Update Strategy transformation, you must create the Hive target table with the following clause in the Hive Data Definition Language:
TBLPROPERTIES ("transactional"="true")
.
To use an Update Strategy transformation with a Hive target, you must configure the following properties in the
hive-site.xml
file:
hive.support.concurrency true hive.enforce.bucketing true hive.exec.dynamic.partition.mode true hive.txn.manager org.apache.hadoop.hive.ql.lockmgr.DbTxnManager hive.compactor.initiator.on true hive.compactor.worker.threads 1
Use the following format to configure the properties in the
hive-site.xml
file:
<property> <name>property-name</name> <value>property-value</value> </property>
If the Update Strategy transformation receives multiple update rows for the same primary key value, the transformation selects one random row to update the target.
The Blaze engine executes transactions in the following order: deletes, updates, inserts. It does not process rows in the same order as the Update Strategy transformation receives them.
The Blaze engine performs Update as Update even if the transformation is configured to Update as Insert or Update else Insert.
Weighted Average
Supported without restrictions.
Transformations not listed in this table are not supported.


Updated November 09, 2018