Hadoop Integration Guide

10.2.1
- 10.5.5
- 10.5.4.1
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0
- 10.2.2 HotFix 1
- 10.2.2 Service Pack 1
- 10.2.2

Back Next

Update Streaming Objects

Big Data Streaming uses Spark Structured Streaming to process data instead of Spark Streaming. To support Spark Structured Streaming, some header ports are added to the data objects, and support to some of the data objects and transformations are deferred to a future release. The behavior of some of the data objects is also updated.

After you upgrade, the existing streaming mappings become invalid because of the unavailable header ports, the unsupported transformations or data objects, and the behavior change of some data objects.

To use an existing mapping, perform the following tasks:

Re-create the physical data objects. After you re-create the physical data objects, the data objects get the required header ports, such as timestamp, partitionID, or key based on the data object.

Re-create the Normalizer transformation. After you re-create the Normalizer transformation, you can change or specify the type configuration for the complex port.

Update the streaming mapping. If the mapping contains Kafka target, Aggregator transformation, Joiner transformation, or Normalizer transformation replace the data object or transformation, and then update the mapping because of the changed behavior of these transformations and data objects.

Verify the deferred data object types. If the streaming mapping contains unsupported transformations or data objects, contact Informatica Global Customer Support.

Complete Upgrade Tasks

Re-create the Physical Data Objects

Update the Streaming Mappings

Verify the Deferred Data Object Types

Complete Upgrade Tasks

Re-create the Physical Data Objects

Update the Streaming Mappings

Verify the Deferred Data Object Types

Download Guide

Watch

Comments

Communities