Integration Guide

10.4.0
- 10.5.5
- 10.5.4.1
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.2.2 HotFix 1
- 10.2.2 Service Pack 1
- 10.2.2
- 10.2.1

Back Next

Update Streaming Objects

Data Engineering Streaming uses Spark Structured Streaming to process data instead of Spark Streaming. To support Spark Structured Streaming, some header ports are added to the data objects, and support to some of the data objects and transformations are deferred to a future release. The behavior of some of the data objects is also updated.

After you upgrade, the existing streaming mappings become invalid because of the unavailable header ports, the unsupported transformations or data objects, and the behavior change of some data objects.

Perform this task in the following situations:
You upgraded from version 10.1.1, 10.2.0, or 10.2.1.

To use an existing streaming mapping, perform the following tasks:

Re-create the physical data objects. After you re-create the physical data objects, the data objects get the required header ports, such as timestamp, partitionID, or key based on the data object.

In a Normalizer transformation, if the

Occurs

column is set to Auto, re-create the Normalizer transformation. You must re-create the Normalizer transformation because the type configuration property of the complex port refers to the physical data object that you plan to replace.

Update the streaming mapping. If the mapping contains Kafka target, Aggregator transformation, Joiner transformation, or Normalizer transformation, replace the data object or transformation, and then update the mapping because of the changed behavior of these transformations and data objects.

Verify the deferred data object types. If the streaming mapping contains unsupported transformations or data objects, contact Informatica Global Customer Support.

Complete Upgrade Tasks

Re-create the Physical Data Objects

Re-create the Normalizer Transformation

Update the Streaming Mappings

Verify the Deferred Data Object Types

Complete Upgrade Tasks

Re-create the Physical Data Objects

Update the Streaming Mappings

Verify the Deferred Data Object Types

Complete Upgrade Tasks

Re-create the Physical Data Objects

Update the Streaming Mappings