Mappings

Back Next

Data flow run order

You can specify the order in which

Data Integration

runs the individual data flows in a mapping. Specify the flow run order when you want

Data Integration

to load the targets in the mapping in a particular order. For example, you might want to specify the flow run order when inserting, deleting, or updating tables with primary or foreign key constraints.

You might want to specify the flow run order to maintain referential integrity when updating tables that have primary or foreign key constraints. Or, you might want to specify the flow run order when you are processing staged data.

If a flow contains multiple targets, you cannot configure the load order of the targets within the flow.

The following image shows a mapping with two data flows:

In this example, the top flow contains two pipelines and the bottom flow contains one pipeline. A pipeline is a source and all the transformations and targets that receive data from that source. When you configure the flow run order, you cannot configure the run order of the pipelines within a data flow.

The following image shows the flow run order for the mapping:

The image shows a table with the sources and targets for two data flows. Source3 and Target3 make up one flow. Source1 and Target1, and Source2 and Target2 make up the second data flow. The arrows at the top right of the able allow you to move a flow up or down.

In this example,

Data Integration

runs the top flow first, and loads Target3 before running the second flow. When

Data Integration

runs the second flow, it loads Target1 and Target2 concurrently.

If you add another data flow to the mapping after you configure the flow run order, the new flow is added to the end of the flow run order by default.

If the mapping contains a mapplet,

Data Integration

uses the data flows in the last version of the mapplet that was synchronized. If you synchronize a mapplet and the new version adds a data flow to the mapping, the new flow is added to the end of the flow run order by default. You cannot specify the flow run order in mapplets.

You can specify the flow run order for data flows with any target type. You can define the data flow run order during SQL ELT optimization in a mapping that uses the following connections:

Amazon Redshift V2

Databricks Delta

Microsoft Azure Synapse SQL

ODBC

Snowflake Data Cloud

You can also specify the run order of data flows in separate mapping tasks with taskflows. Configure the taskflow to run the tasks in a specific order. For more information about taskflows, see

Taskflows

You can't specify the data flow run order in mappings in advanced mode or SQL ELT mode because these mappings can't have multiple flows.

Mapping configuration

Configuring the data flow run order

Download Guide

Watch

Comments

Cloud Data Integration Homepage