Developer Workflow Guide

10.4.0
- 10.5.3
- 10.5.2
- 10.5
- 10.4.1

Back Next

Inclusive Gateway Example

Add Inclusive gateways to a workflow to run multiple tasks in parallel. For example, you can configure an Inclusive gateway with multiple branches and add a Mapping task to each branch. When the workflow runs, the Data Integration Service concurrently runs the mappings that the tasks identify.

You might configure multiple Mapping tasks in parallel when you need to process very large volumes of data. The tasks might identify mappings that read data from different locations and write the data to a warehouse. Add the Mapping tasks to the branches that the Inclusive gateways define. The Data Integration Service treats each mapping as a separate job. If you configure the Data Integration Service to run on a grid, the Data Integration Service can assign the mapping jobs to different nodes on the grid. If you configure the mappings to run in a Hadoop environment, the Data Integration Service pushes the mappings to the Hadoop cluster.

The following image shows a workflow that contains multiple Mapping tasks between Inclusive gateways:

The workflow defines multiple branches. All but one of the branches contain a Mapping task and a Notification task. The Notification task reports that the Mapping task ran. The default sequence flow starts a branch that contains a Notification task. A broken line in the sequence flow to the Notification task indicates that it is the default sequence flow.

The workflow contains the following objects:

Start event. The Start event starts the workflow.

Inclusive gateways. The two gateways split the workflow into multiple branches and merge the branches into a single flow.

Mapping tasks. The workflow branches include a series of Mapping tasks. Each task identifies a mapping that reads a different data source. The mappings write to a common data target.

Notification tasks. Each branch includes a Notification task that sends an email to the data owners when the corresponding mapping runs.

The default sequence flow does not connect to a Mapping task. The sequence flow connects to a Notification task that notifies the workflow owner that the Data Integration Service reached the outgoing gateway.

Mapping task. The final mapping task includes an Expression transformation that verifies the number of records that the earlier mappings wrote to the target database tables.

Notification task. The final Notification task sends an email to the workflow developer to indicate that the workflow tasks are complete.

End event. The End event ends the workflow.

If a Mapping task includes a mapping that you configure to run in a Hadoop environment, do not assign the Mapping task outputs to workflow variables. Mappings that run in a Hadoop environment do not provide the total number of source, target, or error rows. When a Mapping task includes a mapping that runs in a Hadoop environment, the task outputs contain a value of zero (0).

Rename Saved Search

Table of Contents

Developer Workflow Guide

Developer Workflow Guide

Inclusive Gateway Example

Inclusive Gateway Example