Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Amazon Redshift connectors
  3. Part 2: Data Integration with Amazon Redshift V2 Connector
  4. Part 3: Data Integration with Amazon Redshift Connector

Amazon Redshift Connectors

Amazon Redshift Connectors

Determine the order of processing for multiple targets

Determine the order of processing for multiple targets

When you configure a mapping to write to multiple targets in a single pipeline, with each target configured for any write operation, the target operations are processed in a predefined order of delete, update, upsert, and insert.
To process the target operations in the predefined order, you need to set certain properties in the Secure Agent and in the task properties.

Set -DEnableSingleCommit=true in the Secure Agent properties

Perform the following tasks to set the property for the Secure Agent:
  1. Open Administrator and select
    Runtime Environments
    .
  2. Select the Secure Agent for which you want to set the property.
  3. On the upper-right corner of the page, click
    Edit
    .
  4. In the
    System Configuration Details
    section, select the
    Type
    as
    DTM
    for the Data Integration Service.
  5. Edit the JVM options and set the property to
    -DEnableSingleCommit=true
    .

Set the EnableSingleCommit property in the task properties

Perform the following tasks to set the property in the task:
  1. On the
    Runtime Options
    tab in the mapping task properties, navigate to the Advanced Session Properties section.
  2. From the
    Session Property Name
    list, select
    Custom Properties
    , and set the
    Session Property Value
    to
    Yes
    .
Consider the following guidelines when the target operations are processed in a predefined order:
  • After you run a mapping, the results are not displayed in the order of query execution. You can view the queries and the affected rows for each query in the session log.
  • When you select the data driven target operation and set the
    Treat Source Rows As
    option to
    None
    for a target, consider the following guidelines:
    • When you enter a single value, such as, DD_INSERT, DD_UPDATE, or DD_DELETE in the data driven condition, the target operation specified in the data driven condition is considered and the target operations are processed in a predefined order of delete, update, upsert, and insert.
    • When you enter a Nested IFF statement or DD_REJECT in the data driven condition, the data driven target operation is considered and the target operations are not processed in a predefined order.
  • The order of execution of post SQL and pre SQL queries after the -DEnableSingleCommit=true property is set differs from the order of execution before the property was set.

0 COMMENTS

We’d like to hear from you!