Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Amazon Redshift connectors
  3. Part 2: Data Integration with Amazon Redshift V2 Connector
  4. Part 3: Data Integration with Amazon Redshift Connector

Amazon Redshift Connectors

Amazon Redshift Connectors

Troubleshooting for Amazon Redshift V2 Connector

Troubleshooting for Amazon Redshift V2 Connector

Using the JVM option for upsert operation

For an upsert operation, you can set the JVM option
-DuseDeleteForUpsert=true
and connect all the fields of the target table. Use the following rules when you set the JVM option:
  • When you set the JVM option, the Secure Agent deletes the records of the target table based on the primary key match between the source table and the target table. All the records of the source table are inserted into the target table. When you do not set the JVM option, the Secure Agent updates existing rows and inserts other rows as if marked for insert.
  • For elastic mappings, configure the JVM option in the
    Spark Session Properties
    when you create a mapping task.
  • When you use a target table that has all the columns forming a composite key and you do not set an update column in the target, the mapping might fail with a warning. When you set the JVM option, the mapping task runs successfully.
  • When you use a target table that has duplicate values in an update column, the target table has lesser number rows after you run the mapping. When you do not set the JVM option, the target table does not have lesser number of rows.
  • When you use a source table that has duplicate values in a column and select the column as an update column on the target, an additional row is added to the target. When you do not set the JVM option, an additional row is not added to the target.
  • For an upsert operation, when you use a table with column names that contain special characters and do not set the JVM option, the mapping fails.

How to implement the upsert operation using Amazon Redshift V2 Connector?

For information about implementing the upsert operation, see Implementing the upsert operation using Amazon Redshift V2 Connector.

How to configure AWS IAM authentication for Amazon Redshift V2 Connector?

For information about configuring AWS IAM authentication, see Configuring AWS IAM Authentication for Amazon Redshift and Amazon Redshift V2 Connectors.