Cloud Data Integration for PowerCenter
- Cloud Data Integration for PowerCenter
- All Products
ASK INFAPreview
Please log in to access Ask INFA.
ASK INFAPreview
Connection property
| Description
|
|---|---|
Schema
| The Amazon Redshift schema name to be used when creating the object. Th schema name is similar to the schema name specified in the
SCHEMAMAP.properties file.
Default is
schema_changeit .
|
Property name
| Description
|
|---|---|
Schema Name
| Overrides the default schema name.
|
S3 Bucket Name
| Amazon S3 bucket name for staging the data.
|
Read Mode | Specifies the read mode to read data from the
Amazon Redshift source. You can specify one of the following
read modes:
Staging . |
Fetch Size | Determines the number of rows to read in one
resultant set from Amazon Redshift. Applies only when you select
the Direct read mode.Default is
10000 . |
Enable Compression | Compresses the staging files into the Amazon
S3 staging directory. The task performance improves when the
Secure Agent compresses the staging files. Default is
enabled. |
Staging Directory Location | Location of the local staging directory. When
you run a task in the Secure Agent runtime environment, specify
a directory path that is available on the corresponding Secure
Agent machine in the runtime environment. For example,
C:\Temp . Ensure that you have the
write permissions to the directory. |
Unload Options | Unload command options. Enter the options to
the Unload command to extract data from Amazon Redshift and
create staging files on Amazon S3. You can enter the following
unload options:
For example: DELIMITER = \036;ESCAPE =
OFF;NULL=text;PARALLEL =
ON;AWS_IAM_ROLE=arn;aws;iam;;<account
ID>;role/<role-name>;REGION =
ap-south-1 For the AWS_IAM_ROLE option,
provide the Amazon Resource Name (ARN) of an Amazon Redshift
Role. Also, specify a directory on the machine that hosts
the Secure Agent. |
Temporary Credential Duration | The time duration during which an IAM user can
use the dynamically generated temporarily credentials to access
the AWS resource. Enter the time duration in seconds. Default
is 900 seconds. If you require more than 900
seconds, you can set the time duration up to a maximum of 12
hours in the AWS console and then enter the same time
duration in this property. |
Treat NULL Value as NULL | Retains the null values when you read data
from Amazon Redshift. |
Encryption Type | Encrypts the data in the Amazon S3 staging
directory. You can specify the following encryption types:
Default is None. |
Download S3 Files In Multiple Parts | Downloads large Amazon S3 objects in multiple
parts. When the file size of an Amazon S3 object is greater
than 8 MB, you can choose to download the object in multiple
parts in parallel. Default is 5 MB. |
Multipart Download Threshold Size | The maximum threshold size to download an
Amazon S3 object in multiple parts. Default is 5 MB. |
Property | Description |
|---|---|
S3 Bucket Name | Amazon S3 bucket name for writing the files to
Amazon Redshift target. You can also specify the bucket name
with the folder path. If you provide an Amazon S3 bucket
name that is in a different region than the Amazon Redshift
cluster, you must configure the
REGION attribute in the
Copy Options property. |
Enable Compression | Compresses the staging files before writing
the files to Amazon Redshift. The task performance improves
when the Secure Agent compresses the staged files. Default
is enabled. |
Staging Directory Location | Location of the local staging directory.
When you run a task in Secure Agent runtime environment
specify a directory path that is available on the
corresponding Secure Agent machine in the runtime
environment. For example,
C:\Temp . Ensure that you have the write
permissions on the directory. |
Batch Size | Minimum number of rows in a batch. Enter a
number greater than 0. Default is 2000000. |
Max Errors per Upload Batch for INSERT | Number of error rows that causes an upload
insert batch to fail. Enter a positive integer. Default is 1.
If the number of errors is equal to or greater than
the property value, the Secure Agent writes the entire batch
to the error file. |
Require Null Value For Char and Varchar | Replaces the string value with NULL when you
write data to Amazon Redshift columns of Char and Varchar data
types. Default is an empty string. When you run
a mapping to write null values to a table that contains a
single column of the Int, Bigint, numeric, real, or double
data type, the mapping fails. You must provide a value other
than the default value in the Require Null Value
For Char And Varchar property. |
WaitTime In Seconds For S3 File Consistency | Number of seconds to wait for the Secure
Agent to make the staged files consistent with the list of files
available on Amazon S3. Default is 0. |
Copy Options | Copy command options. Enter the options to
the Copy command to write data from Amazon S3 to the Amazon
Redshift target when the default delimiter comma (,) or
double-quote (") is used in the data. Provide the Amazon
Redshift Role Amazon Resource Name (ARN). You can add the following options:
For example:
Specify a directory on the machine that hosts the Secure
Agent. If you do
not add the options to the Copy command manually, the Secure
Agent uses the default values. |
S3 Server Side Encryption | Indicates that Amazon S3 encrypts data during
upload. Provide a customer master key ID in the connection
property to enable this property. Default is not selected.
|
S3 Client Side Encryption | Indicates that the Secure Agent encrypts data
using a private key. Provide a master symmetric key ID in the
connection property to enable this property. If you enable
both server-side and client-side encryptions, the Secure
Agent ignores the server-side encryption. |
Analyze Target Table | Runs an ANALYZE command on the target table.
The query planner on Amazon Redshift updates the
statistical metadata to build and choose optimal plans to
improve the efficiency of queries. |
Vacuum Target Table | Recovers disk space and sorts the row in a
specified table or all tables in the database. You can select
the following recovery options:
Default is None. |
Prefix to retain staging files on S3 | Retains staging files on Amazon S3. Provide
both a directory prefix and a file prefix separated by a
slash (/) or only a file prefix to retain staging files on
Amazon S3. For example,
backup_dir/backup_file or
backup_file . |
Success File Directory | Directory for the Amazon Redshift success
file. Specify a directory on the machine that hosts the
Secure Agent. |
Error File Directory | Directory for the Amazon Redshift error file.
Specify a directory on the machine that hosts the Secure
Agent. |
Transfer Manager Thread Pool Size | Number of threads to write data in parallel.
Default is 10. |
Preserve record order on write | Retains the order of the records when you
read data from a CDC source and write data to an Amazon Redshift
target. Use this property when you create a mapping to
capture the changed record from a CDC source. This property
enables you to avoid inconsistencies between the CDC source
and target. |
Minimum Upload Part Size | Minimum size of the Amazon Redshift object to
upload an object. Default is 5 MB. |
Number of files per batch | Calculates the number of the staging files
per batch. If you do not provide the number of files, Amazon
Redshift V2 Connector calculates the number of the staging
files. When you connect to Amazon Redshift serverless
and do not provide the number of files, 20 files per batch
is considered as the default value. |
Schema Name | Overrides the default schema name. |
Recovery Schema Name | Schema that contains recovery information
stored in the infa_recovery_table table on the
target system to resume the extraction of the changed data from
the last checkpoint. |
Temporary Credential Duration | The time duration during which an IAM user can
use the dynamically generated temporarily credentials to access
the AWS resource. Enter the time duration in seconds. Default
is 900 seconds. If you require more than 900
seconds, you can set the time duration maximum up to 12
hours in the AWS console and then enter the same time
duration in this property. |
Property | Description |
|---|---|
Read Mode | Specifies the read mode to read data from the
Amazon Redshift source. You can specify one of the following
read modes:
Staging . |
Fetch Size | Determines the number of rows to read in one
resultant set from Amazon Redshift. Applies only when you select
the Direct read mode.Default is
10000 . |
S3 Bucket Name | Amazon S3 bucket name for staging the data.
You can also specify the bucket name with the folder
path. If you provide an Amazon S3 bucket name that is in a
different region than the Amazon Redshift cluster, you must
configure the REGION attribute in the
Unload command options. |
Enable Compression | Compresses the staging files into the Amazon
S3 staging directory. The task performance improves when the
Secure Agent compresses the staging files. Default is
selected. |
Staging Directory Location | Location of the local staging directory.
When you run a task in Secure Agent runtime environment,
specify a directory path that is available on the
corresponding Secure Agent machine in the runtime
environment. Specify the directory path in the
following manner: <staging
directory>
For example, C:\Temp . Ensure that
you have the write permissions on the directory.
|
Unload Options | Unload command options. Add options to the
Unload command to extract data from Amazon Redshift and
create staging files on Amazon S3. Provide an Amazon
Redshift Role Amazon Resource Name (ARN). You can add
the following options:
For example: DELIMITER = \036;ESCAPE =
OFF;NULL=text;PARALLEL =
ON;AWS_IAM_ROLE=arn;aws;iam;;<account
ID>;role/<role-name>;REGION = ap-south-1
Specify a directory on the machine that hosts the
Secure Agent. If you
do not add the options to the Unload command manually,
the Secure Agent uses the default values. |
Treat NULL Value as NULL | Retains the null values when you read data
from Amazon Redshift. |
Encryption Type | Encrypts the data in the Amazon S3 staging
directory. You can select the following encryption types:
Default is None. |
Download S3 Files in Multiple Parts | Downloads large Amazon S3 objects in multiple
parts. When the file size of an Amazon S3 object is greater
than 8 MB, you can choose to download the object in multiple
parts in parallel. Default is 5 MB. |
Multipart Download Threshold Size | The maximum threshold size to download an
Amazon S3 object in multiple parts. Default is 5 MB.
|
Schema Name | Overrides the default schema name. You cannot
configure a custom query when you use the schema name.
|
Temporary Credential Duration | The time duration during which an IAM user can
use the dynamically generated temporarily credentials to access
the AWS resource. Enter the time duration in seconds. Default
is 900 seconds. If you require more than 900
seconds, you can set the time duration up to a maximum of 12
hours in the AWS console and then enter the same time
duration in this property. |
