Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Amazon Redshift connectors
  3. Part 2: Data Integration with Amazon Redshift V2 Connector
  4. Part 3: Data Integration with Amazon Redshift Connector

Amazon Redshift Connectors

Amazon Redshift Connectors

Retain staging files

Retain staging files

You can retain staging files on Amazon S3 after the Secure Agent writes data to the target. You can retain files to create a data lake of your organizational data on Amazon S3. The files you retain can also serve as a backup of your data.
When you create a target connection, you can configure a file prefix or directory prefix to save the staging files. After you provide the prefixes, the Secure Agent creates files within the directories at the Amazon S3 location specified in the target. Configure one of the following options for the
Prefix for Retaining Staging Files on S3
property:
  • Provide a directory prefix and a file prefix. For example,
    backup_dir/backup_file
    . The Secure Agent creates the following directories and files:
    • backup_dir_<year>_<month>_<date>_<timestamp_inLong>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Provide a file prefix. For example, backup_file. The Secure Agent creates the following directories and files:
    • <year>_<month>_<date>_<timestamp_inLong><3 digit of random number>00<ProcessID>
      <PartitionId>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Do not provide a prefix. The Secure Agent does not save the staging files.

0 COMMENTS

We’d like to hear from you!