Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Amazon Redshift connectors
  3. Part 2: Data Integration with Amazon Redshift V2 Connector
  4. Part 3: Data Integration with Amazon Redshift Connector

Amazon Redshift Connectors

Amazon Redshift Connectors

Retain staging files

Retain staging files

You can retain staging files on Amazon S3 after the writes data to the target. You can retain files to create a data lake of your organizational data on Amazon S3. The files you retain can also serve as a backup of your data.
When you create a target connection, you can configure a file prefix or directory prefix to save the staging files. After you provide the prefixes, the creates files within the directories at Amazon S3 location specified in the target connection. Configure one of the following options for the
Prefix for Retaining Staging Files on S3
property:
  • Provide a directory prefix and a file prefix. For example, backup_dir/backup_file. The creates the following directories and files:
    • backup_dir_<year>_<month>_<date>_<timestamp_inLong>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Provide a file prefix. For example, backup_file. The creates the following directories and files:
    • <year>_<month>_<date>_<timestamp_inLong><3 digit of random number>00<ProcessID><PartitionId>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Do not provide a prefix. The does not save the staging files.