Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Amazon Redshift
  3. PowerExchange for Amazon Redshift Configuration
  4. Amazon Redshift Connections
  5. PowerExchange for Amazon Redshift Data Objects
  6. Amazon Redshift Mappings
  7. Pushdown Optimization
  8. Amazon Redshift Lookup
  9. Appendix A: Amazon Redshift Datatype Reference
  10. Appendix B: Troubleshooting

PowerExchange for Amazon Redshift User Guide

PowerExchange for Amazon Redshift User Guide

Retain Staging Files

Retain Staging Files

You can retain staging files on Amazon S3 after the Data Integration Service writes data to the target. You can retain files to create a data lake of your organizational data on Amazon S3. The files you retain can also serve as a backup of your data.
When you configure the write operation, you can provide a file prefix or directory prefix to save the staging files. After you provide the prefixes, the Data Integration Service creates files within the directories at Amazon S3 location specified in the target connection. Configure one of the following options for the
Prefix for Retaining Staging Files on S3
property:
  • Provide a directory prefix and a file prefix. For example, backup_dir/backup_file. The Data Integration Service creates the following directories and files:
    • backup_dir_<year>_<month>_<date>_<timestamp_inLong>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Provide a file prefix. For example, backup_file. The Data Integration Service creates the following directories and files:
    • <year>_<month>_<date>_<timestamp_inLong>
    • backup_file.batch_<batch_number>.csv.<file_number>.<encryption_if_applicable>
  • Do not provide a prefix. The Data Integration Service does not save the staging files.
If you run a mapping on the Blaze mode, you must only provide a directory prefix. The file prefix is ignored. The file creation is dynamic on the Blaze engine.

0 COMMENTS

We’d like to hear from you!