Amazon S3 V2 Connector

Back Next

Amazon Redshift manifest file

You can use an Amazon Redshift manifest file created by the UNLOAD command to read multiple flat files from Amazon S3. All flat files must have the same metadata and must be available in the same Amazon S3 bucket.

Create a

.manifest

file and list all the source files with the URL that includes the bucket name and full object path for the file. You must specify the

.manifest

file name in the following format:

<file_name>.manifest

For example, the Amazon Redshift manifest file contains source files in the following format:

{
    "entries": [
            {"url": "s3://mybucket-alpha/2013-10-04-custdata", "mandatory":true},
            {"url": "s3://mybucket-alpha/2013-10-05-custdata", "mandatory":true},
            {"url": "s3://mybucket-beta/2013-10-04-custdata", "mandatory":true},
            {"url": "s3://mybucket-beta/2013-10-05-custdata", "mandatory":true},
    ]
}

The Redshift manifest file format contains the following tags:

url: The url tag consists of the source file in the following format:
"url": "<endpoint name>://<folder path>/<filename>", "mandatory":<value>
mandatory: Amazon S3 V2 Connector uses the
mandatory
tag to determine whether to continue reading the files in the
.manifest
file or not, based on the following scenarios:

If the value of
mandatory
tag is
true
, and the S3 bucket does not have the specified source file, Amazon S3 V2 Connector does not read the rest of the files as well in the
.manifest
file. The mapping task fails.

If the value of
mandatory
tag is
false
, and the S3 bucket does not have the specified file, Amazon S3 V2 Connector continues to read the rest of the files in the
.manifest
file in a sequence.

If the
.manifest
file does not contain any files, the mapping task fails.
By default, the value of
mandatory
tag is
false
.

Rename Saved Search

Table of Contents

Amazon S3 V2 Connector

Amazon S3 V2 Connector

Amazon Redshift manifest file

Amazon Redshift manifest file