Amazon S3 V2 Connector

Back Next

Source types in Amazon S3 V2 sources

You can select the type of source from which you want to read data.

You can select the following type of sources from the

Source Type

option under the Amazon S3 V2 advanced source properties:

File: You must enter the bucket name that contains the Amazon S3 file.
If applicable, include the folder name that contains the target file in the
<bucket_name>/<folder_name>
format.; Amazon S3 V2 Connector provides the option to override the value of the
Folder Path
and
File Name
properties during run time.
If you do not provide the bucket name and specify the folder path starting with a slash (/) in the
/<folder_name>
format, the folder path appends with the folder path that you specified in the connection properties.

For example, if you specify the
/<dir2>
folder path in this property and
<my_bucket1>/<dir1>
folder path in the connection property, the folder path appends with the folder path that you specified in the connection properties in
<my_bucket1>/<dir1>/<dir2>
format.

If you specify the
<my_bucket1>/<dir1>
folder path in the connection property and
<my_bucket2>/<dir2>
folder path in this property, the Secure Agent writes the file in the
<my_bucket2>/<dir2>
folder path that you specify in this property.

Directory: You must select the source file when you create a mapping and select the source type as
Directory
at run time. When you select the
Source Type
option as
Directory
, the value of
File Name
is honored only when you use wildcard characters to specify the folder path or file name, or recursively read files from directories.

For the read operation, if you provide the
Folder Path
value during run time, the Secure Agent considers the value of the
Folder Path
from the advanced source properties. If you do not provide the
Folder Path
value during run time, the Secure Agent considers the value of the
Folder Path
that you specify during the connection creation.

Use the following rules and guidelines to select
Directory
as the source type:
All the source files in the directory must contain the same metadata.
All the files must have data in the same format. For example, delimiters, header fields, and escape characters must be same.
All the files under a specified directory are parsed. The files under subdirectories are parsed only when you recursively read files from directories.