Amazon S3 V2 Connector

Back Next

Amazon S3 V2 targets in mappings

In a mapping, you can configure a Target transformation to represent an Amazon S3 V2 object as the target to write data to Amazon S3.

Specify the name and description of the Amazon S3 V2 target. Configure the Amazon S3 V2 target and advanced properties for the target object.

The following table describes the Amazon S3 V2 target properties that you can configure in a Target transformation:

Property	Description
Connection	Name of the Amazon S3 V2 target connection. Select a target connection or click New Parameter to define a new parameter for the target connection. If you want to overwrite the parameter at runtime, select the Allow parameter to be overridden at run time option when you create a parameter. When the task runs, the agent uses the parameters from the file that you specify in the task advanced session properties.
Target Type	Target type. Select one of the following types: Single Object Parameter: Select Parameter to define the target type when you configure the mapping task.
Object	Name of the target object. You can select an existing object or create an object at runtime. When you create an object at runtime, enter a name and the path for the target object.
Parameter	Select an existing parameter for the source object or click New Parameter to define a new parameter for the target object. The Parameter property appears only if you select Parameter as the target type. If you want to overwrite the parameter at runtime, select the Allow parameter to be overridden at run time option when you create a parameter. When the task runs, the agent uses the parameters from the file that you specify in the task advanced session properties.
Create Target	Creates a target. Enter a name and path for the target object. You can use parameters defined in a parameter file in the target name. For more information, see Rules and guidelines for creating a target.
Format	Specifies the file format that the Amazon S3 V2 Connector uses to write data Amazon S3. You can select the following file format types: None ¹ Flat Avro ORC Parquet JSON² Default is None . If you select None is as the format type, the Secure Agent writes data to Amazon S3 files in binary format. Open the Formatting Options dialog box to define the format of the file. For more information about format options, see File formatting options.
Operation	Type of the target operation. You can perform only insert operation on an Amazon S3 V2 target.
¹Doesn't apply to mappings in advanced mode. ²Applies only to mappings in advanced mode.

The following table describes the Amazon S3 V2 advanced target properties that you can configure in a Target transformation:

Property	Description
Overwrite File(s) If Exists	Overwrites an existing target file. Default is true. For more information about overwriting the existing files, see Overwriting existing files.
Folder Path	Bucket name or folder path where you want to write the Amazon S3 target file. The path that you enter here overrides the path specified for the target configured to create at runtime. If applicable, include the folder name that contains the target file in the <bucket_name>/<folder_name> format. If you do not provide the bucket name and specify the folder path starting with a slash (/) in the /<folder_name> format, the folder path appends with the folder path that you specified in the connection properties. For example, if you specify the /<dir2> folder path in this property and <my_bucket1>/<dir1> folder path in the connection property, the folder path appends with the folder path that you specified in the connection properties in <my_bucket1>/<dir1>/<dir2> format. If you specify the <my_bucket1>/<dir1> folder path in the connection property and <my_bucket2>/<dir2> folder path in this property, the Secure Agent writes the file in the <my_bucket2>/<dir2> folder path that you specify in this property.
File Name	Creates a new file name or overwrites an existing target file name.
Encryption Type	Method you want to use to encrypt data. Select one of the following encryption types: None Client Side Encryption¹ Server Side Encryption Server Side Encryption with KMS Informatica Encryption Default is None . For more information about the encryption type, see Data encryption in Amazon S3 V2 targets.
Staging Directory¹	Enter the path of the local staging directory. Ensure that the user has write permissions on the directory. In addition, ensure that there is sufficient space to enable staging of the entire file. Default staging directory is the /temp directory on the machine that hosts the Secure Agent. When you specify the directory path, the Secure Agent create folders depending on the number of partitions that you specify in the following format: InfaS3Staging<00/11><timestamp>_<partition number> where, 00 represents read operation and 11 represents write operation. For example, InfaS3Staging000703115851268912800_0 The temporary files are created within the new directory. The staging directory target property does not apply to Avro, ORC, and Parquet files.
File Merge	This property is not applicable for Amazon S3 V2 Connector.
Hadoop Performance Tuning Options	This property is not applicable for Amazon S3 V2 Connector.
Compression Format	Compresses data when you write data to Amazon S3. You can compress the data in the following formats: None Bzip2² Deflate Gzip Lzo Snappy Zlib Default is None. Amazon S3 V2 Connector does not support the Lzo compression format even though the option appears in this property. For more information about the compression format, see Data compression in Amazon S3 V2 sources and targets.
Object Tags	The key value pairs to add single or multiple tags to the objects stored on the Amazon S3 bucket. You can either enter the key value pairs or specify the file path that contains the key value pairs. Use this property when you run a mapping to write a file of flat format type. For more information about the object tags, see Object tag.
TransferManager Thread Pool Size¹	The number of threads to write data in parallel. Default is 10. Use this property when you run a mapping to write a file of flat format type. Amazon S3 V2 Connector uses the AWS TransferManager API to upload a large object in multiple parts to Amazon S3. When the file size is more than 5 MB, you can configure multipart upload to upload object in multiple parts in parallel. If you set the value of TransferManager Thread Pool Size to greater than 50, the value reverts to 50.
Merge Partition Files¹	Determines whether the Secure Agent must merge the number of partition files as a single file or maintain separate files based on the number of partitions specified to write data to the Amazon S3 V2 targets.
Temporary Credential Duration	The time duration during which an IAM user can use the dynamically generated temporarily credentials to access the AWS resource. Enter the time duration in seconds. Default is 900 seconds. If you require more than 900 seconds, you can set the time duration maximum up to 12 hours in the AWS console and then enter the same time duration in this property.
Part Size¹	Uploads the part size of an Amazon S3 object in bytes. Default is 5 MB. Use this property when you run a mapping to write a file of flat format type.
Forward Rejected Rows	This property is not applicable for Amazon S3 V2 Connector.
¹Doesn't apply to mappings in advanced mode. ²Applies only to mappings in advanced mode.

When you create a mapping and the column name in the Amazon S3 source or target object contains special characters, the Secure Agent replaces the special characters with an underscore

(_)

and the mapping fails.

Amazon S3 V2 objects in mappings

Download Guide

Watch

Comments

Cloud Connectivity Homepage

Communities

Knowledge Base

Success Portal