Amazon S3 V2 Connector

Back Next

Rules and guidelines for mappings in advanced mode

Consider the following guidelines when you create a mapping in advanced mode:

When you run a mapping in advanced mode, a folder of the following format is created in the target and multiple target files are generated within the folder:

<target_foldername>.<file_extension>

When a mapping in advanced mode writes data to an Amazon S3 file, the file is replaced with a folder and the target file is generated inside the folder. If you create another mapping in advanced mode that references the same Amazon S3 file, the file cannot be found and the mapping fails with the following error message:

Operation failed: Index: 0, Size: 0.

When you configure a read operation to read from a file, the file name should not begin with

When you configure a Lookup transformation and select the Report Error option to report an error on multiple matches, the data from the unmatched columns is written to the target without displaying an error message.

When you read data from a JSON file and if one of the rows contains an incorrect boolean value, all rows are rejected and a null value is written to the target.

When you read data from a JSON file that contains unicode characters, data inconsistencies might occur.

If you select data types that Amazon S3 V2 Connector does not support, the mapping might either fail or reject the rows.

When there are empty array elements such as

"{"Elements":[]}"

as part of the JSON sample data file for metadata resolution, the JSON parser fails with the following error:

[SDK_APP_COM_20000] error [Array must contain at least 1 element for projection].

Provide the entire schema as a sample data row without any empty array for metadata resolution. You can use the

Import from schema file

option to upload this file.

When there are empty struct elements such as

"{"Elements":[]}"

as part of the JSON sample data file for metadata resolution, the JSON parser fails with the following error:

Struct must contain at least one key :: fields

Provide the entire schema as a sample data row without any empty structs for metadata resolution. You can use the

Import from schema file

option to upload this file.

The JSON parser interprets the data type for an array element using the first value from the array. For example, if the first value is an integer and subsequently contains long values, the metadata is resolved as an integer. During runtime, the entire row is dropped because the long value cannot fit into the DTM buffer.

Provide the entire schema as a sample data row with the first array or struct elements containing the data types or sub-fields that are required for the metadata resolution. You can use the

Import from schema file

option to upload the file.

There is a limit of 2 MB to the first row size used to interpret metadata from a JSON file. If the first row in the data file is larger than 2 MB, the JSON parser fails.

Decrease the sample data row size by removing the additional tags from the struct and array elements. The JSON parser only requires the first element within struct and array elements. Provide the data types or sub-fields that are required for metadata resolution in the first element. You can use the

Import from schema file

option to upload the file.

The staging directory source and target advanced property is not applicable. However, you must specify a staging directory on Amazon S3 in

advanced configurations

. For more information, see

Administrator

When you set the qualifier mode to Minimal and use an escape character, the characters are not escaped and quoted in the target. Set the qualifier mode to All.

When you set the qualifier mode to All and do not specify a value for the qualifier, \0 (NUL) is considered as the qualifier.

When you parameterize a flat source file, the FileName field appears in the source fields. As a workaround, add a rule in the incoming fields of the target to exclude the FileName field.

If the table or schema names contain a backward slash (/), the mapping fails.

If the JSON data that you read from a source fails to align with the source schema defined in the schema definition file, the data written to the target appears corrupted.

Rename Saved Search

Table of Contents

Amazon S3 V2 Connector

Amazon S3 V2 Connector

Rules and guidelines for mappings in advanced mode

Rules and guidelines for mappings in advanced mode