Amazon S3 V2 Connector

Back Next

Avro Amazon S3 file data types and transformation data types

Avro Amazon S3 file data types map to transformation data types that the Secure Agent uses to move data across platforms.

The following table lists the Avro Amazon S3 file data types that the Secure Agent supports and the corresponding transformation data types:

Avro Amazon S3 File Data Type	Transformation Data Type	Range and Description
Array¹	Array	Unlimited number of characters
Boolean	Integer	TRUE (1) or FALSE (0)
Date	Date/Time	January 1, 0001 to December 31, 9999.
Decimal	Decimal	For mappings- Precision 18 and 28 digits. Scale 0 to 28. If you specify a precision less than 18 or 28 digits, 18 or 28 is considered as the precision. For mappings in advanced mode- Precision 18, 28, and 38 digits. Scale 0 to 38.
Double	Double	Precision 15
Float	Double	Precision 15
Int	Integer	-2,147,483,648 to 2,147,483,647 Precision 10, scale 0
Long	Bigint	-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0
Map¹	Map	Unlimited number of characters
Null	Integer	-2,147,483,648 to 2,147,483,647 Precision 10, scale 0
Record¹	Struct	Unlimited number of characters
String	String	-1 to 104,857,600 characters
Union¹	Corresponding data type in a union of ["primitive_type\|complex_type", "null"] or ["null", "primitive_type\|complex_type"].	Dependent on primitive or complex data type.
¹Applies only to mappings in advanced mode.

The following table lists the Timestamp data type support for Avro file formats:

Consider the following rules and guidelines when you use Avro Amazon S3 file data types and transformation data types:

When you run a mapping to write data to an Amazon S3 target, you cannot use the fixed type of the Avro data type.

Specify the correct precision and scale in the source file. Otherwise, the decimal point is shifted when you write the source data to a target.

The source file must have a timestamp value greater than or equal to 1900-01-01T00:00:00Z. Otherwise, the mapping fails with an error.

When you use the

Create Target

option and modify the precision of an Avro data type in the upstream transformation, the precision is not honored.

To resolve the issue with the precision, create a target object with the default precision and subsequently edit the precision.

Watch

Comments

0 COMMENTS

We’d like to hear from you! Log in to comment.