Table of Contents

Search

Parquet Amazon S3 File Data Types and Transformation Data Types

Parquet Amazon S3 File Data Types and Transformation Data Types

Amazon S3 file data types map to transformation data types that the Data Integration Service uses to move data across platforms.
The following table lists the Amazon S3 file data types that the Data Integration Service supports and the corresponding transformation data types:
Amazon S3 File Data Type
Transformation Data Type
Range and Description
Boolean
Integer
TRUE (1) or FALSE (0)
Byte_Array
Binary
Arbitrarily long byte array
Double
Double
Precision 15
Float
Double
Precision 15
Applicable when you run a mapping on the Spark engine.
Int32
Integer
-2,147,483,648 to +2,147,483,647
Int64
Bigint
-9,223,372,036,854,775,808 to +9,223,372,036,854,775,807
8-byte signed integer
Int96
Binary
12-byte signed integer
Applicable when you run a mapping on the Spark engine.
The Parquet schema that you specify to read or write a Parquet file must be in smaller case. Parquet does not support case-sensitive schema.
The Developer tool does not support the following Parquet data types:
  • Int96 (TIMESTAMP_MILLIS)
  • Date
  • Timestamp


Updated July 30, 2020