Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Microsoft Azure Blob Storage
  3. PowerExchange for Microsoft Azure Blob Storage Configuration
  4. Microsoft Azure Blob Storage Connections
  5. Microsoft Azure Blob Storage Data Objects
  6. Microsoft Azure Blob Storage Mappings
  7. Data Type Reference

PowerExchange for Microsoft Azure Blob Storage User Guide

PowerExchange for Microsoft Azure Blob Storage User Guide

Parquet File Data Types and Transformation Data Types

Parquet File Data Types and Transformation Data Types

Parquet file data types map to transformation data types that the Data Integration Service uses to move data across platforms.
The following table lists the Parquet file data types that the Data Integration Service supports and the corresponding transformation data types:
Parquet File Data Type
Transformation Data Type
Range and Description
Binary (UTF-8)
String
1 to 104,857,600 characters
Boolean
Integer
TRUE (1) or FALSE (0)
Byte_Array
Binary
Arbitrarily long byte array
Double
Double
Precision 15
Float
Double
Precision 15
Int32
Integer
-2,147,483,648 to +2,147,483,647
Precision of 10, scale of 0
Int64
Bigint
-9,223,372,036,854,775,808 to +9,223,372,036,854,775,807
Precision of 19, scale of 0
Map
Map
Unlimited number of characters.
Struct
Struct
Unlimited number of characters.
Union
Corresponding primitive data type in a union of ["primitive_type", "null"] or ["null", "primitive_type"].
Dependent on primitive data type.
The Parquet schema that you specify to read or write a Parquet file must be in smaller case. Parquet does not support case-sensitive schema.

Parquet Union Data Type

A union indicates that a field might have more than one data type. For example, a union might indicate that a field can be a string or a null. A union is represented as a JSON array containing the data types.The Developer tool only interprets a union of ["primitive_type", "null"] or ["null", "primitive_type"]. The Parquet data type converts to the corresponding transformation data type. The Developer tool ignores the null.

Unsupported Parquet Data Types

The Developer tool does not support the following Parquet data types:
  • int96 (TIMESTAMP_MILLIS)
  • date
  • timestamp