Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Avro Data Types

Avro Data Types

The following table lists the Avro data types that Data Integration Service supports and the corresponding transformation data types:
Avro Data Type
Transformation Data Type
Range and Description
array
array
The elements in the array are of string, double, or int data type. The elements in the array are delimited by commas. For example, an array of fruits is represented as [apple,banana,orange]
boolean
integer
The default transformation type for boolean is integer. You can specify string data type with values of True and False. True is equivalent to the integer 1 and False is equivalent to the integer 0.
datetime
date/time
The time stamp format is YYYY-MM-DD HH:MM:SS.SSS. Precision 29, scale 9.
double
double
-1.79769313486231570E+308 to +1.79769313486231570E+308. Precision 15.
float
double
-1.79769313486231570E+308 to +1.79769313486231570E+308. Precision 15.
int
integer
-2,147,483,648 to 2,147,483,647 Precision 10, scale 0
long
bigint
-9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0.
map
map
Maps contain key-value pairs and are represented as pairs of strings and integers delimited by the =character. String and integer pairs are delimited by commas. For example, a map of fruits is represented as [1=apple,2=banana,3=orange].
record
struct
Unlimited number of characters.
string
string
1 to 104,857,600 characters.
struct
struct
Structs are represented as pairs of strings and integers delimited by the : character. String and integer pairs are delimited by commas. For example, a struct of fruits is represented as [1,apple]
union
Corresponding data type in a union of ["primitive_type|complex_type", "null"] or ["null", "primitive_type|complex_type"]
Dependent on primitive or complex data type.