English
  • Test Data Management 10.5
  • All Products

Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.5.1
  3. Part 2: Versions 10.5 - 10.5.0.1
  4. Part 3: Versions 10.4.1 - 10.4.1.3
  5. Part 4: Versions 10.4 - 10.4.0.2
  6. Part 5: Versions 10.2.2 - 10.2.2 HotFix 1
  7. Part 6: Version 10.2.1
  8. Part 7: Version 10.2 - 10.2 HotFix 2

What's New and Changed (10.5.1)

What's New and Changed (10.5.1)

Back Next

Processing Hierarchical Data on the Spark Engine

Processing Hierarchical Data on the Spark Engine

Effective in version 10.2.1, the Spark engine includes the following additional functionality to process hierarchical data:
Map data type
You can use map data type to generate and process map data in complex files.
Complex files on Amazon S3
You can use complex data types to read and write hierarchical data in Avro and Parquet files on Amazon S3. You project columns as complex data type in the data object read and write operations.
For more information, see the "Processing Hierarchical Data on the Spark Engine" chapter in the
Informatica Big Data Management 10.2.1 User Guide
.