User Guide

Back Next

Complex File Targets on HDFS

Use a PowerExchange for HDFS connection to write data to HDFS data objects.

The following table shows the complex files that a mapping can process within HDFS storage in the Hadoop environment:

File Type	Supported Formats	Supported Engines
Avro	Flat Hierarchical ^{1 2}	Blaze Spark
JSON	Flat¹ Hierarchical ^{1 2}	Blaze Spark
ORC	Flat	Spark
Parquet	Flat Hierarchical ^{1 2}	Blaze Spark
¹ To run on the Blaze engine, the complex file data object must be connected to a Data Processor transformation. ² To run on the Spark engine, the complex file write operation must be enabled to project columns as complex data type.