To create a Data Processor transformation for Parquet output, use the Data Processor transformation wizard. The wizard creates the transformation in the Model repository with the components you need to transform Parquet output. Use the XMap editor to edit the XMap, if included. Add the transformation to a mapping with a complex file writer.
Create the Data Processor transformation with the New Transformation wizard. Add a Parquet schema or example file that defines the expect output structure.
If the transformation has XML, JSON, or another structured format as input, use the XMap editor to edit and customize the XMap that the wizard created in the transformation.
To configure the output settings for the Data Processor transformation, in the
Output Control
panel of the
Settings
view, select the relevant output option in the
Binary output port collection mode
area.
To configure the output to send one record at a time with the schema, select the
Output row for each row
option. To collect all the records with the schema into one output stream, select the
Collect input rows to a single output
option.
Add the Data Processor transformation to a mapping. The transformation remains set to binary output, the default setting for Parquet output.
Create and link a complex file writer to the Data Processor transformation in the mapping to receive the Parquet output from the transformation.
The input setting remains set to binary, the default setting. The complex file writer does not support compression for Parquet output.
Configure the Complex File Writer. In the Data Object Operation, in the