PowerExchange for HDFS User Guide

10.5.4
- 10.5.9
- 10.5.8
- 10.5.7
- 10.5.6
- 10.5.3
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0

Back Next

Schema Properties

The Developer tool displays the schema properties for intelligent structure model, Avro, JSON, ORC and Parquet complex file sources in the Properties view of the

Read

operation.

The following table describes the Schema properties that you configure for the complex file sources:

Property	Description
Column Name	Displays the name of the column.
Column Type	Displays the format of the column.
Enable Column Projection	Displays the column details of the complex files sources.
Schema Format	Displays the schema format that you selected while creating the complex file data object. You can change the schema format and provide respective schema. You can select one of the following options: Avro Json Orc Parquet Xml Intelligent Structure Model Assign Parameter You can change the complex file format without losing the column metadata even after you configure the column projection properties for another complex file format. You can parameterize the schema format using the Assign Parameter option. You can switch from one schema format to another only once. If you change the schema format more than once, you might lose the original datatypes.
Schema	Displays the schema associated with the complex file. You can select a different schema. You can select one of the following options: Browse Assign Parameter Assign Path as Parameter For the Assign Path as Parameter option, the path can be obtained from the server. When you use Refresh Schema for the source or target in a mapping and also, parameterize the schema, the parameterized schema takes precedence over the refresh schema. If you disable the column projection, the schema associated with the complex file is removed. If you want to associate schema again with the complex file, enable the column projection and click Select Schema. When you parameterize the schema in a Parquet complex file, the schema should not contain a String data type, use UTF8 data type instead.
Column Mapping	Displays the mapping between input and output ports. If you disable the column projection, the mapping between input and output ports is removed. If you want to map the input and output ports, enable the column projection and click Select Schema to associate a schema to the complex file.

In the native environment, Data Preview and a mapping may fail in the following scenarios:

When you import an Avro file as a source object and switch the schema format to Parquet and select a Parquet file as the source object.

When you import a JSON file as a source object and switch the schema format to Avro and select an Avro file as the source object.

You must edit the schema as per the selected schema format or enable the refresh schema at runtime option in the mapping if you want to change the schema format.

Rename Saved Search

Table of Contents

PowerExchange for HDFS User Guide

PowerExchange for HDFS User Guide

Schema Properties

Schema Properties