Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for HDFS
  3. PowerExchange for HDFS Configuration
  4. HDFS Connections
  5. HDFS Data Objects
  6. HDFS Data Extraction
  7. HDFS Data Load
  8. HDFS Mappings
  9. Appendix A: Data Type Reference

PowerExchange for HDFS User Guide

PowerExchange for HDFS User Guide

Schema Properties

Schema Properties

The Developer tool displays the schema properties for Avro, JSON, ORC and Parquet complex file targets in the Properties view of the
Write
operation.
The following table describes the schema properties that you configure for Avro, JSON, and Parquet complex file targets:
Property
Description
Column Name
Displays the name of the column.
Column Type
Displays the format of the column.
Enable Column Projection
Displays the column details of the complex file targets.
Schema Format
Displays the schema format that you selected while creating the complex file data object. You can change the schema format and provide respective schema.
You can select one of the following options:
  • Avro
  • Json
  • Parquet
  • Orc
  • Xml
  • Assign Parameter
You can change the complex file format without losing the column metadata even after you configure the column projection properties for another complex file format.
You can switch from one schema format to another only once. If you change the schema format more than once, you might lose the original datatypes.
Schema
Displays the schema associated with the complex file. You can select a different schema.
You can select one of the following options:
  • Browse
  • Assign Parameter
  • Assign Path as Parameter
When you use Refresh Schema for the source or target in a mapping and also, parameterize the schema, the parameterized schema takes precedence over the refresh schema.
  • If you disable the column projection, the schema associated with the complex file is removed. If you want to associate schema again with the complex file, enable the column projection and click Select Schema.
  • When you parameterize the schema in a Parquet complex file, the schema should not contain a String data type, use UTF8 data type instead.
Column Mapping
Displays the mapping between input and output ports.
If you disable the column projection, the mapping between input and output ports is removed. If you want to map the input and output ports, enable the column projection and click Select Schema to associate a schema to the complex file.

0 COMMENTS

We’d like to hear from you!