Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Microsoft Azure Blob Storage
  3. PowerExchange for Microsoft Azure Blob Storage Configuration
  4. Microsoft Azure Blob Storage Connections
  5. Microsoft Azure Blob Storage Data Objects
  6. Microsoft Azure Blob Storage Mappings
  7. Data Type Reference

PowerExchange for Microsoft Azure Blob Storage User Guide

PowerExchange for Microsoft Azure Blob Storage User Guide

Column Projection Properties

Column Projection Properties

The Developer tool displays the column projection properties for intelligent structure model, Avro, JSON, and Parquet complex file sources in the Properties view of the
Read
operation.
The following table describes the column projection properties that you configure for the complex file sources:
Property
Description
Enable Column Projection
Displays the column details of the complex files sources.
Use Schema
Enables the schema selection for the complex file. You can select a different schema.
If you disable the column projection, the schema associated with the complex file is removed. If you want to associate schema again with the complex file, enable the column projection and click
Add Schema
.
Schema Format
Displays the schema format that you selected while creating the complex file data object. You can change the schema format and provide respective schema.
Use Intelligent Structure Model
Select this option to associate an intelligent structure model with the complex file.
Do not associate an intelligent structure model with a Write data object operation. If you use a Write operation that is associated with an intelligent structure model in a mapping, the mapping will not be valid.
Model
Displays the intelligent structure model associated with the complex file. You can select a different model.
If you disable the column projection, the intelligent structure model associated with the data object is removed. If you want to associate an intelligent structure model again with the data object, enable the column projection and click
Select Model
.
Column Mapping
Displays the mapping between input and output ports.
If you disable the column projection, the mapping between input and output ports is removed. If you want to map the input and output ports, enable the column projection and click
Select Schema
to associate a schema to the complex file.
Project Column as Complex Data Type
Displays columns with hierarchical data as a complex data type, such as, array, map, or struct. Select this property when you want to process hierarchical data on the Spark engine.
If you disable the column projection, the data type of the column is displayed as binary type.
In the native environment, Data Preview and a mapping may fail in the following scenarios:
  • When you import an Avro file as a source object and switch the schema format to Parquet and select a Parquet file as the source object.
  • When you import a JSON file as a source object and switch the schema format to Avro and select an Avro file as the source object.
You must edit the schema as per the selected schema format or enable the refresh schema at runtime option in the mapping if you want to change the schema format.