Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Microsoft Azure Blob Storage
  3. PowerExchange for Microsoft Azure Blob Storage Configuration
  4. Microsoft Azure Blob Storage Connections
  5. Microsoft Azure Blob Storage Data Objects
  6. Microsoft Azure Blob Storage Mappings
  7. Data Type Reference

PowerExchange for Microsoft Azure Blob Storage User Guide

PowerExchange for Microsoft Azure Blob Storage User Guide

Flat File Schema Properties

Flat File Schema Properties

You can configure format properties for a flat file that is delimited.
The following table describes the file format and column format properties that you configure for a flat file:
Property
Description
Maximum row to preview
Number of rows to show in data preview. Default is 0.
Delimiters
Character used to separate columns of data. Default is comma. If you enter a delimiter that is the same as the escape character or the text qualifier, you might receive unexpected results. You cannot specify a multibyte character as a delimiter.
Text Qualifier
Quote character that defines the boundaries of text strings. Default is double quotes. If you select a quote character, the Developer tool ignores delimiters within pairs of quotes.
Qualifier Mode
Qualifier behavior for the target object.
You can select one of the following options:
  • Minimal
    . Applies the qualifier to data that contains either a delimiter value or a special character. Otherwise, the Data Integration Service does not apply the qualifier.
  • All
    . Applies qualifier to all data.
Default is minimal.
Row Delimiter
Character used to separate the rows of data.
The default value is
\012 LF (\n)
.
Target Header
Indicates whether you want to write data with or without a header.
Escape Character
Character immediately preceding a column delimiter character embedded in an unquoted string, or immediately preceding the quote character in a quoted string. When you specify an escape character, the Data Integration Service reads the delimiter character as a regular character. Default is backslash (\).
Retain Escape Character in Data
Not applicable.
If you update the flat file format properties during the data object import and want to see the updated format properties in Data Preview, you must parse the flat file again by selecting the
Schema
property.

Parameterize Column Format Properties

You can parameterize the column format properties for flat files in a parameter file. The following table describes property strings that you can use in a parameter file:
Property
Value
maxRowsToPreview
A positive integer value.
Default is 0.
delimiter
Specify the octal code for the character. Preface the octal code with a backslash (\). Specify the following values for the given delimiters:
  • \011 TAB
    for Tab
  • ;
    for semicolon
  • ,
    for comma
  • \040 SP
    for space
Default is comma.
You cannot specify a multibyte character as a delimiter
textQualifier
Specify the following string values:
  • SINGLE_QUOTES for
    '
  • DOUBLE_QUOTES for
    "
  • NO_QUOTES
Default is double quotes.
rowDelimiter
Default value,
\012 LF (\n)
.
escapeCharacter
A string value. Default is
\
.

A Sample JSON Parameter File

{ "maxRowsToPreview": 10, "delimiter": ";", "textQualifier": "SINGLE_QUOTES", "escapeCharacter" : "-" }

Rules and guidelines for writing to a flat file target

Consider the following rules when you run a mapping to write data to a flat file in the native environment or on the Spark engine:
  • For a mapping that runs on the Spark engine, the first row of the header in the flat file contains an additional
    #
    symbol. For example, the header
    ID_Char
    appears as
    #ID_Char
    .
  • For a mapping that runs in the native environment, the header in the flat file does not contain an additional
    #
    symbol. For example, the header
    ID_Char
    appears as
    ID_Char
    .
  • For a mapping that runs on the Spark engine, the double quotes text qualifier is honored. For example, the column name
    abcd
    with the double quotes text qualifier appears as
    "abcd"
    .
  • For a mapping that runs in the native environment, the double quotes text qualifier is not honored. For example, the column name
    abcd
    with the double quotes text qualifier appears as
    abcd
    .