Microsoft Azure Data Lake Storage Gen2 Connector

Back Next

Fixed-width file formats

You can use a fixed-width flat file as a source

or target

in mappings and mapping tasks.

When you configure a Source transformation

or Target transformation

and select the fixed-width flat file type, you must select the most appropriate fixed-width file format to use based on the data in the fixed-width flat file. Ensure that the sample flat file only uses UTF-8 character set encoding.

Consider the following rules and guidelines for a fixed-width flat file:

You cannot use a fixed-width flat file as a source or target for mappings in advanced mode and data transfer tasks.

When you select a fixed-width file format in the target transformation to create a flat file target at runtime, the Secure Agent ignores the fixed-width column boundaries for the target and considers the fixed-width column boundaries specified for the source.

When you write a column of Numeric data type from fixed-width flat file source to an empty fixed-width flat file target that uses the same fixed-width file format as the source file, the Secure Agent appends a null character to the value in Numeric column in the target.

When you create the fixed-width file format, ensure that the sample file uses the following character as the new line symbol, based on the operating system where the Secure Agent is installed:

For Linux, use

character.

For Windows, use

\r\n

character.

The source file must also use the same character as defined in the sample file.

When you use a fixed-width flat file as a source

or target

, you cannot edit the metadata for the fields.

When you read data of the date data type, you can read the date only up to milliseconds.

When you read data of the double data type from a fixed-width file and write the data to a Parquet or Avro file, the double data type is mapped to the decimal data type in target. Hence, the data is written incorrectly.

To write the data correctly to the target, edit the metadata in the Target transformation and change the decimal data type to double.

When you read from a fixed-width file with a header and write to a fixed-width file and map the FileName field, the header is considered as part of the data. To skip the header, set the value of the Number of rows to skip
field to 1 when you create the fixed-width file format.

When you write data to a fixed-width file, you cannot append data to an existing file.

Rename Saved Search

Table of Contents

Microsoft Azure Data Lake Storage Gen2 Connector

Microsoft Azure Data Lake Storage Gen2 Connector

Fixed-width file formats

Fixed-width file formats