Bulk Data Movement Guide

10.5.4
- 10.5.6
- 10.5.5
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0 HotFix 1
- 10.4.0

Back Next

Variable-Length Stream Data

In rare situations where the record boundaries of variable-length records cannot be established from the operating system APIs, you can select

Variable

for the record format and

(variable-length stream) as the variable-length data file type.

Variable-length stream data processing works as follows:

PowerExchange reads a block of data that is large enough to hold the largest record.

PowerExchange tries to map the data onto each record type in the sequence that the record types are defined in the data map.

When a match is found, that record type is used and the pointer within the buffer is moved the required number of bytes.

Informatica recommend that you use variable-length stream data processing only as a last resort because of the following restrictions:

Variable-length stream data processing does not scale well if many record types are used.

You must define all record types before attempting to read the file. As a result, defining the data map is time-consuming.

User error is possible if the records are not sequenced correctly. Record ID conditions are based on the sequence of records in the data map, and PowerExchange does not validate sequencing to prevent user errors.

Considerable CPU time is consumed, because most record types get fully mapped before PowerExchange determines whether to select this record.

Although writing is allowed using a variable-length stream data map, such maps are intended solely for reading. When writing, PowerExchange uses the default record format.

In a mapping that contains user sequence fields, you might be able to write successfully using a variable-length stream data map on z/OS and i5/OS operating systems, which do not add record terminators. But writing might result in unwanted record terminator characters on Linux, UNIX, and Windows.

Rename Saved Search

Table of Contents

Bulk Data Movement Guide

Bulk Data Movement Guide

Variable-Length Stream Data

Variable-Length Stream Data