User Guide

10.2.2
- 10.5.4
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0
- 10.2.2 Service Pack 1
- 10.2.1

Back Next

Temporary Directory for the Complex File Target

At run time, the Data Integration Service creates a temporary directory in the location that you set for the

File Directory

property that you configure for the complex file targets in a streaming mapping. A temporary directory is created for each mapping. A temporary directory contains the target files to which the data is currently written.

Based on the rollover limit, data is written to the target file in the temporary directory. After a target file reaches the specified rollover limit, the target file is closed and moved to the specified target file directory from the temporary directory.

For example, if you set the

File Directory

property to

hdfs://demo.domain.com:8020/tmp/batch/

, you can find the target file to which data is currently written in the following directory:

hdfs://demo.domain.com:8020/tmp/batch/active/586bdfd67170008d125f52f68c1f02/

. The

hdfs://demo.domain.com:8020/tmp/batch/

directory contains the target files that are moved after the rollover limit is reached.

If a mapping fails, all the target files are closed and moved to the specified target file directory from the temporary directory, and the temporary directory will be deleted. In the preceding example, the temporary directory

586bdfd67170008d125f52f68c1f02

will be deleted.

If you use Big Data Management to process batch data, you can use the batch complex files that are created in the file directory as the source objects in Big Data Management.

Rename Saved Search

Table of Contents

User Guide

User Guide

Temporary Directory for the Complex File Target

Temporary Directory for the Complex File Target