Operator Guide

10.4.0
- 10.5
- 10.4.1

Back Next

Automatic Flat File Publication

Use this type of publication to publish data from a flat file source if you want

Data Integration Hub

to create the PowerCenter workflow for the publication.

For automatic publications with a flat file source, define the location of the source files, and then configure flat file sources for the tables in the topic that is associated with the publication. If you are publishing from a Hadoop Distributed File System (HDFS), select the HDFS connection.

If you use the file transfer protocol to move the files, select the file transfer connection.

You must associate at least one topic table with a source file.

Data Integration Hub

maps the source fields to topic fields, based on a name match. You can edit the mapping and manually map source fields to topic fields.

Data Integration Hub

creates the workflow with a mapping, based on the topic tables that are associated with a file, and runs the workflow during the publication process. Topic tables that are not associated with a file are not mapped.

Data Integration Hub

deletes the files after it reads them.

When you use file transfer, you can select whether or not

Data Integration Hub

deletes the files from the remote server after it reads them.

When you publish files from HDFS,

Data Integration Hub

does not delete the files after it reads them. If required, you must delete the files yourself.

The following pages appear in the publication wizard for this type of publication:

If you want to run a pre-process on the publication, you might be required to set pre-processing settings. Before you start creating the publication, get the required parameter settings.

General. Define basic publication properties.

Processing. If you want to run a pre-process on the publication, select a publication pre-processing workflow. If the pre-process includes parameters, the parameters appear in this page. The content of the page depends on the workflow parameters that the developer defines. The developer imports the workflow to

Data Integration Hub

with the parameter definitions. The developer also determines the layout of this page. If required, set the values of the parameters.

Source. Choose the source type. For an HDFS source or when you use file transfer, select the connection to the source from which

Data Integration Hub

reads the files. Enter the location of the file or files from where

Data Integration Hub

reads the data, and configure the format and the structure of flat file sources for the topic tables. You must configure a file source for at least one topic table.

Join. You can pull data from multiple source files into a single topic table by creating joins.

You can create multiple joins, and you can combine data from joins into new joins. Joins are virtual entities, and are not created in the topic.

Field Mapping. Review the publication field mapping, and if required, edit the mapping that

Data Integration Hub

generates by default. You can manually map source fields to topic fields.

Filter. Define the data that the publication publishes by setting filter conditions on source columns.

Schedule. Define the method and the frequency of data publishing. You can select to publish the data immediately after the published files are ready, to run the publication manually or by an external trigger, or to publish the data according to a schedule that you define.

If you select to publish the data immediately after the published files are ready, define the maximal period of time that

Data Integration Hub

waits before it publishes available files. When the maximal period of time ends,

Data Integration Hub

runs the publication even if not all the files are ready to be published.

Summary. Review the publication settings and save the publication.

Publication Types

Download Guide

Watch

Comments

Communities

Knowledge Base