Your organization needs to denormalize employee ID, name, and address details. The employee ID, name, and address details are stored in flat files in HDFS or ViewFS. Create a mapping that reads all the employee ID, name, and address details from the flat files in HDFS. The mapping must convert the denormalized data to hierarchical data and write it to a complex file target in HDFS or ViewFS.

You can use the target data for business analytics.

The following figure shows the example mapping:

You can use the following objects in the HDFS or ViewFS mapping:

HDFS Inputs

The inputs, Read_Address_Flat_File and Read_Name_Flat_File, are flat files stored in HDFS or ViewFS.

Data Processor Transformation

The Data Processor transformation, JSON_R2H_Denormalize_NameAndAddress, reads the flat files, denormalizes the data, and provides a binary, hierarchical output.

HDFS Output

The output, Write_Complex_File, is a complex file stored in HDFS or ViewFS.

When you run the mapping, the Data Integration Service reads the input flat files and passes the employee ID, name, and address data to the Data Processor transformation. The Data Processor transformation denormalizes the employee ID, name, and address data, and provides a hierarchical output in a binary stream. The binary and hierarchical output is written to the HDFS or ViewFS complex file target.

You can configure the mapping to run in a native or Hadoop run-time environment.

Complete the following tasks to configure the mapping:

Create an HDFS or ViewFS connection to read flat files from the Hadoop cluster.
Specify the read properties for the flat files.
Drag and drop the flat files into a mapping.
Create a Data Processor transformation. Set the Data Processor transformation port to binary.
Create an HDFS or ViewFS connection to write data to the complex file target.
Create a complex file data object write operation. Specify the following parameters:
The file as the resource in the data object.
The HDFS or ViewFS file location.
Drag and drop the complex file data object write operation into the mapping.