User Guide

Back Next

Consolidation Job

Use the consolidation job to consolidate the linked data and create a preferred record for each cluster in HDFS. The consolidation job uses the output files of an initial clustering job in HDFS as input. The consolidation job creates the preferred records based on the rules defined in the consolidation rules file.

For incremental data, use the output files of an initial clustering job that you run in the incremental mode with the

--consolidate

option as input for the consolidation job.

The consolidation process ignores any null values.

The following image shows how the consolidation job processes the input data: