Table of Contents

Search

  1. Abstract
  2. Supported Versions
  3. Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Aggregator Transformation

Aggregator Transformation

Disable map-side aggregation if a unique key is used as the group by key in an Aggregator transformation.
In the Blaze engine, map-side aggregation is analogous to the aggregation done in the map phase in a MapReduce job that runs on the Hive engine. Source data is aggregated based on the group-by port set in an Aggregator transformation. The aggregated data then moves to the data shuffle stage for the second level of aggregation.
If you specify a unique key as the group by port, disable the map-side aggregation in a mapping that runs on the Blaze engine.
In the Developer tool, set the following run-time property in the Run-time tab to disable map-side aggregation for the mapping:
Parameter
Value
GridExecutor.EnableMapSideAgg
False

0 COMMENTS

We’d like to hear from you!