Table of Contents

Search

  1. Preface
  2. Transformations
  3. Source transformation
  4. Target transformation
  5. Aggregator transformation
  6. Cleanse transformation
  7. Data Masking transformation
  8. Data Services transformation
  9. Deduplicate transformation
  10. Expression transformation
  11. Filter transformation
  12. Hierarchy Builder transformation
  13. Hierarchy Parser transformation
  14. Hierarchy Processor transformation
  15. Input transformation
  16. Java transformation
  17. Java transformation API reference
  18. Joiner transformation
  19. Labeler transformation
  20. Lookup transformation
  21. Machine Learning transformation
  22. Mapplet transformation
  23. Normalizer transformation
  24. Output transformation
  25. Parse transformation
  26. Python transformation
  27. Rank transformation
  28. Router transformation
  29. Rule Specification transformation
  30. Sequence Generator transformation
  31. Sorter transformation
  32. SQL transformation
  33. Structure Parser transformation
  34. Transaction Control transformation
  35. Union transformation
  36. Velocity transformation
  37. Verifier transformation
  38. Web Services transformation

Transformations

Transformations

Group by fields

Group by fields

Use group by fields to define how to group data for aggregate expressions. Configure group by fields on the
Group By
tab of the
Properties
panel.
To define a group for the aggregate expression, select the appropriate input, input/output, and output fields in the Aggregator transformation. You can select multiple group by fields to create a new group for each unique combination.
Data Integration
then performs the defined aggregation for each group.
When you group values,
Data Integration
produces one row for each group. If you do not group values,
Data Integration
returns one row for all input rows.
If the Aggregator transformation runs on an
advanced cluster
and the input and output fields aren't grouped by field, the transformation might not return the last row of each group with the result of the aggregation.
When you select multiple group by fields in the Aggregator transformation,
Data Integration
uses field order to determine the order by which it groups. The group order can affect the results. Order the group by fields to ensure the appropriate grouping. You can change the field order after you select the fields in the group.
For example, you create aggregate fields called TOTAL_QTY and TOTAL_PRICE to store the total quantity and total price for each item by store. You define the following expressions for each field:
  • TOTAL_QTY:
    SUM (QTY)
  • TOTAL_PRICE:
    SUM (QTY*PRICE)
You define STORE_ID and ITEM as the group by fields.
The input rows contain the following data:
STORE_ID
ITEM
QTY
PRICE
101
'battery'
3
2.99
101
'battery'
1
3.19
101
'battery'
2
2.59
101
'AAA'
2
2.45
201
'battery'
1
1.99
201
'battery'
4
1.59
301
'battery'
1
2.45
Data Integration
performs the aggregate calculations on the following unique groups:
STORE_ID
ITEM
101
'battery'
101
'AAA'
201
'battery'
301
'battery'
Data Integration
returns the store ID, item name, total quantity for each item by store, and total price for each item by store:
STORE_ID
ITEM
TOTAL_QTY
TOTAL_PRICE
101
'AAA'
2
4.90
101
'battery'
6
17.34
201
'battery'
5
8.35
301
'battery'
1
2.45

0 COMMENTS

We’d like to hear from you!