Table of Contents

Search

  1. Preface
  2. Transformations
  3. Source transformation
  4. Target transformation
  5. Access Policy transformation
  6. B2B transformation
  7. Aggregator transformation
  8. Cleanse transformation
  9. Data Masking transformation
  10. Data Services transformation
  11. Deduplicate transformation
  12. Expression transformation
  13. Filter transformation
  14. Hierarchy Builder transformation
  15. Hierarchy Parser transformation
  16. Hierarchy Processor transformation
  17. Input transformation
  18. Java transformation
  19. Java transformation API reference
  20. Joiner transformation
  21. Labeler transformation
  22. Lookup transformation
  23. Machine Learning transformation
  24. Mapplet transformation
  25. Normalizer transformation
  26. Output transformation
  27. Parse transformation
  28. Python transformation
  29. Rank transformation
  30. Router transformation
  31. Rule Specification transformation
  32. Sequence Generator transformation
  33. Sorter transformation
  34. SQL transformation
  35. Structure Parser transformation
  36. Transaction Control transformation
  37. Union transformation
  38. Velocity transformation
  39. Verifier transformation
  40. Web Services transformation

Transformations

Transformations

Group by fields

Group by fields

Use group by fields to define how to group data for aggregate expressions. Configure group by fields on the
Group By
tab of the
Properties
panel.
To define a group for the aggregate expression, select the appropriate input, input/output, and output fields in the Aggregator transformation. You can select multiple group by fields to create a new group for each unique combination.
Data Integration
then performs the defined aggregation for each group.
When you group values,
Data Integration
produces one row for each group. If you do not group values,
Data Integration
returns one row for all input rows.
If the Aggregator transformation runs on an
advanced cluster
and the input and output fields aren't grouped by field, the transformation might not return the last row of each group with the result of the aggregation.
When you select multiple group by fields in the Aggregator transformation,
Data Integration
uses field order to determine the order by which it groups. The group order can affect the results. Order the group by fields to ensure the appropriate grouping. You can change the field order after you select the fields in the group.
For example, you create aggregate fields called TOTAL_QTY and TOTAL_PRICE to store the total quantity and total price for each item by store. You define the following expressions for each field:
  • TOTAL_QTY:
    SUM (QTY)
  • TOTAL_PRICE:
    SUM (QTY*PRICE)
You define STORE_ID and ITEM as the group by fields.
The input rows contain the following data:
STORE_ID
ITEM
QTY
PRICE
101
'battery'
3
2.99
101
'battery'
1
3.19
101
'battery'
2
2.59
101
'AAA'
2
2.45
201
'battery'
1
1.99
201
'battery'
4
1.59
301
'battery'
1
2.45
Data Integration
performs the aggregate calculations on the following unique groups:
STORE_ID
ITEM
101
'battery'
101
'AAA'
201
'battery'
301
'battery'
Data Integration
returns the store ID, item name, total quantity for each item by store, and total price for each item by store:
STORE_ID
ITEM
TOTAL_QTY
TOTAL_PRICE
101
'AAA'
2
4.90
101
'battery'
6
17.34
201
'battery'
5
8.35
301
'battery'
1
2.45

0 COMMENTS

We’d like to hear from you!