Table of Contents

Search

  1. Preface
  2. Transformations
  3. Source transformation
  4. Target transformation
  5. Access Policy transformation
  6. Aggregator transformation
  7. B2B transformation
  8. Chunking transformation
  9. Cleanse transformation
  10. Data Masking transformation
  11. Data Services transformation
  12. Deduplicate transformation
  13. Expression transformation
  14. Filter transformation
  15. Hierarchy Builder transformation
  16. Hierarchy Parser transformation
  17. Hierarchy Processor transformation
  18. Input transformation
  19. Java transformation
  20. Java transformation API reference
  21. Joiner transformation
  22. Labeler transformation
  23. Lookup transformation
  24. Machine Learning transformation
  25. Mapplet transformation
  26. Normalizer transformation
  27. Output transformation
  28. Parse transformation
  29. Python transformation
  30. Rank transformation
  31. Router transformation
  32. Rule Specification transformation
  33. Sequence transformation
  34. Sorter transformation
  35. SQL transformation
  36. Structure Parser transformation
  37. Transaction Control transformation
  38. Union transformation
  39. Vector Embedding transformation
  40. Velocity transformation
  41. Verifier transformation
  42. Web Services transformation

Transformations

Transformations

Group by fields

Group by fields

Use group by fields to define how to group data for aggregate expressions. Configure group by fields on the
Group By
tab of the
Properties
panel.
To define a group for the aggregate expression, select the appropriate input, input/output, and output fields in the Aggregator transformation. You can select multiple group by fields to create a new group for each unique combination.
Data Integration
then performs the defined aggregation for each group.
When you group values,
Data Integration
produces one row for each group. If you do not group values,
Data Integration
returns one row for all input rows.
If the Aggregator transformation runs on an
advanced cluster
and the input and output fields aren't grouped by field, the transformation might not return the last row of each group with the result of the aggregation.
When you select multiple group by fields in the Aggregator transformation,
Data Integration
uses field order to determine the order by which it groups. The group order can affect the results. Order the group by fields to ensure the appropriate grouping. You can change the field order after you select the fields in the group.
For example, you create aggregate fields called TOTAL_QTY and TOTAL_PRICE to store the total quantity and total price for each item by store. You define the following expressions for each field:
  • TOTAL_QTY:
    SUM (QTY)
  • TOTAL_PRICE:
    SUM (QTY*PRICE)
You define STORE_ID and ITEM as the group by fields.
The input rows contain the following data:
STORE_ID
ITEM
QTY
PRICE
101
'battery'
3
2.99
101
'battery'
1
3.19
101
'battery'
2
2.59
101
'AAA'
2
2.45
201
'battery'
1
1.99
201
'battery'
4
1.59
301
'battery'
1
2.45
Data Integration
performs the aggregate calculations on the following unique groups:
STORE_ID
ITEM
101
'battery'
101
'AAA'
201
'battery'
301
'battery'
Data Integration
returns the store ID, item name, total quantity for each item by store, and total price for each item by store:
STORE_ID
ITEM
TOTAL_QTY
TOTAL_PRICE
101
'AAA'
2
4.90
101
'battery'
6
17.34
201
'battery'
5
8.35
301
'battery'
1
2.45

0 COMMENTS

We’d like to hear from you!