Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Preface
  2. Working with Transformations
  3. Address Validator Transformation
  4. Aggregator Transformation
  5. Association Transformation
  6. Bad Record Exception Transformation
  7. Case Converter Transformation
  8. Classifier Transformation
  9. Cleanse transformation
  10. Comparison Transformation
  11. Custom Transformation
  12. Custom Transformation Functions
  13. Consolidation Transformation
  14. Data Masking Transformation
  15. Data Masking Examples
  16. Decision Transformation
  17. Duplicate Record Exception Transformation
  18. Dynamic Lookup Cache
  19. Expression Transformation
  20. External Procedure Transformation
  21. Filter Transformation
  22. HTTP Transformation
  23. Identity Resolution Transformation
  24. Java Transformation
  25. Java Transformation API Reference
  26. Java Expressions
  27. Java Transformation Example
  28. Joiner Transformation
  29. Key Generator Transformation
  30. Labeler Transformation
  31. Lookup Transformation
  32. Lookup Caches
  33. Match Transformation
  34. Match Transformations in Field Analysis
  35. Match Transformations in Identity Analysis
  36. Merge Transformation
  37. Normalizer Transformation
  38. Parser Transformation
  39. Rank Transformation
  40. Router Transformation
  41. Sequence Generator Transformation
  42. Sorter Transformation
  43. Source Qualifier Transformation
  44. SQL Transformation
  45. Using the SQL Transformation in a Mapping
  46. Stored Procedure Transformation
  47. Standardizer Transformation
  48. Transaction Control Transformation
  49. Union Transformation
  50. Unstructured Data Transformation
  51. Update Strategy Transformation
  52. Weighted Average Transformation
  53. XML Transformations

Transformation Guide

Transformation Guide

Using Sorted Input

Using Sorted Input

You can improve Aggregator transformation performance by using the sorted input option. When you use sorted input, the Integration Service assumes all data is sorted by group and it performs aggregate calculations as it reads rows for a group. When necessary, it stores group information in memory. To use the Sorted Input option, you must pass sorted data to the Aggregator transformation. You can gain performance with sorted ports when you configure the session with multiple partitions.
When you do not use sorted input, the Integration Service performs aggregate calculations as it reads. Since the data is not sorted, the Integration Service stores data for each group until it reads the entire source to ensure all aggregate calculations are accurate.
For example, one Aggregator transformation has the STORE_ID and ITEM group by ports, with the sorted input option selected. When you pass the following data through the Aggregator, the Integration Service performs an aggregation for the three rows in the 101/battery group as soon as it finds the new group, 201/battery:
STORE_ID
ITEM
QTY
PRICE
101
'battery'
3
2.99
101
'battery'
1
3.19
101
'battery'
2
2.59
201
'battery'
4
1.59
201
'battery'
1
1.99
If you use sorted input and do not presort data correctly, you receive unexpected results.

0 COMMENTS

We’d like to hear from you!