Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Optimizing the Cache Size

Optimizing the Cache Size

For optimal session performance, configure the cache size so that the Integration Service processes the transformation in memory without paging to disk. Session performance decreases when the Integration Service pages to disk.
When you use the cache calculator to calculate the cache size, the cache calculator estimates the cache size required for optimal session performance based on your input. You can tune the estimate by using the cache size specified in the session log. After you run the session, review the transformation statistics in the session log to get the cache size.
For example, you run an Aggregator transformation called AGGTRANS. The session log contains the following text:
MAPPING> TT_11031 Transformation [AGGTRANS]: MAPPING> TT_11114 [AGGTRANS]: Input Group Index = [0], Input Row Count [110264] MAPPING> TT_11034 [SQ_V_PETL]: Input - 110264 MAPPING> TT_11115 [AGGTRANS]: Output Group Index = [0] MAPPING> TT_11037 [FILTRANS]: Output - 1098,Dropped - 0 MAPPING> CMN_1791 The index cache size that would hold [1098] aggregate groups of input rows for [AGGTRANS], in memory, is [286720] bytes MAPPING> CMN_1790 The data cache size that would hold [1098] aggregate groups of input rows for [AGGTRANS], in memory, is [1774368] bytes
The log shows that the index cache requires 286,720 bytes and the data cache requires 1,774,368 bytes to process the transformation in memory without paging to disk.
The cache size may vary depending on changes to the session or source data. Review the session logs after subsequent session runs to monitor changes to the cache size.
You must set the tracing level to Verbose Initialization in the session properties to enable the Integration Service to write the transformation statistics to the session log.
The session log does not contain transformation statistics for a Sorter, a Joiner transformation with sorted input, an Aggregator transformation with sorted input, or an XML target.

0 COMMENTS

We’d like to hear from you!