Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Session Caches Overview

Session Caches Overview

The Integration Service allocates cache memory for XML targets and Aggregator, Joiner, Lookup, Rank, and Sorter transformations in a mapping. The Integration Service creates index and data caches for the XML targets and Aggregator, Joiner, Lookup, and Rank transformations. The Integration Service stores key values in the index cache and output values in the data cache. The Integration Service creates one cache for the Sorter transformation to store sort keys and the data to be sorted.
You configure memory parameters for the caches in the session properties. When you first configure the cache size, you can calculate the amount of memory required to process the transformation or you can configure the Integration Service to automatically configure the memory requirements at run time.
After you run a session, you can tune the cache sizes for the transformations in the session. You can analyze the transformation statistics to determine the cache sizes required for optimal session performance, and then update the configured cache sizes.
If the Integration Service requires more memory than what you configure, it stores overflow values in cache files. When the session completes, the Integration Service releases cache memory, and in most circumstances, it deletes the cache files.
If the session contains multiple partitions, the Integration Service creates one memory cache for each partition. In particular situations, the Integration Service uses cache partitioning, creating a separate cache for each partition.
The following table describes the type of information that the Integration Service stores in each cache:
Mapping Object
Cache Types and Descriptions
Aggregator
  • Index. Stores group values as configured in the group by ports.
  • Data. Stores calculations based on the group by ports.
Joiner
  • Index. Stores all master rows in the join condition that have unique keys.
  • Data. Stores master source rows.
Lookup
  • Index. Stores lookup condition information.
  • Data. Stores lookup data that is not stored in the index cache.
Rank
  • Index. Stores group values as configured in the group by ports.
  • Data. Stores ranking information based on the group by ports.
Sorter
  • Sorter. Stores sort keys and data.
XML Target
  • Index. Stores primary and foreign key information in separate caches.
  • Data. Stores XML row data while it generates the XML target.


Updated August 27, 2020