Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Session Cache Limits

Session Cache Limits

The Integration Service uses the session cache to allocate memory for transformations that have memory cache set to auto mode. You can limit the amount of memory in the session cache. When you limit the session cache, you limit the amount of memory the Integration Service can use for the session so that some memory remain for other processes.
You must specify the cache memory limit both as a numeric value and as a percentage of total memory. The Integration Service bases the percentage value on the total physical memory of the machine where the Integration Service runs.
The Integration Service compares the numeric value and the percentage value to determine which value is lower. It uses the lower value as the total memory to allocate to the session cache.
The following attributes of the session configuration object set limits to the memory cache allocated for the session:
Maximum Memory Allowed for Auto Memory Attributes
Amount of memory to allocate for the session cache. The total auto cache memory cannot exceed the value of this property even if the percentage of the machine memory used is less than the value in the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes property. This situation can occur when a session runs on a machine with a large amount of physical memory.
Maximum Percentage of Total Memory Allowed for Auto Memory Attributes
Percentage of machine memory to allocate for the session cache. The total auto cache memory cannot exceed this percentage even if the value in the Maximum Memory Allowed for Auto Memory Attributes property is higher. This situation can occur when a session runs on a machine with very little physical memory.
The Integration Service allocates memory from the session cache to all transformations with cache memory set to auto. It divides the memory among all transformations caches.
For example, you configure automatic caching for three Lookup transformations in a session. Then, you configure a memory cache limit of 500 MB for the session. When you run the session, the Integration Service divides the 500 MB of allocated memory among the index and data caches for all three Lookup transformations. The memory cache limit for the session does not apply to transformations that you did not configure for automatic caching.
If the session cache is set to automatic allocation, the Integration Service allocates a minimum of 1 MB for the index cache and 2 MB for the data cache of each transformation that is set to automatic cache allocation. If the session cache limit does not provide enough memory for the minimum index and data cache allocation, the Integration Service overrides the cache limit and allocates the minimum amount of memory to the index and data caches.
For example, the session cache is limited to 4 MB of memory and two transformations are set to automatic cache allocation. The Integration Service overrides the session cache limit and allocates the minimum 1 MB for the index cache and 2 MB for the data cache of each transformation that is set to automatic cache allocation. The total amount of memory allocated to the transformation caches is 6 MB.
When you run a session on a grid and you configure Maximum Memory Allowed For Auto Memory Attributes, the Integration Service divides the allocated memory cache among all the nodes in the grid. When you configure Maximum Percentage of Total Memory Allowed For Auto Memory Attributes, the Integration Service allocates the specified percentage of memory cache to each node in the grid.

0 COMMENTS

We’d like to hear from you!