Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Auto Cache Size

Auto Cache Size

By default, the memory cache for a transformation is set to auto mode. The Integration Service automatically allocates cache memory to all transformations that have their caches set to auto mode. You can set the maximum amount of cache memory that the Integration Service can allocate to the transformations.
To set the maximum cache memory for transformations in auto cache mode, configure the following session properties:
Maximum Memory Allowed for Auto Memory Attributes
Maximum amount of memory to allocate for the session cache. The Integration Service allocates memory from the session cache to all transformations with cache memory set to auto. The default unit is bytes. Append KB, MB, or GB to the value to specify other units. For example, 1048576 or 1024 KB or 1 MB.
Maximum Percentage of Total Memory Allowed for Auto Memory Attributes
Percentage of machine memory to allocate for the session cache. The Integration Service allocates memory from the session cache to all transformations with cache memory set to auto.
When you set the maximum cache size for the session, the Integration Service calculates the maximum percentage of memory and compares that against the maximum amount of memory that you specify. Then it allocates the lower amount of memory to transformations in auto cache mode. If multiple transformations are in auto cache mode, the Integration Service allocates the memory to all transformations in auto cache mode.
For example, the machine that hosts the Integration Service has 1 GB of memory. You set the Maximum Memory Allowed for Auto Memory Attributes property to 800 MB. You also set the Maximum Percentage of Total Memory Allowed for Auto Memory Attributes property to 10%. The Integration Service allocates 102.4 MB of memory to the session cache and divides the cache memory among all transformations in auto cache mode.
The maximum session cache size you set affects only transformations with cache mode set to auto. The Integration Service allocates memory separately to transformations for which you configure a specific cache size.
If a session has multiple transformations that require caching, you can set the cache mode for some transformations to auto and specify a cache size for other transformations. The Integration Service allocates the memory specified for transformations in auto cache mode in addition to the memory it allocates to transformations configured with numeric cache sizes.
For example, a session has three transformations that require caching. You set two transformations to auto cache mode and specify a maximum memory cache size of 800 MB for the session. You also specify a cache size of 500 MB for the third transformation. The Integration Service allocates a total of 1,300 MB of memory.
If the Integration Service uses cache partitioning, the Integration Service distributes the maximum cache size specified for the auto cache memory across all transformations in the session and divides the cache memory for each transformation among all of its partitions.


Updated November 14, 2019