Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Configuring the Cache Sizes for a Joiner Transformation

Configuring the Cache Sizes for a Joiner Transformation

You can configure the index and data cache sizes for a Joiner transformation session properties.
When you use 1:
n
partitioning, the Integration Service replicates the memory cache for each partition. Each partition requires as much memory as the total requirements for the transformation. When you configure the cache size for the Joiner transformation with 1:
n
partitioning, set the cache size to the total requirements for the transformation.
When you use
n
:
n
partitioning, each partition requires a portion of the total memory required to process the transformation. When you configure the cache size for the Joiner transformation with
n
:
n
partitioning, calculate the total requirements for the transformation, and then divide it by the number of partitions.
You can use the cache calculator to determine the cache size required to process the transformation. For example, you use the cache calculator to determine that the Joiner transformation requires 2,000,000 bytes of memory for the index cache and 4,000,000 bytes of memory for the data cache. You create four partitions for the pipeline. If you use 1:
n
partitioning, configure 2,000,000 bytes for the index cache and 4,000,000 bytes for the data cache. If you use
n
:
n
partitioning, configure 500,000 bytes for the index cache and 1,000,000 bytes for the data cache.
The following table describes the input you provide to calculate the Joiner cache sizes:
Input
Description
Number of Master Rows
Number of rows in the master source. Applies to a Joiner transformation with unsorted input. The number of master rows does not affect the cache size for a sorted Joiner transformation.
If rows in the master source share unique keys, the cache calculator overestimates the index cache size.
Data Movement Mode
The data movement mode of the Integration Service. The cache requirement varies based on the data movement mode. ASCII characters use one byte. Unicode characters use two bytes.
Enter the input and then click Calculate to calculate the data and index cache sizes. The calculated values appear in the Data Cache Size and Index Cache Size fields.


Updated June 25, 2020