Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Adding and Deleting Partition Points

Adding and Deleting Partition Points

Partition points mark the thread boundaries in a pipeline and divide the pipeline into stages.
When you add partition points, you increase the number of transformation threads, which can increase session performance. The Integration Service can redistribute rows of data at partition points, which can also increase session performance. When you create a session, the Workflow Manager creates one partition point at each transformation in the pipeline.
You can retain or delete the partition points based on the following transformations or target instances in the pipeline:
Source Qualifier Transformation
Controls how the Integration Service extracts data from the source and passes it to the source qualifier. You cannot delete this partition point.
Normalizer Transformation
Controls how the Integration Service extracts data from the source and passes it to the source qualifier. You cannot delete this partition point.
Rank Transformation
Ensures that the Integration Service groups rows properly before it sends them to the transformation. You can delete these partition points if the pipeline contains only one partition or if the Integration Service passes all rows in a group to a single partition before they enter the transformation.
Unsorted Aggregator Transformation
Ensures that the Integration Service groups rows properly before it sends them to the transformation. You can delete these partition points if the pipeline contains only one partition or if the Integration Service passes all rows in a group to a single partition before they enter the transformation.
Target Instances
Controls how the writer passes data to the targets. You cannot delete this partition point.
Multiple Input Group Transformation
The Workflow Manager creates a partition point at a multiple input group transformation when it is configured to process each partition with one thread, or when a downstream multiple input group Custom transformation is configured to process each partition with one thread.
For example, the Workflow Manager can create a partition point at a sorted Joiner transformation. The Workflow Manager creates a partition point when you connect the Joiner transformation to a downstream Custom transformation configured to use one thread per partition.
This ensures that the Integration Service uses one thread to process each partition at a Custom transformation that requires one thread per partition. You cannot delete this partition point.

0 COMMENTS

We’d like to hear from you!