Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Partitioning a Source Qualifier with Multiple Sources

Partitioning a Source Qualifier with Multiple Sources

A relational source qualifier can receive data from multiple source tables. The Integration Service creates SQL queries for database partitions based on the number of partitions in the database table with the most partitions. It creates an SQL join condition to retrieve the data from the database partitions.
For example, a source qualifier receives data from two source tables. Each source table has two partitions. If the session has three partitions and the database table has two partitions, one of the session partitions receives no data.
The Integration Service generates the following SQL statements for Oracle:
Session Partition 1: SELECT <column list> FROM t1 PARTITION (p1), t2 WHERE <join clause> Session Partition 2: SELECT <column list> FROM t1 PARTITION (p2), t2 WHERE <join clause> Session Partition 3: No SQL query.
The Integration Service generates the following SQL statements for IBM DB2:
Session Partition 1: SELECT <column list> FROM t1,t2 WHERE ((nodenumber(t1 column1)=0) AND <join clause> Session Partition 2: SELECT <column list> FROM t1,t2 WHERE ((nodenumber(t1 column1)=1) AND <join clause> Session Partition 3: No SQL query.


Updated June 25, 2020