Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Configuring Sessions to Use a Single Thread

Configuring Sessions to Use a Single Thread

To configure a session to read a file with a single thread, pass empty data through partitions 2-
n.
To pass empty data, create a file with no data, such as “empty.txt,” and put it in the source file directory. Then, use “empty.txt” as the source file name.
You cannot configure single-threaded reading for partitioned sources that use a command to generate source data.
The following table shows the source file name and values when the Integration Service creates one thread to read ProductsA.txt. It reads rows in the file sequentially. After it reads the file, it passes the data to three partitions in the transformation pipeline:
Source File Name
Value
Partition #1
ProductsA.txt
Partition #2
empty.txt
Partition #3
empty.txt
The following table shows the source file name and values when the Integration Service creates two threads. It creates one thread to read ProductsA.txt, and it creates one thread to read ProductsB.txt. It reads the files concurrently, and it reads rows in the files sequentially:
Source File Name
Value
Partition #1
ProductsA.txt
Partition #2
empty.txt
Partition #3
ProductsB.txt
If you use FTP to access source files, you can choose a different connection for each direct file.

0 COMMENTS

We’d like to hear from you!