Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Partitioning File Sources

Partitioning File Sources

When a session uses a file source, you can configure it to read the source with one thread or with multiple threads. The Integration Service creates one connection to the file source when you configure the session to read with one thread, and it creates multiple concurrent connections to the file source when you configure the session to read with multiple threads.
Use the following types of partitioned file sources:
  • Flat file.
    You can configure a session to read flat file, XML, or COBOL source files.
  • Command.
    You can configure a session to use an operating system command to generate source data rows or generate a file list.
When connecting to file sources, you must choose the same connection type for all partitions. You may choose different connection objects as long as each object is of the same type.
To specify single- or multi-threaded reading for flat file sources, configure the source file name property for partitions 2-
n
. To configure for single-threaded reading, pass empty data through partitions 2-
n
. To configure for multi-threaded reading, leave the source file name blank for partitions 2-
n
.

0 COMMENTS

We’d like to hear from you!