Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Partitioning FTP File Targets

Partitioning FTP File Targets

When you choose an FTP connection type for the partitioned targets in a session, you configure FTP settings for the target partitions.
You can merge the target files or individual target files for each partition.
Use the following rules and guidelines when you configure FTP settings for target partitions:
  • You must use an FTP connection for each target partition.
  • You can choose to stage the files when selecting the connection object for the target partition. You must stage the files to use sequential merge.
  • If the FTP connections for the target partitions have any settings other than a remote file name, the Integration Service does not create a merge file.
The following table describes the actions that Integration Service completes for partitioned FTP file targets:
Merge Type
Integration Service Behavior
No Merge
The Integration Service generates one target file for each partition. If you stage the files, the Integration Service transfers the target files to the remote location at the end of the session. If you do not stage the files, the Integration Service generates the target files at the remote location.
Sequential Merge
Enables the
Is Staged
option in the connection object. The Integration Service creates one output file for each partition. At the end of the session, the Integration Service completes the following actions:
  1. Merges the individual output files into a merge file.
  2. Deletes the individual output files.
  3. Transfers the merge file to the remote location.
File List
If you stage the files, the Integration Service creates the following files:
  • Output file for each partition
  • File list that contains the names and paths of the local files
  • File list that contains the names and paths of the remote files
At the end of the session, the Integration Service transfers the files to the remote location. If the individual target files are in the Merge File Directory, file list contains relative paths. Otherwise, the file list contains absolute paths.
If you do not stage the files, the Integration Service writes the data for each partition at the remote location and creates a remote file list that contains a list of the individual target files.
Use the file list as a source file in another mapping.
Concurrent Merge
If you stage the files, the Integration Service concurrently writes the data for all target partitions to a local merge file. At the end of the session, the Integration Service transfers the merge file to the remote location. The Integration Service does not write to any intermediate output files.
If you do not stage the files, the Integration Service concurrently writes the target data for all partitions to a merge file at the remote location.

0 COMMENTS

We’d like to hear from you!