Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Rules and Guidelines for Configuring a Workflow or Session to Run on a Grid

Rules and Guidelines for Configuring a Workflow or Session to Run on a Grid

Use the following rules and guidelines when you configure a session or workflow to run on a grid:
  • To run sessions over the grid, verify that the operating system and bit mode is the same for each node of the grid. A session might not run on the grid if the nodes run on different operating systems or bit modes.
  • If you override a service process variable, ensure that the Integration Service can access input files, caches, logs, storage and temporary directories, and source and target file directories.
  • To ensure that a Session, Command, or predefined Event-Wait task runs on a particular node, configure the Integration Service to check resources and specify a resource requirement for a the task.
  • To ensure that session threads for a mapping object run on a particular node, configure the Integration Service to check resources and specify a resource requirement for the object.
  • When you run a session that creates cache files, configure the root and cache directory to use a shared location to ensure consistency between cache files.
  • Ensure the Integration Service builds the cache in a shared location when you add a partition point at a Joiner transformation and the transformation is configured for 1:
    n
    partitioning. The cache for the Detail pipeline must be shared.
  • Ensure the Integration Service builds the cache in a shared location when you add a partition point at a Lookup transformation, and the partition type is not hash auto-keys.
  • When you run a session that uses dynamic partitioning, and you want to distribute session threads across all nodes in the grid, configure dynamic partitioning for the session to use the “Based on number of nodes in the grid” method.
  • You cannot run a debug session on a grid.
  • You cannot configure a resume recovery strategy for a session that you run on a grid.
  • Configure the session to run on a grid when you work with sessions that take a long time to run.
  • Configure the workflow to run on a grid when you have multiple concurrent sessions.
  • You can run a persistent profile session on a grid, but you cannot run a temporary profile session on a grid.
  • When you use a Sequence Generator transformation, increase the number of cached values to reduce the communication required between the master and worker DTM processes and the repository.
  • To ensure that the Log Viewer can accurately order log events when you run a workflow or session on a grid, use time synchronization software to ensure that the nodes of a grid use a synchronized date/time.
  • If the workflow uses an Email task in a Windows environment, configure the same Microsoft Outlook profile on each node to ensure the Email task can run.


Updated November 14, 2019