Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Running Sessions on a Grid

Running Sessions on a Grid

When you run a session on a grid, the master service process runs the workflow and all tasks except Session, Command, and predefined Event-Wait tasks as it does when you run a workflow on a grid. The Scheduler runs on the master service process node, so it uses the date and time for the master service process node to start scheduled workflows. In addition, the Load Balancer distributes session threads to DTM processes running on different nodes.
When you run a session on a grid, the Load Balancer distributes session threads based on the following factors:
  • Node availability
    . The Load Balancer verifies which nodes are currently running, enabled, and available for task dispatch.
  • Resource availability
    . If the Integration Service is configured to check resources, it identifies nodes that have resources required by mapping objects in the session.
  • Partitioning configuration.
    The Load Balancer dispatches groups of session threads to separate nodes based on the partitioning configuration.
You might want to configure a session to run on a grid when the workflow contains a session that takes a long time to run.
For example, a workflow contains a session with one partition. To balance the load, you configure the session to run on a grid and configure the Integration Service to check resources. The Load Balancer distributes the reader, writer, and transformation threads to DTM processes running on the nodes in the grid. The reader threads require a resource, so the Load Balancer distributes them to a DTM process on the node where resources are available.
The following image shows session threads distributed to DTM processes running on nodes in a grid:
The workflow has a session and a command task. The threads for the session and the command task are dispatched to nodes with available resources.
  1. Reader threads run on node where resources are available.
  2. Transformation threads run on available node.
  3. Writer threads run on available node.
  4. Command task runs on available node.

0 COMMENTS

We’d like to hear from you!