Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Resuming Sessions

Resuming Sessions

When you configure session recovery to resume from the last checkpoint, the Integration Service creates checkpoints in $PMStorageDir to determine where to start processing session recovery. When the Integration Service resumes a session, it restores the session state of operation, including the state of each source, target, and transformation. The Integration Service determines how much of the source data it needs to process.
When the Integration Service resumes a session, the recovery session must produce the same data as the original session. The session is not valid if you configure recovery to resume from the last checkpoint, but the session cannot produce repeatable data.
The Integration Service can recover flat file sources including FTP sources. It can truncate or append to flat file and FTP targets.
When you recover a session from the last checkpoint, the Integration Service restores the session state of operation to determine the type of recovery it can perform:
  • Incremental
    . The Integration Service starts processing data at the point of interruption. It does not read or transform rows that it processed before the interruption. By default, the Integration Service attempts to perform incremental recovery.
  • Full
    . The Integration Service reads all source rows again and performs all transformation logic if it cannot perform incremental recovery. The Integration Service begins writing to the target at the last commit point. If any session component requires full recovery, the Integration Service performs full recovery on the session.
The following table describes when the Integration Service performs incremental or full recovery, depending on the session configuration:
Component
Incremental Recovery
Full Recovery
Commit type
The session uses a source-based commit. The mapping does not contain any transformation that generates commits.
The session uses a target-based commit or user-defined commit.
Transformation Scope
Transformations propagate transactions and the transformation scope must be Transaction or Row.
At least one transformation is configured with the All transformation scope.
File Source
A file source supports incremental reads.
n/a
FTP Source
The FTP server must support the seek operation to allow incremental reads.
The FTP server does not support the seek operation.
Relational Source
A relational source supports incremental reads when the output is deterministic and repeatable. If the output is not deterministic and repeatable, the Integration Service supports incremental relational source reads by staging SQL results to a storage file.
n/a
VSAM Source
n/a
Integration Service performs full recovery.
XML Source
n/a
Integration Service performs full recovery.
XML Generator Transformation
An XML Generator transformation must be configured with Transaction transformation scope.
n/a
XML Target
An XML target must be configured to generate a new XML document on commit.
n/a

0 COMMENTS

We’d like to hear from you!