Network
Data Engineering
Data Engineering Integration
Enterprise Data Catalog
Enterprise Data Preparation
Cloud Integration
Cloud Application Integration
Cloud Data Integration
Cloud Customer 360
DiscoveryIQ
Cloud Data Wizard
Informatica for AWS
Informatica for Microsoft
Cloud Integration Hub
Complex Event Processing
Proactive Healthcare Decision Management
Proactive Monitoring
Real-Time Alert Manager
Rule Point
Data Integration
B2B Data Exchange
B2B Data Transformation
Data Integration Hub
Data Replication
Data Services
Data Validation Option
Fast Clone
Informatica Platform
Metadata Manager
PowerCenter
PowerCenter Express
PowerExchange
PowerExchange Adapters
Data Quality
Axon Data Governance
Data as a Service
Data Explorer
Data Quality
Data Security Group (Formerly ILM)
Data Archive
Data Centric Security
Secure@Source
Secure Testing
Master Data Management
Identity Resolution
MDM - Relate 360
Multidomain MDM
MDM Registry Edition
Process Automation
ActiveVOS
Process Automation
Product Information Management
Informatica Procurement
MDM - Product 360
Ultra Messaging
Ultra Messaging Options
Ultra Messaging Persistence Edition
Ultra Messaging Queuing Edition
Ultra Messaging Streaming Edition
Edge Data Streaming
Knowledge Base
Resources
PAM (Product Availability Matrices)
Support TV
Velocity (Best Practices)
Mapping Templates
Debugging Tools
User Groups
Documentation
English
English
English
Español
Spanish
Deutsch
German
Français
French
日本語
Japanese
한국어
Korean
Português
Portuguese
中文
Chinese
Log Out
Log In
Sign Up
PowerCenter
10.4.1
H2L
10.4.1
10.4.0
10.2 HotFix 2
10.2 HotFix 1
10.2
10.1.1 HotFix 2
10.1.1 HotFix 1
10.1.1
10.1
10.0
Advanced Workflow Guide
PowerCenter
All Products
Table of Contents
Search
No Results
Preface
Understanding Pipeline Partitioning
Understanding Pipeline Partitioning Overview
Partitioning Attributes
Partition Points
Number of Partitions
Partitioning Multiple Input Group Transformations
Partition Types
Dynamic Partitioning
Configuring Dynamic Partitioning
Rules and Guidelines for Dynamic Partitioning
Using Dynamic Partitioning with Partition Types
Configuring Partition-Level Attributes
Cache Partitioning
Mapping Variables in Partitioned Pipelines
Partitioning Rules
Partition Restrictions for Editing Objects
Before You Create a Session
After You Create a Session with Multiple Partitions
Partition Restrictions for PowerExchange
Configuring Partitioning
Adding Partition Points to a Pipeline
Configuring a Partition Point
Partition Points Node
Edit Partition Point
Edit Partition Key
Non-Partition Points Node
Partition Points
Partition Points Overview
Adding and Deleting Partition Points
Rules and Guidelines for Adding and Deleting Partition Points
Partitioning Relational Sources
Entering an SQL Query
Entering a Filter Condition
Partitioning File Sources
Rules and Guidelines for Partitioning File Sources
Using One Thread to Read a File Source
Using Multiple Threads to Read a File Source
Configuring for File Partitioning
Configuring Sessions to Use a Single Thread
Configuring Sessions to Use Multiple Threads
Configuring Concurrent Read Partitioning
Partitioning Relational Targets
Database Compatibility
Partitioning File Targets
Configuring Connection Settings
Configuring File Properties
Configuring Commands for Partitioned File Targets
Configuring Merge Options
Partitioning Custom Transformations
Working with Multiple Partitions
Creating Partition Points
Working with Threads
One Input Group
Multiple Input Groups
Partitioning Joiner Transformations
Partitioning Sorted Joiner Transformations
Using Sorted Flat Files
Using 1:n Partitions
Using n:n Partitions
Using Sorted Relational Data
Using 1:n Partitions
Using n:n Partitions
Using Sorter Transformations
Optimizing Sorted Joiner Transformations with Partitions
Add a Hash Auto-keys Partition Upstream of the Sort Origin
Use n:n Partitions
Partitioning Lookup Transformations
Cache Partitioning Lookup Transformations
Sharing Partitioned Caches
Partitioning Pipeline Lookup Transformation Cache
Partitioning Sequence Generator Transformations
Partitioning Sorter Transformations
Configuring Sorter Transformation Work Directories
Partitioning XML Generator Transformations
Restrictions for Transformations
Restrictions for Numerical Functions
Partition Types
Partition Types Overview
Setting Partition Types in the Pipeline
Setting Partition Types
Database Partitioning Partition Type
Partitioning Database Sources
Database Partitioning with One Source
Partitioning a Source Qualifier with Multiple Sources
Integration Service Handling with Source Database Partitioning
Rules and Guidelines for Source Database Partitioning
Target Database Partitioning
Rules and Guidelines for Target Database Partitioning
Hash Auto-Keys Partition Type
Hash User Keys Partition Type
Key Range Partition Type
Adding a Partition Key
Adding Key Ranges
Adding Filter Conditions
Rules and Guidelines for Creating Key Ranges
Pass-Through Partition Type
Round-Robin Partition Type
Pushdown Optimization
Pushdown Optimization Overview
Pushdown Optimization Types
Running Source-Side Pushdown Optimization Sessions
Running Target-Side Pushdown Optimization Sessions
Running Full Pushdown Optimization Sessions
Integration Service Behavior with Full Optimization
Active and Idle Databases
Working with Databases
Pushdown Optimization to Databases Using the ODBC Connection
Comparing the Output of the Integration Service and Databases
Rules and Guidelines for IBM DB2
Rules and Guidelines for Netezza
Rules and Guidelines for PostgreSQL
Rules and Guidelines for Teradata
Rules and Guidelines for Vertica
Rules and Guidelines for Microsoft Azure SQL Data Warehouse
Pushdown Compatibility
Incompatible Users for Database Connections
Qualifying Names of Tables in Idle Databases
Working with Dates
Working with Expressions
Operators
Variables
Functions
Rules and Guidelines for Functions in Pushdown Optimization
Error Handling, Logging, and Recovery
Error Handling
Logging
Recovery
Working with Slowly Changing Dimensions
Working with Sequences and Views
Sequences
Sequence Creation Example
Views
View Creation Example
Troubleshooting Orphaned Sequences and Views
Identifying Orphaned Objects Using Session Logs
Identifying Orphaned Objects Using an SQL Query
Removing the Orphaned Objects
Using the $$PushdownConfig Mapping Parameter
Configuring Sessions for Pushdown Optimization
Pushdown Options
Partitioning
Pushdown Optimization for Pass-Through Partitioning
Pushdown Optimization for Key-Range Partitioning
Example of Pushdown Optimization for Session with Multiple Partitions
Rules and Guidelines for Sessions with Multiple Partitions
Target Load Rules
Viewing Pushdown Groups
Pushdown Optimization and Transformations
Pushdown Optimization and Transformations Overview
General Pushdown Restrictions
Aggregator Transformation
Expression Transformation
Filter Transformation
Joiner Transformation
Lookup Transformation
Unconnected Lookup Transformation
Lookup Transformation with an SQL Override
Router Transformation
Sequence Generator Transformation
Sorter Transformation
Source Qualifier Transformation
Source Qualifier Transformation with an SQL Override
Target
Union Transformation
Update Strategy Transformation
Real-time Processing
Real-time Processing Overview
Understanding Real-time Data
Messages and Message Queues
Web Service Messages
Change Data from PowerExchange CDC Sources
Configuring Real-time Sessions
Terminating Conditions
Idle Time
Message Count
Reader Time Limit
Flush Latency
Commit Type
Message Recovery
Prerequisites
Steps to Enable Message Recovery
Recovery File
Message Recovery for JMS and WebSphere MQ Sources
Message Recovery for SAP IDoc, TIBCO, and webMethods Sources
Message Recovery
Session Recovery Data Flush
Recovery Table
PM_REC_STATE Table
Message Processing
Message Recovery
Recovery Queue and Recovery Topic
Message Processing
Message Recovery
Recovery Ignore List
Stopping Real-time Sessions
Restarting and Recovering Real-time Sessions
Restarting Real-time Sessions
Recovering Real-time Sessions
Restart and Recover Commands
Rules and Guidelines for Real-time Sessions
Rules and Guidelines for Message Recovery
Real-time Processing Example
PowerCenter Real-time Products
Commit Points
Commit Points Overview
Target-Based Commits
Source-Based Commits
Determining the Commit Source
Switching from Source-Based to Target-Based Commit
Connecting XML Sources in a Mapping
Connecting Multiple Output Group Custom Transformations in a Mapping
User-Defined Commits
Rolling Back Transactions
Rollback Evaluation
Roll Back Open Transaction
Roll Back on Error
Roll Back on Failed Commit
Understanding Transaction Control
Transformation Scope
Understanding Transaction Control Units
Rules and Guidelines for Working with Transaction Control
Creating Target Files by Transaction
Setting Commit Properties
Row Error Logging
Row Error Logging Overview
Error Log Code Pages
Understanding the Error Log Tables
PMERR_DATA
PMERR_MSG
PMERR_SESS
PMERR_TRANS
Understanding the Error Log File
Configuring Error Log Options
Workflow Recovery
Workflow Recovery Overview
State of Operation
Workflow State of Operation
Session State of Operation
Target Recovery Tables
Creating Target Recovery Tables
Recovery Options
Suspending the Workflow
Configuring Suspension Email
Configuring Workflow Recovery
Recovering Stopped, Aborted, and Terminated Workflows
Recovering Suspended Workflows
Configuring Task Recovery
Task Recovery Strategies
Command Task Strategies
Session Task Strategies
Automatically Recovering Terminated Tasks
Resuming Sessions
Working with Repeatable Data
Source Repeatability
Relational Source
SDK Source
Flat File Source
Transformation Repeatability
Output is Deterministic
Output is Repeatable
Configuring a Mapping for Recovery
Steps to Recover Workflows and Tasks
Recovering a Workflow
Recovering a Workflow Using the Workflow Monitor
Recovering a Session
Recovering a Workflow From a Session
Rules and Guidelines for Session Recovery
Configuring Recovery to Resume from the Last Checkpoint
Unrecoverable Workflows or Tasks
Stopping and Aborting
Stopping and Aborting Overview
Types of Errors
Threshold Errors
Fatal Errors
Integration Service Handling for Session Failure
Stopping or Aborting the Workflow
Stopping or Aborting a Task
Stopping or Aborting a Session Task
Steps to Stop or Abort
Concurrent Workflows
Concurrent Workflows Overview
Configuring Unique Workflow Instances
Recovering Workflow Instances by Instance Name
Rules and Guidelines for Running Concurrent Instances of the Same Instance Name
Configuring Concurrent Workflows of the Same Name
Running Concurrent Web Service Workflows
Configuring Workflow Instances of the Same Name
Recovering Workflow Instances of the Same Name
Rules and Guidelines for Running Concurrent Instances of the Same Instance Name
Using Parameters and Variables
Accessing the Run Instance Name or Run ID
Steps to Configure Concurrent Workflows
Starting and Stopping Concurrent Workflows
Starting Workflow Instances from Workflow Designer
Starting One Concurrent Workflow
Starting Concurrent Workflows from the Command Line
Creating Workflow Instances from the Command Line
Stopping or Aborting Concurrent Workflows
Monitoring Concurrent Workflows
Viewing Session and Workflow Logs
Log Files for Unique Workflow Instances
Log Files for Workflow Instances of the Same Name
Rules and Guidelines for Concurrent Workflows
Grid Processing
Grid Processing Overview
Running Workflows on a Grid
Running Sessions on a Grid
Working with Partition Groups
Forming Partition Groups Without Resource Requirements
Forming Partition Groups With Resource Requirements
Rules and Guidelines for Creating Partition Groups
Working with Caches
Grid Connectivity and Recovery
Configuring a Workflow or Session to Run on a Grid
Rules and Guidelines for Configuring a Workflow or Session to Run on a Grid
Load Balancer
Load Balancer Overview
Assigning Service Levels to Workflows
Assigning Resources to Tasks
Workflow Variables
Workflow Variables Overview
Predefined Workflow Variables
Using Predefined Workflow Variables in Expressions
Evaluating Condition in a Workflow
Evaluating Task Status in a Workflow
Evaluating Previous Task Status in a Workflow
User-Defined Workflow Variables
Workflow Variable Start and Current Values
Datatype Default Values
Creating User-Defined Workflow Variables
Using Worklet Variables
Persistent Worklet Variables
Overriding the Initial Value
Rules and Guidelines for Using Worklet Variables
Assigning Variable Values in a Worklet
Passing Variable Values between Worklets
Configuring Variable Assignments
Parameters and Variables in Sessions
Working with Session Parameters
Changing the Session Log Name
Changing the Target File and Directory
Changing Source Parameters in a File
Changing Connection Parameters
Getting Run-Time Information
Rules and Guidelines for Creating File Parameters and Database Connection Parameters
Mapping Parameters and Variables in Sessions
Assigning Parameter and Variable Values in a Session
Passing Parameter and Variable Values between Sessions
Configuring Variable Assignments
Parameter Files
Parameter Files Overview
Parameter and Variable Types
Where to Use Parameters and Variables
Overriding Connection Attributes in the Parameter File
Parameter File Structure
Parameter File Sections
Comments
Null Values
Sample Parameter File
Configuring the Parameter File Name and Location
Using a Parameter File with Workflows or Sessions
Entering a Parameter File in the Workflow Properties
Entering a Parameter File in the Session Properties
Using Variables to Specify Session Parameter Files
Using a Parameter File with pmcmd
Parameter File Example
Guidelines for Creating Parameter Files
Troubleshooting Parameters and Parameter Files
Tips for Parameters and Parameter Files
FastExport
Using FastExport Overview
Step 1. Create a FastExport Connection
Verifying the Code Page Mapping File
Step 2. Change the Reader
Step 3. Change the Source Connection
Step 4. Override the Control File (Optional)
Rules and Guidelines for Using FastExport
External Loading
External Loading Overview
Before You Begin
External Loader Behavior
Loading Data to a Named Pipe
Staging Data to a Flat File
Partitioning Sessions with External Loaders
Loading to IBM DB2
IBM DB2 EE External Loader
Processing LOB Data
IBM DB2 EEE External Loader
Rules and Guidelines for IBM DB2 EEE External Loaders
Setting Operation Modes
Configuring Authorities, Privileges, and Permissions
Configuring IBM DB2 EE External Loader Attributes
Loading Blank Spaces using the IBM DB2 EE External Loader
IBM DB2 EE External Loader Return Codes
Configuring IBM DB2 EEE External Loader Attributes
Loading to Oracle
Rules and Guidelines for Oracle External Loaders
Loading Multibyte Data to Oracle
Configuring Oracle External Loader Attributes
Loading to Sybase IQ
Rules and Guidelines for Sybase IQ External Loaders
Loading Multibyte Data to Sybase IQ
Delimited Flat File Targets
Fixed-Width Flat File Targets
Configuring Sybase IQ External Loader Attributes
Loading to Teradata
Rules and Guidelines for Teradata External Loaders
Overriding the Control File
Creating User Variables in the Control File
Example
Configuring Teradata MultiLoad External Loader Attributes
Configuring Teradata TPump External Loader Attributes
Configuring Teradata FastLoad External Loader Attributes
Configuring External Loading in a Session
Configuring a Session to Write to a File
Configuring File Properties
Selecting an External Loader Connection
Troubleshooting External Loading
FTP
FTP Overview
Rules and Guidelines for Using FTP
SFTP
Integration Service Behavior
Using FTP with Source Files
Using FTP with Target Files
Configuring FTP in a Session
Configuring SFTP in a Session
Selecting an FTP Connection
Configuring Source File Properties
Configuring Target File Properties
Partitioning FTP File Targets
Session Caches
Session Caches Overview
Cache Memory
Cache Files
Naming Convention for Cache Files
Cache File Directory
Configuring the Cache Size
Calculating the Cache Size
Auto Cache Size
Configuring a Numeric Cache Size
Steps to Configure the Cache Size
Cache Partitioning
Configuring the Cache Size for Cache Partitioning
Aggregator Caches
Incremental Aggregation
Configuring the Cache Sizes for an Aggregator Transformation
Troubleshooting Aggregator Caches
Joiner Caches
1:n Partitioning
n:n Partitioning
Configuring the Cache Sizes for a Joiner Transformation
Troubleshooting Joiner Caches
Lookup Caches
Sharing Caches
Configuring the Cache Sizes for a Lookup Transformation
Rank Caches
Configuring the Cache Sizes for a Rank Transformation
Sorter Caches
Configuring the Cache Size for a Sorter Transformation
XML Target Caches
Configuring the Cache Size for an XML Target
Optimizing the Cache Size
Incremental Aggregation
Incremental Aggregation Overview
Integration Service Processing for Incremental Aggregation
Reinitializing the Aggregate Files
Moving or Deleting the Aggregate Files
Finding Index and Data Files
Partitioning Guidelines with Incremental Aggregation
Preparing for Incremental Aggregation
Configuring the Mapping
Configuring the Session
Session Log Interface
Session Log Interface Overview
Implementing the Session Log Interface
The Integration Service and the Session Log Interface
Rules and Guidelines for Implementing the Session Log Interface
Functions in the Session Log Interface
INFA_InitSessionLog
INFA_OutputSessionLogMsg
INFA_OutputSessionLogFatalMsg
INFA_EndSessionLog
INFA_AbnormalSessionTermination
Session Log Interface Example
Building the External Session Log Library
Building the Library in UNIX
Building the Library in Windows
Using the External Session Log Library
Understanding Buffer Memory
Understanding Buffer Memory Overview
Automatic Buffer Memory Settings
Using Session Configuration Objects for Memory Configuration
Configuring Buffer Memory
Configuring Session Cache Memory
Session Cache Limits
Configuring Automatic Memory Settings for Session Caches
High Precision Data
High Precision Data Overview
Bigint
Decimal
Advanced Workflow Guide
Advanced Workflow Guide
10.4.1
10.4.0
10.2 HotFix 2
10.2 HotFix 1
10.2
10.1.1 HotFix 1
10.1.1
10.1
10.0
Back
Next
Building the External Session Log Library
Building the External Session Log Library
Use the make files provided with the sample program demo_sesslog.cpp to build the external library. The command to compile the library depends on the platform on which you build it.
Session Log Interface Example
Building the Library in UNIX
Building the Library in Windows
Updated June 25, 2020
Download Guide
Send Feedback
Explore Informatica Network
Communities
Knowledge Base
Success Portal
Back to Top
Back
Next