Table of Contents

Search

  1. Preface
  2. Performance Tuning Overview
  3. Target Optimization
  4. Source Optimization
  5. Transformation Optimization
  6. Mapping Optimization
  7. Partitioned Mapping Optimization
  8. Run-time Optimization
  9. SQL Data Service Optimization
  10. Web Service Optimization
  11. Connections Optimization
  12. Data Transformation Optimization

Performance Tuning Guide

Performance Tuning Guide

Single-Pass Reading

Single-Pass Reading

Single-pass reading allows you to populate multiple targets with one customized data object. Consider using single-pass reading if you have multiple mappings that use the same sources.
Consider the following solutions for single-pass reading bottlenecks:
Combine the transformation logic for each mapping in one mapping and use one customized data object for each source.
The Data Integration Service reads each source once and then sends the data into separate pipelines. A particular row can be used by all the pipelines, by any combination of pipelines, or by no pipelines.
For example, you have the Purchasing source table, and you use that source daily to perform an aggregation and a ranking. If you place the Aggregator and Rank transformations in separate mappings, you force the Data Integration Service to read the same source table twice. However, if you include the aggregation and ranking logic in one mapping with one source qualifier, the Data Integration Service reads the Purchasing source table once, and then sends the appropriate data to the separate pipelines.
Factor out common functions from mappings.
When changing mappings to take advantage of single-pass reading, you can optimize this feature by factoring out common functions from mappings. For example, if you need to subtract a percentage from the Price ports for both the Aggregator and Rank transformations, you can minimize work by subtracting the percentage before splitting the pipeline. You can use an Expression transformation to subtract the percentage, and then split the mapping after the transformation.

0 COMMENTS

We’d like to hear from you!