English
  • PowerCenter 10.2 HotFix 2
  • All Products

Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Back Next

Real-time Processing Example

Real-time Processing Example

The following example shows how you can use PowerExchange for IBM WebSphere MQ and PowerCenter to process real-time data.
You want to process purchase orders in real-time. A purchase order can include multiple items from multiple suppliers. However, the purchase order does not contain the supplier or the item cost. When you receive a purchase order, you must calculate the total cost for each supplier. You have a master database that contains your suppliers and their respective items and item cost. You use PowerCenter to look up the supplier and item cost based on the item ID. You also use PowerCenter to write the total supplier cost to a relational database.
Your database administrator recommends that you update the target up to 1,000 messages in a single commit. You also want to update the target every 2,000 milliseconds to ensure that the target is always current.
To process purchase orders in real time, you create and configure a mapping.
The following figure shows a mapping that processes purchase orders in real time:
""
The sample mapping includes the following components:
  • Source.
     WebSphere MQ. Each message is in XML format and contains one purchase order.
  • XML Parser transformation.
     Receives purchase order information from the MQ Source Qualifier transformation. It parses the purchase order ID and the quantity from the XML file.
  • Lookup transformation.
     Looks up the supplier details for the purchase order ID. It passes the supplier information, the purchase item ID, and item cost to the Expression transformation.
  • Expression transformation.
    Calculates the order cost for the supplier.
  • Target.
     Oracle relational database. It contains the supplier information and the total supplier cost.
You create and configure a session and workflow with the following properties:
Property
Value
Message count
1,000
Flush latency interval
2,000 milliseconds
Commit type
Source-based commit
Workflow schedule
Run continuously
The following steps describe how the Integration Service processes the session in real-time:
  1. The Integration Service reads messages from the WebSphere MQ queue until it reads 1,000 messages or after 2,000 milliseconds. When it meets either condition, it stops reading from the WebSphere MQ queue.
  2. The Integration Service looks up supplier information and calculates the order cost.
  3. The Integration Service writes the supplier information and order cost to the Oracle relational target.
  4. The Integration Service starts to read messages from the WebSphere MQ queue again.
  5. The Integration Service repeats steps 1 through 4 as you configured the workflow to run continuously.