Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Mappings in the Hadoop Environment
  4. Mapping Sources in the Hadoop Environment
  5. Mapping Targets in the Hadoop Environment
  6. Mapping Transformations in the Hadoop Environment
  7. Processing Hierarchical Data on the Spark Engine
  8. Configuring Transformations to Process Hierarchical Data
  9. Processing Unstructured and Semi-structured Data with an Intelligent Structure Model
  10. Stateful Computing on the Spark Engine
  11. Monitoring Mappings in the Hadoop Environment
  12. Mappings in the Native Environment
  13. Profiles
  14. Native Environment Optimization
  15. Cluster Workflows
  16. Connections
  17. Data Type Reference
  18. Function Reference
  19. Parameter Reference

Big Data Management User Guide

Big Data Management User Guide

Financial Plans Example

Financial Plans Example

You are a banker with information about the financial plans of two of your customers. Each plan has an associated start date.
For each customer, you want to know the expiration date for the current plan based on the activation date of the next plan. The previous plan ends when a new plan starts, so the end date for the previous plan is the start date of the next plan minus one day.
The following table lists the customer codes, the associated plan codes, and the start date of each plan:
CustomerCode
PlanCode
StartDate
C1
00001
2014-10-01
C2
00002
2014-10-01
C2
00002
2014-11-01
C1
00004
2014-10-25
C1
00001
2014-09-01
C1
00003
2014-10-10
Define partition and order keys
You partition the data by customer code and order the data by ascending start date.
You configure the following windowing properties:
Property
Description
Order key
StartDate Ascending
. Arranges the data chronologically by ascending start date.
Partition key
CustomerCode
. Groups the rows according to customer code so that calculations are based on individual customers.
Frame
Not specified. Window functions access rows based on the offset argument and ignore the frame specification.
The following image shows the windowing properties you configure on the Windowing tab:
On the Windowing tab, StartDate is selected as the order key in ascending order. CustomerCode is selected as the partition key. The frame specification is not configured.
The following table lists the data grouped by customer code and ordered by start date:
CustomerCode
PlanCode
StartDate
C1
00001
2014-09-01
C1
00002
2014-10-01
C1
00003
2014-10-10
C1
00004
2014-10-25
C2
00001
2014-10-01
C2
00002
2014-11-01
The start dates for each customer are arranged in ascending order so that the dates are chronological.
Define a window function
You define a LEAD function to access the subsequent row for every input.
You define the following function on the Ports tab of the Expression transformation:
LEAD ( StartDate, 1, '01-Jan-2100' )
Where:
  • StartDate
    indicates the target column that the function operates on.
  • 1
    is the offset. This value accesses the next immediate row.
  • 01-Jan-2100
    is the default value. The expression returns "01-Jan-2100" if the returned value is outside the bounds of the partition.
Define an ADD_TO_DATE function
You use an ADD_TO_DATE function to subtract one day from the date you accessed.
You define the following expression on the Ports tab of the Expression transformation:
ADD_TO_DATE ( LEAD ( StartDate, 1, '01-Jan-2100' ), 'DD', -1, )
By subtracting one day from the start date of the next plan, you find the end date of the current plan.
The following table lists the end dates of each plan:
CustomerCode
PlanCode
StartDate
EndDate
C1
00001
2014-09-01
2014-09-30
C1
00002
2014-10-01
2014-10-09
C1
00003
2014-10-10
2014-10-24
C1
00004
2014-10-25
2099-12-31*
C2
00001
2014-10-01
2014-10-31
C2
00002
2014-11-01
2099-12-31*
*The LEAD function returned the default value because these plans have not yet ended. The rows were outside the partition, so the ADD_TO_DATE function subtracted one day from 01-Jan-2100, returning 2099-12-31.

0 COMMENTS

We’d like to hear from you!