Table of Contents

Search

  1. Preface
  2. Understanding Pipeline Partitioning
  3. Partition Points
  4. Partition Types
  5. Pushdown Optimization
  6. Pushdown Optimization and Transformations
  7. Real-time Processing
  8. Commit Points
  9. Row Error Logging
  10. Workflow Recovery
  11. Stopping and Aborting
  12. Concurrent Workflows
  13. Grid Processing
  14. Load Balancer
  15. Workflow Variables
  16. Parameters and Variables in Sessions
  17. Parameter Files
  18. FastExport
  19. External Loading
  20. FTP
  21. Session Caches
  22. Incremental Aggregation
  23. Session Log Interface
  24. Understanding Buffer Memory
  25. High Precision Data

Advanced Workflow Guide

Advanced Workflow Guide

Rules and Guidelines for Functions in Pushdown Optimization

Rules and Guidelines for Functions in Pushdown Optimization

Use the following rules and guidelines when pushing functions to a database:

Relational Databases

  • If you use ADD_TO_DATE in transformation logic to change days, hours, minutes, or seconds, you cannot push the function to a Teradata database.
  • When you push LAST_DAY() to Oracle, Oracle returns the date up to the second. If the input date contains subseconds, Oracle trims the date to the second.
  • When you push LTRIM, RTRIM, or SOUNDEX to a database, the database treats the argument (' ') as NULL, but the PowerCenter Integration Service treats the argument (' ') as spaces.
  • An IBM DB2 database and the PowerCenter Integration Service produce different results for STDDEV and VARIANCE. IBM DB2 uses a different algorithm than other databases to calculate STDDEV and VARIANCE.
  • When you push SYSDATE or SYSTIMESTAMP to the database, the database server returns the timestamp in the time zone of the database server, not the PowerCenter Integration Service.
  • If you push SYSTIMESTAMP to an IBM DB2 or a Sybase database, and you specify the format for SYSTIMESTAMP, the database ignores the format and returns the complete time stamp.

PowerExchange for Amazon Redshift

  • To push TRUNC(DATE) to Amazon Redshift, you must define the date and format arguments. Otherwise, the PowerCenter Integration Service does not push the function to Amazon Redshift.
  • The aggregator functions for Amazon Redshift accept only one argument, a field set for the aggregator function. The filter condition argument is not honored. In addition, make sure that all ports mapped to the output are listed in the GROUP BY clause.
  • For Amazon Redshift, when you define only a string argument for TO_DATE() and TO_CHAR(), the PowerCenter Integration Service considers the default date format present in the session property. The default date format in the session property is: MM/DD/YYYY HH24:MI:SS.US
  • Do not specify a format for SYSTIMESTAMP() to push the SYSTIMESTAMP to Amazon Redshift. The Amazon Redshift database returns the complete time stamp.
  • To push INSTR() to Amazon Redshift, you must only define string, search_value, and start arguments. Amazon Redshift does not support occurrence and comparison_type arguments.
  • The flag argument is ignored when you push TO_BIGINT and TO_INTEGER to Amazon Redshift.
  • The CaseFlag argument is ignored when you push IN() to Amazon Redshift.
  • If you use the NS format as part of the ADD_TO_DATE() function, the PowerCenter Integration Service does not push the function to Amazon Redshift.
  • If you use any of the following formats as part of the TO_CHAR() and TO_DATE() functions, the PowerCenter Integration Service does not push the function to Amazon Redshift:
    • NS
    • SSSS
    • SSSSS
    • RR
  • To push TRUNC(DATE) and DATE_DIFF() to Amazon Redshift, you must use the following formats:
    • D
    • HH24
    • MI
    • MM
    • MS
    • SS
    • US
    • YYYY
  • To push GET_DATE_PART() to Amazon Redshift, you must use the following formats:
    • D
    • DDD
    • HH24
    • MI
    • MM
    • MS
    • SS
    • US
    • YYYY

PowerExchange for Netezza

  • You can push SYSTIMESTAMP(‘SS’) to a Netezza database, but not SYSTIMESTAMP(‘MS’) or SYSTIMESTAMP(‘US’).
  • When you push TO_CHAR(DATE) or TO_DATE() to Netezza, dates with subsecond precision must be in the YYYY-MM-DD HH24:MI:SS.US format. If the format is different, the PowerCenter Integration Service does not push the function to Netezza.

PowerExchange for Vertica

  • If you use any of the following formats as part of the DATE_DIFF() function, the PowerCenter Integration Service does not push the function to Vertica:
    • YYY
    • MON
    • MONTH
    • HH12
    • HH24
  • When you push the DATE_DIFF function to Vertica, Vertica rounds the date difference value to the nearest integer. However, the PowerCenter Integration Service returns a float value. For example, if the first date is 2000-08-15 and the second date is 1997-08-16, Vertica rounds the date difference value to 3, but the PowerCenter Integration Service returns 2.99731182795699. If you want the date difference to be treated as a float value in the Vertica database, you can disable pushdown optimization.
  • When you specify the format as Y and push the DATE_DIFF function to Vertica, Vertica calculates the difference in the dates in terms of number of days. However, the PowerCenter Integration Service calculates the difference in terms of number of years. If you want the difference value to be treated in terms of number of years, you can disable pushdown optimization.

PowerExchange for Greenplum

  • To push TRUNC(DATE) to Greenplum, you must use the following formats:
    • YYYY
    • DD
    • DOY
    • HH
    • US
    • MS
    • MI
    • MM
    • SS

PowerExchange for Snowflake

  • To push the TRUNC(DATE) or TO_CHAR() function to the Snowflake database, you must define the date and format arguments.
  • The Snowflake aggregate functions accept only one argument, which is a field set for the aggregate function. The PowerCenter Integration Service ignores any filter condition defined in the argument. Ensure that all fields mapped to the target are listed in the GROUP BY clause.
  • When you push the SYSTIMESTAMP() or SYSDATE() functions to the Snowflake database, do not specify any format. The Snowflake database returns the complete time stamp.
  • You cannot push the TO_BIGINT() or TO_INTEGER() function with more than one argument to the Snowflake database.
  • When you push the REPLACECHR() or REPLACESTR() function to the Snowflake database, the PowerCenter Integration Service ignores the caseFlag argument.
    For example, both REPLACECHR(false, in_F_CHAR, 'a', 'b') and REPLACECHR(true, in_F_CHAR, 'a', 'b') arguments return the same value.
  • You cannot use millisecond and microsecond values when you push functions to the Snowflake database.
  • You can use nanosecond values in the ADD_TO_DATE() and TRUNC(DATE) functions.
  • To push the TRUNC(DATE), GET_DATE_PART(), or DATE_DIFF() functions to the Snowflake database, you must use the following time formats as arguments:
    • D
    • DDD
    • HH
    • MI
    • MM
    • SS
    • YYYY
    For example,
    TRUNC(<datefieldname>, 'dd')
    .
    For information on date and time related functions, see the following website: https://docs.snowflake.net/manuals/sql-reference/functions-date-time.html#label-supported-date-time-parts


Updated November 14, 2019