Table of Contents

Search

  1. Preface
  2. Part 1: Getting Started with Snowflake Data Cloud Connector
  3. Part 2: Data Integration with Snowflake Data Cloud Connector
  4. Part 3: SQL ELT with Snowflake Data Cloud Connector
  5. Appendix A: Data type reference
  6. Appendix B: Additional runtime configurations
  7. Appendix C: Upgrading to Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Rules and guidelines in mappings in SQL ELT mode

Rules and guidelines in mappings in SQL ELT mode

Consider the following rules and guidelines when you run mappings in SQL ELT mode:
General guidelines
Consider the following general guidelines when you run mappings:
  • A mapping that uses input parameters for a custom query to read from a Snowflake table fails.
  • To process a data type used in the Expression transformation, ensure that the precision of the data type matches the precision of the data type returned by Snowflake. Otherwise, the mapping fails.
Data types
Consider the following rules and guidelines when you read or writes certain data types:
  • When you read from and write to Snowflake, ensure that the source data doesn't contain the Time and TIMESTAMPTZ data types.
  • A mapping that reads the Array data type from Microsoft Azure Data Lake Storage Gen2 and writes to Snowflake fails, if the Array data type contains the boolean data.
  • When you use input parameters for the source filter, ensure that you use only the String data type in the filter expression.
  • You can't read the Array data type from the Amazon S3 V2 source in a mapping in SQL ELT mode.
  • A mapping that reads the Array or Object data type from a Snowflake source and write to a Snowflake target created at run time fails.
  • When you run a mapping to write data of the DATE or TIMESTAMPLTZ data type to a Snowflake target created at run time, the mapping converts the date/time data type to the TIMESTAMPNTZ data type and writes the TIMESTAMPNTZ data type with a default precision of 9.
Functions
Consider the following rules and guidelines when you use certain functions in an Expression transformation:
  • When you pass date or time values in the Date or Time function, the mapping fails. To run mapping successfully, select the required fields from the date and time function list to define the field expression and run the mapping.
  • When you pass the date_or_time_part argument in the date or time function, ensure that you enclose the date_or_time_part argument with the single quote character.
    For example, when you use the DATEADD function, use the following format:
    DATEADD('Year', <Value>, <Field name>)
  • When you use the TO_NUMERIC or TO_NUMBER function to process the boolean data type, you need to configure the field that contains the boolean data type as an argument to the function.
    For example, when you use the TO_NUMBER function, use the following syntax in the Expression transformation:
    TO_NUMBER(TO_BOOLEAN(bool_field))
  • Ensure that you pass the uuid and name arguments for the UUID_STRING function.
    For example, when you use the UUID_STRING function, use the following format:
    UUID_STRING(uuid,name)
    If you pass the UUID_STRING function without any argument, Snowflake returns a random UUID.
  • To configure the Window functions in an Expression transformation, you need to select the
    Enable window properties
    check box on the
    Window
    tab.
  • You can't pass more than four arguments for a function in an Expression transformation.
  • To pass an integral argument for the APPROX_TOP_K, APPROX_TOP_K_ACCUMULATE, or MINHASH function, add an integer constant in the argument for the integral argument.
    For more information about how to pass integral arguments, see the Snowflake documentation.
  • To pass the percentile argument for the APPROX_PERCENTILE function, add a numeric constant in the argument that ranges from 0 to 1.
    For more information about how to pass percentile arguments, see the Snowflake documentation.
  • To configure a function that contains multiple arguments with the ANY data type, ensure that the data type for the arguments are homogeneous.
    For example, when you pass the expr1 and expr2 arguments with the ANY data type for the NVL function and the expr1 argument contains the integer data type, the expr2 argument also needs to be of the integer data type.
  • A mapping configured with the NVL or NVL2 function shows the following behavior:
    • Returns incorrect results when all of the following conditions are true:
      • The first argument in the function is NULL.
      • The Snowflake target fields have the Datetime data type.
    • Validates the argument configured in the function, but the mapping fails at runtime if all of the following conditions are true:
      • The first argument in the function is NULL.
      • The Snowflake target fields have the Binary data type.
Target operations
Consider the following rules and guidelines when you configure the Snowflake target objects:
  • When you define the warehouse name and role assigned to the user in the Target transformation, the mapping doesn't honor the value specified in the
    Warehouse
    and
    Role
    target properties. You need to specify the warehouse name and role in the Snowflake Data Cloud connection.
  • To perform an upsert operation on the target, you need to select the target operation as
    Update
    and update mode as
    Update Else Insert
    in the Snowflake Target transformation.
Expression validations
Consider the following rules and guidelines on validation errors for expressions that you use in an Aggregator or Expression transformation:
  • The expression validation fails for an argument if it contains the following configurations:
    • An expression for the UNIFORM function contains the cast operator (::).
    • An expression for a function contains the DISTINCT keyword or asterisk (*) wildcard.
    • A string argument with Numeric values for the Bitwise expression function.
  • The expression validates the argument for a function, but the mapping fails at run time if it contains the following configurations:
    • Optional arguments in the SEQ1, SEQ2, SEQ4, or SEQ8 function use any value other than 0 or 1.
    • A string argument for the Date or Time function is not valid.
    • An Aggregator transformation uses a Window function that is not listed in the functions list.
      For more information about the list of functions that you can use, see Functions in mappings in SQL ELT mode.
    • An expression for a function returns the String data type and the output port is Integer, Bigint, Text, Decimal, or Double data type.
    • An expression for a function contains binary comparison in the argument.
    • An expression for a function contains columns with the Boolean data types.
    • An argument in a function uses a value that is not in the specified range.
      For example, to pass the second argument for the NTH_VALUE function, add a numeric constant in the argument that ranges from 1 to 1000.
      For more information about the range that you can add as an argument in a function, see the Snowflake documentation.
    • An expression contains nested Window functions.
      For example, the expression validates the argument, but the mapping fails at run time when you configure the AVG Window function within the CONDITIONAL_CHANGE_EVENT Window function in the following format, where the argument is SALARY for the AVG function:
      CONDITIONAL_CHANGE_EVENT(AVG(SALARY))

0 COMMENTS

We’d like to hear from you!