Table of Contents

Search

  1. Preface
  2. Part 1: Getting Started with Snowflake Data Cloud Connector
  3. Part 2: Data Integration with Snowflake Data Cloud Connector
  4. Part 3: SQL ELT with Snowflake Data Cloud Connector
  5. Appendix A: Data type reference
  6. Appendix B: Additional runtime configurations
  7. Appendix C: Upgrading to Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Configure load properties in mappings

Configure load properties in mappings

You can configure write properties to load data to Snowflake in the
Additional Write Runtime Parameters
field in the Snowflake Data Cloud advanced target properties of the Target transformation.
The following table lists some of the additional runtime parameters that you can specify to load data to Snowflake:
Property
Description
oneBatch
Process all data in a single batch.
Type is Boolean.
Default is false.
remoteStage
Specifies to use internal stage. External stage is not applicable.
Type is String.
Default is "~"(user stage).
onError
Specifies the action to perform when an error is encountered while loading data from a file.
For example,
onError option ABORT_STATEMENT|CONTINUE|SKIP_FILE
Type is String.
Default is CONTINUE.
compressFileByPut
Compress file by PUT.
Type is Boolean.
Default is false.
compressDataBeforePut
Compress data before PUT.
The loader compresses the data to a gzip format before uploading the data.
Type is Boolean.
Default is true.
copyEmptyFieldAsEmpty
The COPY command option to set incoming empty fields as null.
Type is Boolean.
enablePGZIP
Enables parallelism for the file compression.
Type is Boolean.
Default is true.
onError=ABORT_STATEMENT&oneBatch=true
Load the entire data in single batch and to stop the task if an error occurs. Simultaneously, validate the user-specified reject file path and write the error records to this file and to the session log.
Type is onError - String or oneBatch - Boolean.
When you set the values in the additional runtime parameters field, every configured partition initializes a new loader instance and the configured values apply similarly across all the partitions.

Example 1. Compress files

You want to compress files by using the Put command before loading data to Snowflake.
Specify the following compression option:
compressDataBeforePut=false&compressFileByPut=true
If you specify both the options as true, Snowflake considers the
compressDataBeforePut
option.

Example 2. Replace empty values as null

You want to replace the incoming fields with empty values as NULL while loading the data to Snowflake.
Specify the
copyEmptyFieldAsEmpty Boolean
option and set the value to true or false based on your requirement.
Consider the following scenarios before you configure the
copyEmptyFieldAsEmpty Boolean
parameter:
  • If you do not configure this parameter, Null values are received as NULL, and empty values are received as Empty. This is the default behavior.
  • If you set the parameter
    copyEmptyFieldAsEmpty=false
    , Null values as received as Null and empty values are received as Null.
  • If you set the parameter
    copyEmptyFieldAsEmpty=true
    , Null values are received as empty, while empty values are received as empty.

Example 3. Write data in a single batch

You want to write source data in a single batch to Snowflake. If an error is encountered while loading, you also want to capture the errors.
Specify the
onError=ABORT_STATEMENT&oneBatch=true
property based on your requirement.
While loading in a single batch, if an error occurs, the Secure Agent checks for the specified reject file name, runs the COPY command, validates the reject file, and then passes the file name to capture the errors, if any.

0 COMMENTS

We’d like to hear from you!