Snowflake Data Cloud Connector

Back Next

Configure load properties in mappings

You can configure write properties to load data to Snowflake in the Additional Write Runtime Parameters
field in the Snowflake Data Cloud advanced target properties of the Target transformation.

The following table lists some of the additional runtime parameters that you can specify to load data to Snowflake:

Property	Description
oneBatch	Process all data in a single batch. Type is Boolean. Default is false.
remoteStage	Specifies to use internal stage. External stage is not applicable. Type is String. Default is "~"(user stage).
onError	Specifies the action to perform when an error is encountered while loading data from a file. For example, onError option ABORT_STATEMENT\|CONTINUE\|SKIP_FILE Type is String. Default is CONTINUE.
compressFileByPut	Compress file by PUT. Type is Boolean. Default is false.
compressDataBeforePut	Compress data before PUT. The loader compresses the data to a gzip format before uploading the data. Type is Boolean. Default is true.
copyEmptyFieldAsEmpty	The COPY command option to set incoming empty fields as null. Type is Boolean.
enablePGZIP	Enables parallelism for the file compression. Type is Boolean. Default is true.
onError=ABORT_STATEMENT&oneBatch=true	Load the entire data in single batch and to stop the task if an error occurs. Simultaneously, validate the user-specified reject file path and write the error records to this file and to the session log. Type is onError - String or oneBatch - Boolean.

When you set the values in the additional runtime parameters field, every configured partition initializes a new loader instance and the configured values apply similarly across all the partitions.

Example 1. Compress files

You want to compress files by using the Put command before loading data to Snowflake.

Specify the following compression option:

compressDataBeforePut=false&compressFileByPut=true

If you specify both the options as true, Snowflake considers the

compressDataBeforePut

option.

Example 2. Replace empty values as null

You want to replace the incoming fields with empty values as NULL while loading the data to Snowflake.

Specify the

copyEmptyFieldAsEmpty Boolean

option and set the value to true or false based on your requirement.

Consider the following scenarios before you configure the

copyEmptyFieldAsEmpty Boolean

parameter:

If you do not configure this parameter, Null values are received as NULL, and empty values are received as Empty. This is the default behavior.

If you set the parameter

copyEmptyFieldAsEmpty=false

, Null values as received as Null and empty values are received as Null.

If you set the parameter

copyEmptyFieldAsEmpty=true

, Null values are received as empty, while empty values are received as empty.

Example 3. Write data in a single batch

You want to write source data in a single batch to Snowflake. If an error is encountered while loading, you also want to capture the errors.

Specify the

onError=ABORT_STATEMENT&oneBatch=true

property based on your requirement.

While loading in a single batch, if an error occurs, the Secure Agent checks for the specified reject file name, runs the COPY command, validates the reject file, and then passes the file name to capture the errors, if any.

Rename Saved Search

Table of Contents

Snowflake Data Cloud Connector

Snowflake Data Cloud Connector

Configure load properties in mappings

Configure load properties in mappings

Example 1. Compress files

Example 2. Replace empty values as null

Example 3. Write data in a single batch