Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Transformation in Streaming Mappings
  8. Window Transformation
  9. Appendix A: Connections
  10. Appendix B: Monitoring REST API Reference
  11. Appendix C: Sample Files

Advanced Properties

Advanced Properties

Snowflake data object write operation properties include run-time properties that apply to the Snowflake data object.
The Developer tool displays advanced properties for the Snowflake data object operation in the Advanced view.
The following table describes the Advanced properties for a Snowflake data object write operation:
Property
Description
UpdateMode
Not applicable for streaming mappings.
Database
Overrides the database name specified in the connection.
Schema
Overrides the schema name specified in the connection.
Warehouse
Overrides the Snowflake warehouse name specified in the connection.
Role
Overrides the Snowflake user role specified in the connection.
Pre SQL
Not applicable for streaming mappings.
Post SQL
Not applicable for streaming mappings.
Batch Row Size
Not applicable for streaming mappings.
Number of local staging files
Not applicable for streaming mappings.
Truncate Target Table
Truncates the database target table before inserting new rows.
Select one of the following options:
  • True. Truncates the target table before inserting all rows.
  • False. Inserts new rows without truncating the target table.
Default is false.
Additional Write Runtime Parameters
Specify additional run-time parameters.
Separate multiple runtime parameters with &.
To run streaming mappings to a Snowflake target, configure the following properties:
  • pem_private_key
    . The private key in PEM format for key pair authentication to access Snowflake.
  • streaming_stage
    . The internal stage created in Snowflake from which Snowpipe reads the data files.
    When you specify an internal stage name and the internal stage does not already exist, PowerExchange for Snowflake creates the stage with the name you specify. You must specify a unique name for the streaming stage.
    Ensure that at a time only one pipe reads from the internal stage and only one Snowflake streaming mapping writes to the internal stage.
For example, consider the following run-time parameters:
Specify the
pem_private_key
and the
streaming_stage
properties in the following format:
pem_private_key=<PEM_key_value>&streaming_stage=<stage_name>
The
pem_private_key
and
streaming_stage
properties are applicable for streaming mappings on the Spark engine.
Table Name
Overrides the table name of the Snowflake target table.
Rejected File Path
Not applicable for streaming mappings.
Target Schema Strategy
Target schema strategy for the Snowflake target table.
You can select one of the following target schema strategies:
  • RETAIN - Retain an existing target schema.
  • CREATE - Create a target if it does not exist.
  • To use the CREATE option, you must provide the value of the database and schema name property in the Snowflake connection.
  • Assign Parameter.

0 COMMENTS

We’d like to hear from you!