Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Snowflake
  3. Snowflake Connections
  4. PowerExchange for Snowflake Data Objects
  5. PowerExchange for Snowflake Mappings
  6. PowerExchange for Snowflake Dynamic Mappings
  7. Snowflake Run-Time Processing
  8. Pushdown Optimization
  9. Appendix A: Snowflake Data Type Reference

PowerExchange for Snowflake User Guide

PowerExchange for Snowflake User Guide

Partitioning

Partitioning

When you read data from or write data to Snowflake, you can configure partitioning to optimize the mapping performance at run time. You can configure partitioning for Snowflake mappings that you run in the native or Spark engine. The partition type controls how the Data Integration Service distributes data among partitions at partition points.
You can define the partition type as key range partitioning. To configure key range partitioning, open the Snowflake data object read or write operation, and select the
Key Range
partition type option on the
Run-time
tab.
When you configure key range partitioning, the Data Integration Service distributes rows of data based on a port or set of ports that you define as the partition key. You can define a range of values for each port. The Data Integration Service uses the key and ranges to send rows to the appropriate partition.
A Snowflake source mapping configured for key range partitioning does not work in the non-native environment. If you specify a condition for key range partitioning on the Snowflake source and run the mapping on the Spark engine, the Data Integration Service does not append the condition to the select query at run time.

0 COMMENTS

We’d like to hear from you!