PowerExchange for Google BigQuery User Guide

10.5.2
- 10.5.9
- 10.5.8
- 10.5.7
- 10.5.6
- 10.5.5
- 10.5.4
- 10.5.3
- 10.5
- 10.4.1
- 10.4.0

Back Next

Partitioning

When you read data from or write data to Google BigQuery, you can configure partitioning to optimize the mapping performance at run time. You can configure partitioning for Google BigQuery mappings that you run in the native or Spark engine. The partition type controls how the Data Integration Service distributes data among partitions at partition points. You can configure a partition key for a Google BigQuery data object that uses a simple or hybrid connection mode.

You can define the partition type as key range partitioning. To configure key range partitioning, open the Google BigQuery data object read operation, and select the

Key Range

partition type option on the

Run-time

tab.

When you configure key range partitioning, the Data Integration Service distributes rows of data based on a port or set of ports that you define as the partition key. You can define a range of values for each port. The Data Integration Service uses the key and ranges to send rows to the appropriate partition.

When you use a simple connection mode in a Google BigQuery connection, you can configure a partition key for fields of the following data types:

Integer

Float

When you use a hybrid connection mode in a Google BigQuery connection, you can configure a partition key for fields of the following data types:

Integer

Float

Numeric

Timestamp

You cannot configure a partition key for Record data type columns and repeated columns.

You can configure dynamic partitioning for Google BigQuery data object write operation. To configure dynamic partitioning, open the Google BigQuery data object write operation, and select the

Dynamic

partition type option on the

Runtime

tab.

Google BigQuery Run-Time Processing

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal