When you read from Google BigQuery, you can configure pass-through partitioning to optimize the session performance at run time.
When you specify pass-through partitioning for a Google BigQuery Source Qualifier transformation, you can specify filter conditions in the Google BigQuery session properties to override the filter condition you specify in the source qualifier. The PowerCenter Integration Service uses the filter condition you specify in the session properties when it filters data from the source.
To configure pass-through partitioning, select the Source Qualifier transformation, and add a partition point from the
Mapping
tab of the session properties. Add the number of partitions you require and select the partition type as
Pass Through
for each of the partitions.
Based on the number or partitions you add, the PowerCenter Integration Service adds those many number of partition fields for the
Filter Override
,
Filter Override
,
Local Staging File Directory
,
SQL Override Query
, and
Use Legacy SQL for SQL override
property in the session properties. Specify the filter override condition for each of the partitions. The PowerCenter Integration Service uses the filter conditions you specify to pass data through the appropriate partition.
You can configure filter override condition for the columns of the following data types:
Integer
Sting
Timestamp
Numeric
When you define filter override condition for columns of Numeric data type, you must ensure that you use a Google BigQuery connection in hybrid connection mode.