Rules and guidelines for Google BigQuery CDC target
Rules and guidelines for Google BigQuery CDC target
Consider the following limitations when working with a Google BigQuery target:
When you use a Google BigQuery connection in complex mode, you cannot write changed data from a CDC source to a Google BigQuery target.
When you capture changed data from a CDC source, you can only configure a single Google BigQuery target definition in a session. You cannot configure multiple Google BigQuery targets to write changed data from a CDC source.
You cannot configure multiple pipelines in a workflow to write changed data from multiple CDC sources to multiple Google BigQuery targets.
You must define a column as required in the Google BigQuery target table.
If you define a column as required in the Google BigQuery target table, you must map a column in the CDC source to the required column in the Google BigQuery target in the mapping.
When you map a column in the CDC source to a required column in the Google BigQuery target, you must ensure that the column in the CDC source does not contains NULL values. Otherwise, the session fails.
You can only configure the following target session properties for CDC mode:
Target Dataset ID
Target Table Name
Job Poll Interval in Seconds
Pre SQL
Pre SQL Configuration
Post SQL
Post SQL Configuration
Informatica recommends that the PowerCenter Integration Service, the CDC source and PowerExchange for CDC are configured in the same region as Google BigQuery.
To increase performance and avoid run-time environment memory issues, increase the Java heap size in the JVM options for PowerCenter Integration Service. Set
JVMOption1
to
-Xmx1024m
in the
Custom Properties
section of the
Processes
tab of the PowerCenter Integration Service.
To improve performance, specify a higher commit interval for the
Maximum Rows Per Commit
property in the CDC source. However, in case of failure, recovery takes more time for a higher commit interval.
Informatica recommends to use update queries on the CDC source database only if the Google BigQuery target table is partitioned and clustered.