Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for Google BigQuery
  3. PowerExchange for Google BigQuery Configuration
  4. Google BigQuery Sources and Targets
  5. Google BigQuery Mappings
  6. Google BigQuery Sessions
  7. Google BigQuery as CDC Target
  8. Google BigQuery Pushdown Optimization
  9. Appendix A: Google BigQuery Data Type Reference

PowerExchange for Google BigQuery User Guide for PowerCenter

PowerExchange for Google BigQuery User Guide for PowerCenter

Rules and guidelines for Google BigQuery CDC target

Rules and guidelines for Google BigQuery CDC target

Consider the following limitations when working with a Google BigQuery target:
  • When you use a Google BigQuery connection in complex mode, you cannot write changed data from a CDC source to a Google BigQuery target.
  • When you capture changed data from a CDC source, you can only configure a single Google BigQuery target definition in a session. You cannot configure multiple Google BigQuery targets to write changed data from a CDC source.
  • You cannot configure multiple pipelines in a workflow to write changed data from multiple CDC sources to multiple Google BigQuery targets.
  • You must define a column as required in the Google BigQuery target table.
  • If you define a column as required in the Google BigQuery target table, you must map a column in the CDC source to the required column in the Google BigQuery target in the mapping.
  • When you map a column in the CDC source to a required column in the Google BigQuery target, you must ensure that the column in the CDC source does not contains NULL values. Otherwise, the session fails.
  • You can only configure the following target session properties for CDC mode:
    • Target Dataset ID
    • Target Table Name
    • Job Poll Interval in Seconds
    • Pre SQL
    • Pre SQL Configuration
    • Post SQL
    • Post SQL Configuration
  • Informatica recommends that the PowerCenter Integration Service, the CDC source and PowerExchange for CDC are configured in the same region as Google BigQuery.
  • To increase performance and avoid run-time environment memory issues, increase the Java heap size in the JVM options for PowerCenter Integration Service. Set
    JVMOption1
    to
    -Xmx1024m
    in the
    Custom Properties
    section of the
    Processes
    tab of the PowerCenter Integration Service.
  • To improve performance, specify a higher commit interval for the
    Maximum Rows Per Commit
    property in the CDC source. However, in case of failure, recovery takes more time for a higher commit interval.
  • Informatica recommends to use update queries on the CDC source database only if the Google BigQuery target table is partitioned and clustered.

0 COMMENTS

We’d like to hear from you!