Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Data Integration Connectors
  • All Products

Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Google BigQuery connectors
  3. Part 2: Data Integration with Google BigQuery V2 Connector
  4. Part 3: Data Integration with Google BigQuery Connector

Google BigQuery Connectors

Google BigQuery Connectors

Back Next

Google BigQuery Storage API

Google BigQuery Storage API

When you configure a connection, you can choose to use Google BigQuery Storage to stage the data. To enable Google BigQuery Storage, select the
Enable BigQuery Storage API
 property in the connection advanced properties.
You can use hybrid connection mode to import the metadata and stage files that contain hierarchical data types such as record and repeat.
You can use the standard SQL format for the custom query, but you cannot use the legacy SQL format.

Read operation

  • Uses the Staging mode as read mode.
  • You can configure the following runtime attributes for the read operation:
    • Connection
    • Source Type
    • Object
    • Parameter
    • Query
    • Filter
    • Source Dataset ID
    • Source Table Name
    • Source Staging Dataset
    • Number of Rows to Read
    • Allow Large Results
    • Query Results Table Name
    • Job Poll Interval In Seconds
    • Read Mode
    • Persist Destination Table
    • pre SQL
    • post SQL
    • pre SQL Configuration
    • post SQL Configuration
    • SQL Override Query
    • Use Legacy SQL For SQL Override

Write operation

  • Uses the Bulk mode as write mode.
  • You cannot use update, delete, and merge target operations on the rows in a target table that were written in the last 30 minutes. If you configure a mapping with this scenario, the mapping fails with one of the following errors:
    • [ERROR] The Google BigQuery V2 Target definition post-SQL operation failed with the following error: [UPDATE or DELETE statement over table table1_BQ_Storage.tgt_mct_151_table1 would affect rows in the streaming buffer, which is not supported].
    • [ERROR] Error occured while trying to Initialize Data Source Operation | com.informatica.cci.runtime.internal.utils.impl.CExceptionImpl: Unable to render embedded object: File (Truncate Target Failed UPDATE or DELETE statement over table automation-bigquery-project.table1_BQ_Storage.tgt_mct_151_table1 would affect rows in the streaming buffer, which is not supported) not found.
  • You cannot use the
    write empty
     option in the
    Write Disposition
     property.
  • You can use the
    write truncate
     option in the
    Write Disposition
     property only with a truncated target table.
  • You can configure the following runtime attributes for the write operation:
    • Connection
    • Object
    • Target Type. You can use single object as the Target Type.
    • Operation. You can use only the Insert operation.
    • Write Mode
    • Create New at Runtime
    • Data Driven Condition
    • Update Columns
    • Target Dataset ID
    • UpdateMode
    • Enable Data Driven
    • Enable Merge
    • Update Override
    • Target Table Name
    • Target Staging Dataset
    • Create Disposition
    • Job Poll Interval In Seconds
    • pre SQL
    • post SQL
    • Suppress post SQL on Error
    • pre SQL Configuration
    • post SQL Configuration
    • Truncate target table
0 COMMENTS
We’d like to hear from you!