Google BigQuery Connectors

Back Next

Google BigQuery Storage API

When you configure a connection, you can choose to use Google BigQuery Storage to staging when you read or write data. To use Google BigQuery Storage, select the

Enable BigQuery Storage API

connection advanced property.

You can use hybrid connection mode to import the metadata and stage files that contain hierarchical data types such as record and repeat.

You can use the standard SQL format for the custom query, but you cannot use the legacy SQL format.

Read operation

Uses the Staging mode as read mode.

You can configure the following runtime attributes for the read operation:

Connection

Source Type

Object

Parameter

Query

Filter

Source Dataset ID

Source Table Name

Source Staging Dataset

Number of Rows to Read

Allow Large Results

Query Results Table Name

Job Poll Interval in Seconds

Read Mode

Persist Destination Table

pre SQL

pre SQL Configuration

post SQL

post SQL Configuration

SQL Override Query

Use Legacy SQL for SQL Override

Write operation

Uses the Bulk mode as write mode.

You cannot use update, delete, and merge target operations on the rows in a target table that were written in the last 30 minutes. If you configure a mapping with this scenario, the mapping fails with one of the following errors:

[ERROR] The Google BigQuery V2 Target definition post-SQL operation failed with the following error: [UPDATE or DELETE statement over table table1_BQ_Storage.tgt_mct_151_table1 would affect rows in the streaming buffer, which is not supported].

[ERROR] Error occured while trying to Initialize Data Source Operation | com.informatica.cci.runtime.internal.utils.impl.CExceptionImpl: Unable to render embedded object: File (Truncate Target Failed UPDATE or DELETE statement over table automation-bigquery-project.table1_BQ_Storage.tgt_mct_151_table1 would affect rows in the streaming buffer, which is not supported) not found.

You cannot use the

write empty

option in the

Write Disposition

property.

You can use the

write truncate

option in the

Write Disposition

property only with a truncated target table.

You can configure the following runtime attributes for the write operation:

Connection

Object

Target Type. You can use single object as the Target Type.

Operation. You can use only the Insert operation.

Write Mode

Create New at Runtime

Data Driven Condition

Update Columns

Target Dataset ID

Update Mode

Enable Data Driven

Enable Merge

Update Override

Target Table Name

Target Staging Dataset

Create Disposition

Job Poll Interval in Seconds

Pre SQL

Pre SQL Configuration

Post SQL

Suppress post SQL on Error

Post SQL Configuration

Truncate target table

Mappings for Google BigQuery V2

Download Guide

Watch

Comments

Cloud Connectivity Homepage

Communities

Knowledge Base

Success Portal

0 COMMENTS

We’d like to hear from you! Log in to comment.

Rename Saved Search

Table of Contents

Google BigQuery Connectors

Google BigQuery Connectors

Google BigQuery Storage API

Google BigQuery Storage API

Read operation

Write operation