Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Google BigQuery connectors
  3. Part 2: Data Integration with Google BigQuery V2 Connector
  4. Part 3: Data Integration with Google BigQuery Connector

Google BigQuery Connectors

Google BigQuery Connectors

Rules and guidelines for Google BigQuery V2 connection modes

Rules and guidelines for
Google BigQuery V2
connection modes

Simple mode

Consider the following rules and guidelines when you configure a
Google BigQuery V2
connection to use simple connection mode:
  • You cannot configure mappings in advanced mode.
  • You can read data from a repeated column from a
    Google BigQuery
    source table only when you select
    Direct
    as the
    Read Mode
    .
  • You cannot create a
    Google BigQuery
    target table that contains repeated columns using the
    Create Target
    option.
  • If the
    Google BigQuery
    source table contains repeated columns, you cannot configure data filters for these columns.
  • If the
    Google BigQuery
    table contains more than one repeated column, you cannot preview data.
  • If the
    Google BigQuery
    target table contains a repeated column of the Record data type, you cannot configure update, upsert, and delete operations for these columns.
  • You can use CSV format as the data format of the staging file only when the
    Google BigQuery
    table does not contain columns of the Record data type or repeated columns.
  • If the
    Google BigQuery
    target table contains columns of the Record data type and repeated columns, you cannot configure update, upsert, and delete operations for these columns when you do not use the Merge query.
  • When you read data from a
    Google BigQuery
    source, you must not map more than one repeated column in a single mapping. You must create multiple mappings for each repeated column.
  • You cannot import multiple source tables in a Source transformation.

Hybrid mode

Consider the following rules and guidelines when you configure a
Google BigQuery V2
connection to use hybrid connection mode:
  • You cannot preview data.
  • You cannot use a legacy SQL statement to define a custom query. You must use a standard SQL to define a custom query
  • If the
    Google BigQuery
    source table contains columns of the Record data type and repeated columns, you cannot configure data filters for these columns.
  • When you do not use the Merge query and the key field is a column of the Record data type or a repeated column, you cannot configure update, upsert, and delete operations.
  • You must select JSON (Newline Delimited) format as the data format of the staging file under the advanced target properties. You can use CSV format as the data format of the staging file only when the
    Google BigQuery
    table does not contain columns of the Record data type or repeated columns.
  • The following CSV formatting options in the advanced target properties are not applicable:
    • Allow Quoted Newlines
    • Field Delimiter
    • Allow Jagged Rows

Complex mode

Consider the following rules and guidelines when you configure a
Google BigQuery V2
connection to use complex connection mode:
  • You cannot configure mappings in advanced mode.
  • You cannot import multiple source tables in a Source transformation.
  • You cannot preview data.
  • You cannot use a legacy SQL statement to define a custom query. You must use a standard SQL to define a custom query
  • You cannot create a
    Google BigQuery
    target table using the
    Create Target
    option.
  • You cannot truncate the
    Google BigQuery
    target table before loading data to the target using the
    Truncate target table
    option.
  • When you configure a
    Google BigQuery
    source connection to use complex connection mode, you cannot configure data filters for the source.
  • You cannot configure update, upsert, and delete operations.
  • You must select JSON (Newline Delimited) format as the data format of the staging file under the advanced target properties.
  • You cannot use CSV format as the data format of the staging file. The following CSV formatting options in the advanced target properties are not applicable:
    • Allow Quoted Newlines
    • Field Delimiter
    • Allow Jagged Rows
  • You cannot use key range partitioning for
    Google BigQuery
    sources.

0 COMMENTS

We’d like to hear from you!