Table of Contents

Search

  1. Preface
  2. Connectors and connections
  3. Connection configuration
  4. Connection properties
  5. Swagger file generation for REST V2 connections

Connections

Connections

Google BigQuery V2 connection properties

Google BigQuery V2 connection properties

When you create a Google BigQuery V2 connection, configure the connection properties.
The following table describes the Google BigQuery V2 connection properties:
Property
Description
Connection Name
Name of the connection.
Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -,
Maximum length is 255 characters.
Description
Description of the connection. Maximum length is 4000 characters.
Type
The Google Big Query V2 connection type.
Runtime Environment
Name of the runtime environment where you want to run the tasks.
Select a Secure Agent, Hosted Agent, or serverless runtime environment.
You cannot run a database ingestion task on a Hosted Agent or serverless runtime environment.
Service Account ID
The client_email value in the JSON file that you download after you create a service account.
Service Account Key
The private_key value in the JSON file that you download after you create a service account.
Project ID
The project_id value in the JSON file that you download after you create a service account.
If you have created multiple projects with the same service account, enter the ID of the project that contains the dataset that you want to connect to.
Storage Path
Path in Google Cloud Storage where the agent creates a local stage file to store the data temporarily.
Applies to tasks that read or write large volumes of data. Use this property when you read data in staging mode or write data in bulk mode.
You can either enter the bucket name or the bucket name and folder name.
Use one of the following formats:
  • gs://<bucket name>
  • gs://<bucket name>/<folder_name>
Connection mode
The mode that you want to use to read data from or write data to Google BigQuery.
Select one of the following connection modes:
  • Simple. Flattens each field within the Record data type field as a separate field in the mapping.
  • Hybrid¹. Displays all the top-level fields in the Google BigQuery table including Record data type fields. Google BigQuery V2 Connector displays the top-level Record data type field as a single field of the String data type in the mapping.
  • Complex¹. Displays all the columns in the Google BigQuery table as a single field of the String data type in the mapping.
Default is Simple.
Schema Definition File Path¹
Directory on the Secure Agent machine where the Secure Agent must create a JSON file with the sample schema of the Google BigQuery table. The JSON file name is the same as the Google BigQuery table name.
Alternatively, you can specify a storage path in Google Cloud Storage where the Secure Agent must create a JSON file with the sample schema of the Google BigQuery table. You can download the JSON file from the specified storage path in Google Cloud Storage to a local machine.
The schema definition file is required if you configure complex connection mode in the following scenarios:
  • You add a Hierarchy Builder transformation in a mapping to read data from relational sources and write data to a Google BigQuery target.
  • You add a Hierarchy Parser transformation in a mapping to read data from a Google BigQuery source and write data to relational targets.
When you use a serverless runtime environment, you must specify a storage path in Google Cloud Storage.
Use Legacy SQL For Custom Query¹
Select this option to use a legacy SQL to define a custom query. If you clear this option, you must use a standard SQL to define a custom query.
Not applicable when you configure the Google BigQuery V2 connection in hybrid or complex mode.
Dataset Name for Custom Query¹
When you define a custom query, you must specify a Google BigQuery dataset.
Region Id
The region name where the Google BigQuery dataset that you want to access resides.
You must ensure that you specify a bucket name or the bucket name and folder name in the
Storage Path
property that resides in the specified region.
For more information about the regions supported by Google BigQuery, see Dataset locations.
Staging Dataset¹
The Google BigQuery dataset name where you want to create the staging table to stage the data. You can define a Google BigQuery dataset that is different from the source or target dataset.
Optional Properties¹
Specifies whether you can configure source and target functionality through custom properties.
You can select one of the following options:
  • None. If you do not want to configure any custom properties, select None.
  • Required. If you want to specify custom properties to configure the source and target functionalities.
Default is None.
Provide Optional Properties¹
Comma-separated key-value pairs of custom properties in the Google BigQuery V2 connection to configure certain source and target functionalities.
Appears when you select
Required
in the Optional Properties.
For more information about the list of custom properties that you can specify, see the Informatica Knowledge Base article: https://kb.informatica.com/faq/7/Pages/26/632722.aspx
¹ Doesn't apply to mappings in advanced mode.
Ensure that you specify valid credentials in the connection properties. The test connection is successful even if you specify incorrect credentials in the connection properties.

Retry Strategy

When you read data from Google BigQuery in staging mode, you can configure the retry strategy when the Google BigQuery V2 connection fails to connect to the Google BigQuery source.
The following table describes the retry properties for the Google BigQuery V2 connection:
Property
Description
Enable Retry¹
Indicates that the Secure Agent attempts to retry the connection when there is a failure.
Select this option to enable connection retry.
Default is unselected.
Maximum Retry Attempts¹
The maximum number of retry attempts that the Secure Agent performs to receive the response from the Google BigQuery endpoint.
If the Secure Agent fails to connect to Google BigQuery within the maximum retry attempts, the connection fails.
Default is 6.
Appears when you select the
Enable Retry
property.
Initial Retry Delay¹
The initial wait time in seconds before the Secure Agent attempts to retry the connection.
Default is 1.
Appears when you select the
Enable Retry
property.
Retry Delay Multiplier¹
The multiplier that the Secure Agent uses to exponentially increase the wait time between successive retry attempts up to the maximum retry delay time.
Default is 2.0.
Appears when you select the
Enable Retry
property.
Maximum Retry Delay¹
The maximum wait time in seconds that the Secure Agent waits between successive retry attempts.
Default is 32.
Appears when you select the
Enable Retry
property.
Total Timeout¹
The total time duration in seconds that the Secure Agent attempts to retry the connection after which the connection fails.
Default is 50.
Appears when you select the
Enable Retry
property.
¹ Applies only to mappings.

0 COMMENTS

We’d like to hear from you!