Connector Release Notes

Connector Release Notes

Google BigQuery V2 Connector

Google BigQuery V2
Connector

Fixed issues

The following table describes fixed issues:
Issue
Description
CCON-94513
When you configure the BigQuery Storage API to stage files in the connection and run a mapping, the mapping fails in the following scenarios:
  • When the mapping reads data from multiple source objects.
  • When an connected uncached Lookup transformation is configured for view objects.
  • When the mapping is enabled with source
    SQL ELT optimization
    .
(April 2024)
CCON-93132
When you configure a mapping with Avro staging format and use the Decimal data type, the mapping fails with the following error:
[ERROR] The [LOAD] job failed with the following error: [Error while reading data, error message: The value for field 'COL_NUMERIC' cannot be converted to NUMERIC: Value is out of range after scaling to NUMERIC type; input length: 18; scale: 9; input data: 0dc67ba953936d39d87f2b05780000000000 File: bigstore/automation_bq_bucket1/tgt.avro.zz]
(February 2024)
CCON-79744
When you configure an uncached Lookup transformation in a mapping to look up data from a Google BigQuery source, and the source column contains a NULL value in the first row, data corruption occurs in the target.
If the source contains a NULL value in any other rows of the column, the mapping considers the filter condition for the previous row. This results in data duplication in the target.
(April 2024)
CCON-63141
When you set the JVM option system property for the DTM type to
-DHonorInfaDateFormat=true
for the Secure Agent and configure a mapping enabled with
SQL ELT optimization
to read string data type from an Amazon S3 source and write to byte data type in Google BigQuery target in a flat file, the mapping fails with the following error:
[ERROR] The Secure Agent failed to run the full pushdown query due to the following error: [Query column 9 has type TIMESTAMP which cannot be inserted into column col10_byte, which has type BYTES at [14:4]]
(November 2023)

Known issues

The following table describes known issues:
Issue
Description
CCORE-6229
When you configure a mapping task with the parameter value
$PMSessionErrorThreshold
for the
Stop on errors
session property, the mapping does not honor the value set for the property.
(April 2024)
CCON-97331
When you run a mapping with Streaming as the write mode, the mapping fails with the following error for the Byte, Date/Time, and Numeric data types:
  • Byte:
    [ERROR] The following error occurred while loading data: [Failed to decode bytes input. Byte fields must be base64 encoded, value: java.nio.HeapByteBuffer[pos=0 lim=14 cap=14].]
  • Date/Time:
    [ERROR] The following error occurred while loading data: [Epoch second value -2189418433211000 out of range]
  • Numeric:
    [ERROR] Error Occured for Transformation - Source while writing the data to DTM Buffer - Data Conversion Failed
(April 2024)
CCON-96963
When the BigQuery Storage API is enabled in the connection and the source object in a mapping is a view with key-range partitioning defined, the mapping runs successfully initially. However, when you rerun the mapping, the staging tables created during the first run are not deleted and the mapping fails with the following error:
[ERROR] The following error occurred: [Already Exists: Table automation-bigquery-project:QATEST.query_tab_view_p_2]
(April 2024)
CCON-96384
When you run a mapping where the source data includes Repeated data type columns and a filter is configured on a column that does not contain Repeated data type, the mapping fails with the following error based on the read mode set in the mapping:
  • Direct:
    [ERROR] The following error occurred: [For input string: "[\{v=1}, \{v=2}]"]
  • Staging:
    [ERROR] The [QUERY] job failed with the following error: [Invalid schema update. Field <column_name> has changed mode from NULLABLE to REPEATED]
(April 2024)

0 COMMENTS

We’d like to hear from you!