Table of Contents

Search

  1. Preface
  2. Part 1: Introduction to Google BigQuery connectors
  3. Part 2: Data Integration with Google BigQuery V2 Connector
  4. Part 3: Data Integration with Google BigQuery Connector

Google BigQuery Connectors

Google BigQuery Connectors

Stop on errors

Stop on errors

You can configure the maximum number of load job errors at the target that a mapping task can encounter before it fails.
The error threshold can include malformed rows and data conversion errors in the load job, but the errors resulting from update, delete, or merge queries are excluded.
You can set the error threshold value in the
Stop on errors
session property in a mapping task. You can set the value from 0 to 2147483647. Values less than 0, alphabets, or special characters are not allowed.
To configure the stop on error functionality, ensure that the mapping task is configured with the following conditions:
  • The
    Enable BigQuery Storage API
    property is not selected in the connection.
  • Bulk is selected as the write mode.
  • CSV or JSON is selected as the data format.
The mapping task maintains an independent error count for each transformation
and table partition
. When the error threshold exceeds in any of the transformations in a mapping, the mapping fails.
The following examples describe the mapping behavior based on the error threshold you set:
  • If you don't set a value or set the value to 0, the mapping fails at the first error.
  • If you set the value to 3, the mapping runs successfully for the first two errors and then fails when the third error occurs.
If you want to parameterize the
Stop on Errors
session property, you need to set the
$PMSessionErrorThreshold
property and its threshold value in the DTM custom configurations of the Secure Agent. This threshold value applies to all mapping tasks and you cannot override it with a different value in the mapping task properties.

Rules and guidelines

Consider the following rules and guidelines when you configure the stop on error functionality in a mapping task:
  • You cannot configure the stop on error functionality in the following mapping scenarios:
    • Mapping in advanced mode.
    • Mapping enabled with staging optimization.
    • Mapping enabled with
      SQL ELT optimization
      .
  • You cannot access the
    Error Rows File
    for the mapping task from the
    Results
    section.

0 COMMENTS

We’d like to hear from you!