Snowflake Data Cloud Connector

Back Next

Capturing changed data from CDC sources

You can use Snowflake Data Cloud Connector to write changed data from a CDC source such as Oracle CDC and Oracle CDC V2 and write the changed data to a Snowflake target.

When you configure a mapping, add the CDC sources and then run the associated mapping task to write the changed data to Snowflake. If you define a column as required in the Snowflake target table, map a column in the CDC source to the required column in the Snowflake target in the mapping before you run the task.

When the mapping task processes the changed data from a CDC source, Snowflake Data Cloud Connector creates a state table in Snowflake. When the changed data is received from the CDC source, Snowflake Data Cloud Connector uploads the changed data to the staging table. Then, it generates a

Job_Id

and writes the

Job_Id

to the state table along with the restart information.

The connector then merges the stage table with the actual target table in Snowflake. Each time you run the mapping task, Snowflake Data Cloud Connector creates the state table, if it does not exist, to store the state information.

If you stop a CDC mapping task enabled with recovery strategy, or if it terminates unexpectedly, you can also restart it from the beginning of the task or resume it from where the task last ended from the job details page.

For more information about restarting and resuming a mapping task, see

Tasks

in the Data Integration documentation.

Snowflake Data Cloud Connector uses the following naming convention for the tables:

State table name:

Staging table name:

Restrictions

Consider the following restrictions when you capture changed data from CDC sources:

Source or lookup object in a CDC mapping: You can use a Snowflake target in a CDC mapping to write data from CDC sources. You can't use a Snowflake source or lookup in a CDC mapping.

Staging optimization property: The optimization property that you set for staging data in the DTM of the Secure Agent is not applicable. If you run a mapping with both CDC and staging property enabled, the mapping runs successfully. However, staging optimization is disabled and you can view a message logged in the session logs.
Recovery initialization error: When you run a CDC mapping to write data from a CDC source to a Snowflake target created at runtime, you might encounter the following error:; Error occured while initializing CCI State Handler com.informatica.cci.runtime.internal.utils.impl.CExceptionImpl: Internal error: Recovery Init failed; To avoid this error, you must have the grant permissions to create a table in Snowflake.
DTM error: When you run a CDC mapping that uses a Snowflake target connection configured with the ProcessConnDB and ProcessConnSchema parameters in the
Additional JDBC URL Parameters
field, the mapping might fail with the following DTM error:; Internal error. The DTM process terminated unexpectedly. Contact Informatica Global Customer Support.; To avoid this error, remove the ProcessConnDB and ProcessConnSchema parameters from the
Additional JDBC URL Parameters
field in the Snowflake Data Cloud connection. However, if you want to create temporary stage tables and recovery state tables using the ProcessConnDB and ProcessConnSchema parameters, verify that you have the necessary permissions in the required database and schema.