Database Ingestion and Replication

Back Next

Database Ingestion and Replication

can ingest and replicate data at scale from relational databases to multiple types of targets, including cloud data warehouses, data lakes, streaming systems, and other relational databases.

is a feature of the Cloud

Data Ingestion and Replication

service.

Database Ingestion and Replication

provides an easy-to-use interface for configuring and deploying

database ingestion and replication

tasks. You can start the task wizard from the

Ingest

panel on the Data Integration unified Home page or from the

New

option in the navigation bar of the Home page. A deployed task is called a job or job instance. To run and monitor

database ingestion and replication

jobs, you can use any of the following monitoring interfaces: the

My Jobs

page accessed from the navigation bar on the Home page, the

All Jobs

and

Running Jobs

pages in the Monitor service, or the

Overview

and

All Jobs

pages in Operational Insights.

Database Ingestion and Replication

can perform the following types of data load operations:

Initial load
. Loads source data read at a single point in time to a target in batch mode. After the data is loaded, the ingestion job ends. You can use this load type to materialize a target to which incremental changes will be sent later, to migrate from an on-premises database system to a cloud-based system, or to add data to a data lake or data warehouse.

Incremental load
. Replicates data changes continuously or until the job is stopped or ends. The job replicates the changes that have occurred since the last time it ran or from a specific start point. You can use this load type to keep data in separate reporting and analytics systems up to date so that you can make informed decisions for your business or organization based on the latest data. You can also use this load type to feed the latest changes to data warehouses and cloud data lakes for big data processing.

Initial and incremental load
. Performs an initial load of point-in-time data to the target and then automatically switches to replicating incremental data changes made to the same source tables on a continuous basis.

For more information about the sources and targets supported for each load type, see Database Ingestion and Replication source and target types.

database ingestion and replication

task automatically maps source tables and fields to target tables and fields based on name matching. You can define rules to customize the target table names.

Use cases

Database Ingestion and Replication source and target types

Database Ingestion and Replication architecture

Database Ingestion and Replication system requirements

Database Ingestion and Replication general limitations and guidelines

Database Ingestion and Replication sources - preparation and usage

Database Ingestion and Replication targets - preparation and usage

Schema drift handling

Apply an audit history of all source table DML changes to target tables

Ability to apply deletes as soft deletes on the target

Configuring a database ingestion and replication task

Deploying a database ingestion and replication task