High availability for the Data Integration Service
High availability for the Data Integration Service
The Data Integration Service has the following high availability features:
Restart and failover
When a Data Integration Service process becomes unavailable, the Service Manager tries to restart the process or fails the process over to another node based on the service configuration.
Workflow recovery
When a Data Integration Service process shuts down unexpectedly, the Data Integration Service can automatically recover canceled workflow instances.
Data engineering recovery
If a Data Integration Service process or node fails unexpectedly, the Data Integration Service can recover jobs running on the Spark engine.
Data Integration Service grid
When you use a Data Integration Service grid, configure a different Data Integration Service on each node in the grid to allow load distribution and failover. The grid leverages the processing capabilities of all nodes at the same time.
Each grid can have one profiling warehouse. A profiling warehouse stores profiling and scorecard results for profiling and data discovery jobs.
To enable fault tolerance for the profile warehouse, complete the following tasks:
Replicate the profiling warehouse.
Set up a load balancer to store metadata in a backup database in case the primary database fails.