Table of Contents

Search

  1. Preface
  2. Introduction to Mass Ingestion
  3. Prepare
  4. Create
  5. Deploy
  6. Run
  7. Monitor
  8. infacmd mi Command Reference

Mass Ingestion Guide

Mass Ingestion Guide

Run Overview

Run Overview

After you deploy the mass ingestion specification, you run the mass ingestion specification.
Each time that you run the specification, the Mass Ingestion Service generates a new run instance for the specification. The Data Integration Service processes the specification and connects to the Hadoop environment. In the Hadoop environment, the Blaze, Spark, and Hive engines run individual jobs to ingest source tables into the target.
When you run the mass ingestion specification, the job ingests all data to the target. The mass ingestion solution does not provide the option to append only the data that has been recently updated to the target. When the Blaze, Spark, and Hive engines run the ingestion job, the existing data in the Hive or HDFS target is deleted and replaced with the data configured in the mass ingestion specification.
If the ingestion job stops responding or takes an excessive amount of time to complete, you can cancel the job in the Monitor tool.
For more information on canceling ingestion jobs, see Canceling Ingestion Jobs in the "Monitor" chapter.

0 COMMENTS

We’d like to hear from you!