Table of Contents

Search

  1. Preface
  2. Introduction to Mass Ingestion
  3. Prepare
  4. Create
  5. Deploy
  6. Run
  7. Monitor
  8. Appendix A: infacmd mi Command Reference

Mass Ingestion Guide

Mass Ingestion Guide

Definition

Definition

The definition describes the type of mass ingestion specification that you want to create.
In the definition, you specify a name and a description and select the scenario for the specification. The scenario determines whether the specification ingests data from a relational database to a Hive target or from a relational database to an HDFS target. The scenario that you specify defines the properties that you can configure in the rest of the specification.
You can also enable incremental load. If you enable incremental load, you can ingest incremental data when you run the specification. After you enable incremental load, you must configure incremental load options when you configure the relational source and the Hive or HDFS target.
The following image shows the Definition page of the mass ingestion specification:
This screenshot shows the Definition page of the mass ingestion specification. On the Definition page, you can configure the following properties: Name, Description, and Scenario. In the top-right corner, you have the option Next to go to the next page, or the button X to discard the specification.
The following table describes the properties that you can configure in the definition:
Property
Description
Name
Required. Name of the mass ingestion specification. The name is not case sensitive and must be unique.
You cannot edit the name after you create the mass ingestion specification.
Description
Optional. Description of the specification.
Ingestion Scenario
Required. The scenario where data ingestion takes place. You can ingest data from a relational database to a Hive target or from a relational database to an HDFS target.
You cannot edit the ingestion scenario after you create the mass ingestion specification.
Enable Incremental Load
Optional. Allows you to ingest data using an incremental load when you run the specification.

0 COMMENTS

We’d like to hear from you!