Table of Contents

Search

  1. Preface
  2. Introduction to Mass Ingestion
  3. Prepare
  4. Create
  5. Deploy
  6. Run
  7. Monitor
  8. Appendix A: infacmd mi Command Reference

Mass Ingestion Guide

Mass Ingestion Guide

Incremental Modes

Incremental Modes

If you enable incremental load in a mass ingestion specification, you must select a mode to ingest the data.
You can select append mode or overwrite mode. Append mode appends the incremental data to the data in the target. Overwrite mode overwrites the data in the target with the incremental data. You might use overwrite mode if the target is a staging area and not the final operational data store.
For example, the target might contain the following data from a previous run of the specification:
EmpID
EmpLastName
481530
'Basquez'
481531
'Savage'
481532
'Greene'
The following table shows the incremental data that the Spark engine ingests in the current run of the specification:
EmpID
EmpLastName
481533
'Caldwell'
481534
'Galloway'
If you use append mode, the incremental data is appended to the target. When the ingestion job is complete, the target contains the following data:
EmpID
EmpLastName
481530
'Basquez'
481531
'Savage'
481532
'Greene'
481533
'Caldwell'
481534
'Galloway'
If you use overwrite mode, the incremental data overwrites the target. When the ingestion job is complete, the target contains the following data:
EmpID
EmpLastName
481533
'Caldwell'
481534
'Galloway'

0 COMMENTS

We’d like to hear from you!