stage process transfers source data from a landing table to the staging table associated with a particular base object. The complete stage process occurs in the
MDM Hub
.
The following image shows the
MDM Hub
stage process where source data is transferred from a landing table to a staging table and reject table:
Before you perform the
MDM Hub
stage process, load data from the external data source to the landing tables. You define mappings between the landing and staging tables. Mappings link a source column in the landing table with a target column in the staging table. If you need to cleanse data before the
MDM Hub
moves the data to the staging table, configure data cleansing in the mapping. When you run the stage job, the
MDM Hub
transfers data based on the mappings from the columns in a landing table to the columns in a staging table.
During the stage process, the
MDM Hub
processes one block of 250 records at a time. If there is a problem with a record in the block, the
MDM Hub
moves the record to the reject table. A record could be rejected because the value of a cell is too long, or because the record’s update date is later than the current date. After the
MDM Hub
moves the rejected record, the
MDM Hub
stops processing the remaining records in the block, and moves on to other blocks. When the stage process completes, run the job again. The records that were not processed are picked up again and processed.
You can retain the history of data in landing tables. When you enable audit trail for a staging table, the landing table data is archived in a raw table. The MDM Hub retains the landing table data in the raw table for the number of stage job runs or the retention period that you configure. After the
MDM Hub
reaches the number of stage job runs or the retention period that you specify, it retains one record for each primary key of the source object in the raw table.
You can configure the
MDM Hub
to identify new and updated records in the landing table. If you enable delta detection for a staging table, the
MDM Hub
processes new and updated records and ignores unchanged records.
The
MDM Hub
can transfer data from one landing table to multiple staging tables. However, each staging table receives data from only one landing table.
The stage process prepares data for the load process, which loads data from the staging table into a target base object.