Table of Contents

Search

  1. Preface
  2. Introduction to Mass Ingestion
  3. Prepare
  4. Create
  5. Deploy
  6. Run
  7. Monitor
  8. Appendix A: infacmd mi Command Reference

Mass Ingestion Guide

Mass Ingestion Guide

Configuring the Transformation Override

Configuring the Transformation Override

Optionally, configure the transformation override to override the parameters that are applied to specific source tables.
For some parameters, you can click the arrow next to the text box to select columns from a list of columns that exist in the table.
  1. If you enabled incremental load in the definition of the mass ingestion specification, edit the incremental key column for a source table in the
    Incremental Column
    column.
  2. In the
    Filter By
    column, edit the filter clause.
  3. In the
    Drop Columns
    column, edit the list of columns or the regular expression.
  4. In the
    Trim
    column, edit the list of columns or the regular expression.
  5. In the
    Convert to Uppercase
    column, edit the list of columns or the regular expression.
  6. In the
    Convert to Lowercase
    column, edit the list of columns or the regular expression.
  7. In the
    Replace Columns
    column, click the pencil.
    1. For
      Replace
      , edit the list of columns or the regular expression.
    2. For
      Replace Criteria
      , select
      Entire String
      or
      Pattern
      .
    3. If you select Entire String, edit the new string.
    4. If you select Pattern, edit the previous string or pattern, or edit the new string.
  8. If the target is a Hive target, click the pencil in the
    Hive
    column and choose one of the following options:
    • Select
      Hive Options
      and enter the partition key, the cluster key, and the number of buckets.
    • Select
      DDL Query
      and edit the custom DDL query.

0 COMMENTS

We’d like to hear from you!