Table of Contents

Search

  1. Preface
  2. Introduction to Mass Ingestion
  3. Prepare
  4. Create
  5. Deploy
  6. Run
  7. Monitor
  8. Appendix A: infacmd mi Command Reference

Mass Ingestion Guide

Mass Ingestion Guide

Transformation Override

Transformation Override

Configure the transformation override to override the mass transformations that are applied to all source tables in a mass ingestion specification. When you configure the override, you can configure parameters to apply to specific tables that you ingest from the relational database.
You can configure the transformation override in one of the following ways:
  • Before you configure the transformation override, configure mass transformations to apply parameters to all source tables. Then, navigate to each table and edit the applied parameter.
  • Clear the mass transformations, and configure specific parameters for each table.
The following image shows the Transformation Override page for an HDFS target:
This screenshots shows the Transformation Override page of the mass ingestion specification. Each row defines one table in the specification and each column defines a different parameter that you can configure. The cells define the specific parameter that you configure for a specific table. You edit the parameters by clicking an empty box or by clicking the pencil icon. In the top-right corner, you have the option Save to save the specification, or the button X to discard the specification.
You can edit the following parameters for each source table:
  • Incremental Column. If incremental load is enabled, edit the column that determines incremental data in the source tables.
  • Filter By. Filter the columns in the source tables.
  • Drop Columns. Drop columns from the source tables.
  • Replace Columns. Replace values in the source table columns.
  • Trim. Trim spaces from the beginning and end of values in source table columns.
  • Convert to Uppercase. Convert values in source table columns to uppercase.
  • Convert to Lowercase. Convert values in source table columns to lowercase.
  • Hive. If the specification ingests data to a Hive target, specify Hive properties for the target tables.
When you configure parameters for a source table, you cannot configure different parameters for different columns in the table. If you configure parameters for a column that does not exist in the table, the table fails to be ingested.

0 COMMENTS

We’d like to hear from you!