Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Sample Files

Advanced Properties

Advanced Properties

The following table describes the advanced properties that you configure for Hive targets in streaming mappings:
Property
Description
Tracing level
Controls the amount of detail in the mapping log file.
Target Schema Strategy
Type of target schema strategy for the target table.
You can select one of the following target schema strategies:
  • RETAIN. The Data Integration Service retains the existing target schema.
  • CREATE. The Data Integration Service drops the target table at run time and replaces it with a table based on a target table that you identify.
  • APPLYNEWCOLUMNS. The Data Integration Service alters the target table by applying new columns from the associated data object or mapping flow to the target table.
  • APPLYNEWSCHEMA. The Data Integration Service alters the target table and applies the new schema from the associated data object or mapping flow to the target table.
  • FAIL. The Data Integration Service fails the mapping if the target schema from the mapping flow is different from the schema of the target table.
  • Assign Parameter. You can assign a parameter to represent the value for the target schema strategy and then change the parameter at run time.
DDL Query
The DDL query based on which the Data Integration Service creates or replace the target table at run time.
You cannot create a custom DDL query that creates or replaces a Hive table at run time in streaming mappings.
Truncate target table
Truncates the target before loading data.
If the mapping target is a Hive partition table, you can choose to truncate the target table only with Hive version 0.11.
Truncate Hive Target Partition
Truncates the external table before loading data.
Streaming mappings do not support this property. The property is ignored.
PreSQL
The SQL command the Data Integration Service runs against the target database before it reads the source.
The Developer tool does not validate the SQL.
PostSQL
The SQL command that the Data Integration Service runs against the target database after it writes to the target.
The Developer tool does not validate the SQL.
Streaming mappings do not support Post-SQL queries. Post-SQL queries in streaming mappings are ignored.
Maintain Row Order
Maintain row order while writing data to the target. Select this option if the Data Integration Service should not perform any optimization that can change the row order.
When the Data Integration Service performs optimizations, it might lose the row order that was established earlier in the mapping. You can establish row order in a mapping with a sorted flat file source, a sorted relational source, or a Sorter transformation. When you configure a target to maintain row order, the Data Integration Service does not perform optimizations for the target.

0 COMMENTS

We’d like to hear from you!