Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Connections
  9. Data Type Reference
  10. Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Transformation Validation

Transformation Validation

When you validate a mapping, the Developer tool performs validation on the transformations that are part of the streaming mapping.
The Developer tool performs the following validations:
  • A mapping cannot contain a transformation other than the Aggregator , Expression, Filter , Joiner, Lookup, Router, Union, and Window transformations.
  • A Window transformation is added between a streaming source and a Sorter, Aggregator, or Joiner transformation.
  • A Window transformation has at least one upstream streaming source.
  • All Window transformations have a slide interval that is a multiple of the mapping batch interval.
  • A Window transformation that is downstream from another Window transformation must have a slide interval that is a multiple of the slide interval of the upstream Window transformation.
  • The slide interval of a sliding Window transformation must be less than window size.
  • The format of the parameter of the window size must have the TimeDuration parameter type.
  • The window size and slide interval of a Window transformation must be greater than 0.
  • The downstream Window transformation in the pipelines leading to a Joiner transformation must have the same slide intervals.
  • A Window transformation cannot be added to a Logical Data Object mapping, REST mapping, or a mapplet.
  • If one pipeline leading to a Union transformation has a Window transformation, all streaming pipelines must have a Window transformation. All downstream Window transformations in the pipelines leading to the Union transformations must have the same slide interval.
  • A Union transformation cannot be used to merge data from streaming and non-streaming pipelines.
  • A Union transformation does not require a Window transformation between a streaming source and itself.