Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Connections
  9. Sample Files

Transformation Validation

Transformation Validation

When you validate a mapping, the Developer tool performs validation on the transformations that are part of the streaming mapping.
The Developer tool performs the following validations:
  • You must use a Window transformation directly upstream from an Aggregator transformation in a streaming mapping.
  • You cannot use multiple Aggregator transformations in the same streaming pipeline.
  • If you connect the Window port from a Window transformation to an Aggregator transformation, you cannot connect the Window port to any downstream transformation.
  • If a mapplet contains an Aggregator transformation, you must include a Window transformation directly upstream from the mapplet.
  • You must use a Window transformation directly upstream from a Joiner transformation in a streaming mapping.
  • You cannot use multiple Joiner transformations in the same streaming pipeline.
  • A mapping contains only those transformations supported for streaming mappings.
  • You cannot use an Aggregator transformation and a passive Lookup transformation that is configured with an inequality lookup condition in the same pipeline.
  • You cannot use more than one passive Lookup transformation that is configured with an inequality lookup condition in the same pipeline.
  • You cannot use an Aggregator tranformation anywhere before a Joiner transformation in a streaming mapping.
  • If you connect the Window port from a Window transformation to a Joiner transformation, you cannot connect the Window port to any downstream transformation.
  • If a mapplet contains a Joiner transformation, you must include a Window transformation directly upstream from the mapplet.
  • The window port from the Window transformation cannot be connected to more than one downstream transformation.
  • A Window transformation is added between a streaming source and an Aggregator, or Joiner transformation.
  • A Window transformation has at least one upstream streaming source.
  • All Window transformations have a slide interval that is a multiple of the mapping batch interval.
  • The slide interval of a sliding Window transformation is less than window size.
  • The format of the parameter of the window size has the TimeDuration parameter type.
  • The window size and slide interval of a Window transformation are greater than 0.
  • The downstream Window transformation in the pipelines leading to a Joiner transformation has the same slide intervals.
  • A Window transformation is not added to a Logical Data Object mapping, REST mapping, or a mapplet.
  • If one pipeline leading to a Union transformation has a Window transformation, all streaming pipelines have a Window transformation. All downstream Window transformations in the pipelines leading to the Union transformations have the same slide intervals.
  • A Union transformation does not merge data from streaming and non-streaming pipelines.
  • A Joiner transformation does not join data from streaming and non-streaming pipelines.