Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Transformation Validation

Transformation Validation

When you validate a mapping, the Developer tool performs validation on the transformations that are part of the streaming mapping.
The Developer tool performs the following validations:
  • A mapping cannot contain a transformation other than the Aggregator , Expression, Filter , Joiner, Lookup, Router, Union, and Window transformations.
  • A Window transformation is added between a streaming source and a Sorter, Aggregator, or Joiner transformation.
  • A Window transformation has at least one upstream streaming source.
  • All Window transformations have a slide interval that is a multiple of the mapping batch interval.
  • A Window transformation that is downstream from another Window transformation must have a slide interval that is a multiple of the slide interval of the upstream Window transformation.
  • The slide interval of a sliding Window transformation must be less than window size.
  • The format of the parameter of the window size must have the TimeDuration parameter type.
  • The window size and slide interval of a Window transformation must be greater than 0.
  • The downstream Window transformation in the pipelines leading to a Joiner transformation must have the same slide intervals.
  • A Window transformation cannot be added to a Logical Data Object mapping, REST mapping, or a mapplet.
  • If one pipeline leading to a Union transformation has a Window transformation, all streaming pipelines must have a Window transformation. All downstream Window transformations in the pipelines leading to the Union transformations must have the same slide interval.
  • A Union transformation cannot be used to merge data from streaming and non-streaming pipelines.
  • A Union transformation does not require a Window transformation between a streaming source and itself.

0 COMMENTS

We’d like to hear from you!