Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Transformation in Streaming Mappings
  8. Window Transformation
  9. Appendix A: Connections
  10. Appendix B: Monitoring REST API Reference
  11. Appendix C: Sample Files

Rules and Guidelines for the Window Transformations

Rules and Guidelines for the Window Transformations

Certain transformations are valid with restrictions with the Window transformation. The following table describes the rules and guidelines for transformations:
Transformation
Rules and Guidelines
Aggregator
The Aggregator transformation is a multi-group active transformation.
The following rules apply to Aggregator transformations:
  • You must use a Window transformation directly upstream from an Aggregator transformation in a streaming mapping.
  • If you connect the Window port from a Window transformation to an Aggregator transformation, you cannot connect the Window port to any downstream transformation.
  • If a mapplet contains an Aggregator transformation, you must include a Window transformation directly upstream from the mapplet.
  • You cannot perform the group by aggregations on date/time data type port marked as a Window port. If you want to perform aggregations on the date/time data type port, you must create a date/time data type port with the timestamp values, and then perform the group by aggregations on the newly created data type port.
Joiner
The Joiner transformation is a multi-group active transformation.
The following rules apply to Joiner transformations:
  • You must use a Window transformation directly upstream from a Joiner transformation in a streaming mapping.
  • You must use a Window transformation between the streaming source and any Joiner transformation in a streaming mapping.
  • The upstream Window transformations in pipelines to a Joiner transformation must have the same slide intervals.
  • The downstream Window transformations in pipelines to a Joiner transformation must have the same slide intervals.
  • If you connect the Window port from a Window transformation to a Joiner transformation, you cannot connect the Window port to any downstream transformation.
  • If a mapplet contains a Joiner transformation, you must include a Window transformation directly upstream from the mapplet.
Lookup
A Lookup transformation does not require a Window transformation between a streaming source and itself.
Rank
A Rank transformation in a streaming mapping or a mapplet must have an upstream Window transformation.
Sorter
The Sorter transformation is an active transformation.
The following rules apply to Sorter transformations:
  • You must use a Window transformation between the streaming source and the Sorter transformation in a streaming mapping.
  • The Window transformation upstream from an Aggregator transformation will be ignored if the mapping contains a Sorter transformation.
Union
The following rules apply to Union transformations:
  • A Union transformation does not require a Window transformation between a streaming source and itself.
  • If one pipeline leading to a Union transformation has a Window transformation, all streaming pipelines must have a Window transformation. All downstream Window transformations in the pipelines leading to the Union transformations must have the same slide intervals.
Window
The following rules apply to Window transformations:
  • You cannot add a Window transformation to a Logical Data Object mapping or mapplet.
  • A Window transformation must have at least one upstream streaming source.
  • All Window transformations must have a slide interval that is a multiple of the mapping batch interval.
  • A Window transformation that is downstream from another Window transformation must have a slide interval that is a multiple of the slide interval of the upstream Window transformation.
  • The slide interval of a sliding Window transformation must be less than window size.
  • The format of the parameter of the window size must have the TimeDuration parameter type.
  • The window size and the slide interval of a Window transformation must be greater than 0.
  • The window port from the Window transformation cannot be connected to more than one downstream transformation.
  • A Window transformation must be added between a streaming source and an Aggregator, or a Joiner transformation.
  • A Window transformation must not be added to a Logical Data Object mapping, REST mapping, or a mapplet.

0 COMMENTS

We’d like to hear from you!