Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Sliding Window Transformation Example

Sliding Window Transformation Example

You want to calculate the maximum value of a stock price every minute for stock prices collected over a five-minute time interval. You can use a sliding Window transformation.
Create a mapping that reads stock prices and calculates the maximum value every minute.
The following image shows the example mapping:
The sliding Window mapping example shows a Kafka input, a sliding Window transformation, an Aggregator transformation, and a Kafka output.
You can use the following objects in your mapping:
Kafka Input
The input, Stock_Read, is a Kafka broker.
Window Transformation
The Window transformation, Window_Sliding, accumulates data and returns a data group every minute. Configure a window size of 5 minutes and a slide interval of 1 minute. The transformation streams data for five minutes and returns a data group every minute.
Aggregator
The Aggregator transformation calculates the maximum value of the stock price.
Kafka Ouptut
The output, Stock_Write, is a Kafka broker.
When you run the mapping, the Data Integration Service reads the data from the Kafka broker and passes it to the Window transformation. The window transformation groups the data and provides a data group every minute. The Aggregator transformation provides the maximum stock price. The output is written to a Kafka broker.

0 COMMENTS

We’d like to hear from you!