Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Tumbling Window Transformation Example

Tumbling Window Transformation Example

You want to calculate the maximum value of a stock price every five minutes for stock prices collected over a five-minute time interval. You can use a tumbling Window transformation.
Create a mapping that reads stock prices and calculates the maximum value every five minute.
The following figure shows the example mapping:
The tumbling Window mapping example shows a Kafka input, a tumbling Window transformation, an Aggregator transformation, and a Kafka output.
You can use the following objects in your mapping:
Kafka Input
The input, Stock_Read, is a Kafka broker.
Window Transformation
The Window transformation, Window_Tumbling, accumulates data and returns a data group every five minute. Configure a window size of 5 minutes. The default slide interval is 5 minutes. The transformation streams data for five minutes and returns a data group every five minutes.
Aggregator
The Aggregator transformation calculates the maximum value of the stock price.
Kafka Ouptut
The output, Stock_Write, is a Kafka broker.
When you run the mapping, the Data Integration Service reads the data from the Kafka broker and passes it to the Window transformation. The window transformation groups the data and provides a data group every five minutes. The Aggregator transformation provides the maximum stock price. The output is written to a Kafka broker.

0 COMMENTS

We’d like to hear from you!