Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Data Type Reference
  10. Appendix C: Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Example

Example

You want to compute average temperature of cities in a region. Add an Expression transformation to your streaming mapping.
Configure the following ports in the transformation:
  • city. Input port of string data type.
  • temperature. Input port of double data type.
  • avg. Output port that displays the average temperature.
  • count. Variable port of integer data type with expression ‘count+1’.
  • average. Variable port of double data type with expression (average * (count-1) + temperature/(count)
You partition the data by "city". The "average" corresponds to previously stored value for the "average" and is computed with each update in the value of "count". Since "count" is already incremented when "average" is evaluated, specify "count-1" in the expression to get the new average.
The following image shows an Expression transformation: