English
  • Test Data Management 10.4.1
  • All Products

Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.4.1
  3. Part 2: Version 10.4.0
  4. Part 3: Version 10.2.2
  5. Part 4: Version 10.2.1
  6. Part 5: Version 10.2
  7. Part 6: Version 10.1.1
  8. Part 7: Version 10.1

Release Guide

Release Guide

Back Next

Stateful Computing on the Spark Engine

Stateful Computing on the Spark Engine

Effective in version 10.2, you can use window functions in an Expression transformation to perform stateful calculations on the Spark engine. Window functions operate on a group of rows and calculate a single return value for every input row. You can use window functions to perform the following tasks:
  • Retrieve data from previous or subsequent rows.
  • Calculate a cumulative sum based on a group of rows.
  • Calculate a cumulative average based on a group of rows.
For more information, see the "Stateful Computing on the Spark Engine" chapter of the
Big Data Management 10.2 User Guide
.