Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.2.2
  3. Part 2: Version 10.2.1
  4. Part 3: Version 10.2
  5. Part 4: Version 10.1.1
  6. Part 5: Version 10.1

Spark Structured Streaming

Spark Structured Streaming

Effective in version 10.2.2, Big Data Streaming uses Spark Structured Streaming to process streaming data.
Spark Structured Streaming is a scalable and fault-tolerant open source stream processing engine built on the Spark engine. It can handle late arrival of streaming events and process streaming data based on source timestamp.
The Spark engine runs the streaming mapping continuously. It reads the data, divides the data into micro batches, processes the micro batches, publishes the results, and then writes to a target.
For more information, see the
Informatica Big Data Streaming 10.2.2 User Guide
.


Updated July 27, 2020