Spark Structured Streaming
Effective in version 10.2.2, Big Data Streaming uses Spark Structured Streaming to process streaming data.
Spark Structured Streaming is a scalable and fault-tolerant open source stream processing engine built on the Spark engine. It can handle late arrival of streaming events and process streaming data based on source timestamp.
The Spark engine runs the streaming mapping continuously. It reads the data, divides the data into micro batches, processes the micro batches, publishes the results, and then writes to a target.
For more information, see the
Informatica Big Data Streaming 10.2.2 User Guide.