Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Connections
  9. Sample Files

Sources in a Streaming Mapping Overview

Sources in a Streaming Mapping Overview

You can access log file data, sensor data, Supervisory Control And Data Acquisition (SCADA) data, message bus data, Programmable logic controller (PLC) data on the Spark SQL engine in the Hadoop environment.
You can create physical data objects to access the different types of data. Based on the type of source you are reading from, you can create the following data objects:
Amazon Kinesis
A physical data object that represents data in an Amazon Kinesis Stream. Create an Amazon Kinesis data object to read from an Amazon Kinesis Stream.
Azure Event Hubs
A physical data object that represents data in Microsoft Azure Event Hubs data streaming platform and event ingestion service.
HBase
A physical data object that represents data in an HBase resource. Create an HBase data object with a read operation to perform an uncached lookup on HBase data.
JMS
A physical data object that accesses a JMS server. Create a JMS data object to read from a JMS server.
Support for JMS source is deferred. Support will be reinstated in a future release.
Kafka
A physical data object that accesses a Kafka broker. Create a Kafka data object to read from a Kafka broker.
MapR Streams
A MapR Streams data object is a physical data object that represents data in a MapR Stream. Create a MapR Streams data object to read from a MapR Stream.
Support for MapR Streams data objects is deferred. Support will be reinstated in a future release.