Table of Contents

Search

  1. Preface
  2. Introduction to Big Data Streaming
  3. Big Data Streaming Configuration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Connections
  9. Data Type Reference
  10. Sample Files

Big Data Streaming User Guide

Big Data Streaming User Guide

Sources in a Streaming Mapping Overview

Sources in a Streaming Mapping Overview

You can access log file data, sensor data, Supervisory Control And Data Acquisition (SCADA) data, message bus data, Programmable logic controller (PLC) data on the Spark engine in the Hadoop environment.
You can create physical data objects to access the different types of data. Based on the type of source you are reading from, you can create the following data objects:
AmazonKinesis
A physical data object that represents data in an Amazon Kinesis Stream. Create an AmazonKinesis data object to read from an Amazon Kinesis Stream.
Azure Event Hub
A physical data object that represents data in Microsoft Azure Event Hubs data streaming platform and event ingestion service.
HBase
A physical data object that represents data in an HBase resource. Create an HBase data object with a read operation to perform an uncached lookup on HBase data.
JMS
A physical data object that accesses a JMS server. Create a JMS data object to read from a JMS server.
Kafka
A physical data object that accesses a Kafka broker. Create a Kafka data object to read from a Kafka broker.
MapRStreams
A MapRStreams data object is a physical data object that represents data in a MapR Stream. Create a MapRStreams data object to read from a MapR Stream.