Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Window Transformation
  8. Appendix A: Connections
  9. Appendix B: Monitoring REST API Reference
  10. Appendix C: Sample Files

Data Objects

Data Objects

Based on the type of source you are reading from or target you are writing to you can create physical data objects to access the different types of data.
The following table lists the data objects that you can include in streaming mappings and the read and write data object operations for each:
Data Object
Source
Data Object Operation
Target
Data Object Operation
Amazon Kinesis
Amazon Kinesis Stream
Read
Amazon Kinesis Firehose Delivery Stream
Write
Amazon S3
-
-
Amazon S3
Write
Azure Data Lake Store
-
-
Microsoft Azure Data Lake Store
Write
Azure Event Hub
Azure Event Hub Service
Read
Azure Event Hub Service
Write
Complex File
-
-
HDFS sequence file or binary file
Write
HBase
-
-
HBase tables
Write
JMS
JMS server
Read
JMS server
Write
Kafka
Kafka broker
Read
Kafka broker
Write
MapR Streams
MapR Stream
Read
MapR Stream
Write
Relational
-
-
Hive table
JDBC-compliant database
Write