Table of Contents

Search

  1. Preface
  2. Introduction to Data Engineering Streaming
  3. Data Engineering Streaming Administration
  4. Sources in a Streaming Mapping
  5. Targets in a Streaming Mapping
  6. Streaming Mappings
  7. Transformation in Streaming Mappings
  8. Window Transformation
  9. Appendix A: Connections
  10. Appendix B: Monitoring REST API Reference
  11. Appendix C: Sample Files

Data Objects

Data Objects

Based on the type of source you are reading from or target you are writing to you can create physical data objects to access the different types of data.
The following table lists the data objects that you can include in streaming mappings and the read and write data object operations for each:
Data Object
Data Object Operation
Amazon Kinesis
- Read. Amazon Kinesis Stream
-Write. Amazon Kinesis Firehose Delivery Stream
Amazon S3
- Read. Not applicable
- Write. Amazon S3
Azure Data Lake Store
- Read. Not applicable
-Write. Microsoft Azure Data Lake Store
Azure Event Hub
- Read. Azure Event Hub Service
- Write. Azure Event Hub Service
Complex File
- Read. Not applicable
- Write. HDFS sequence file or binary file
HBase
- Read. Not applicable
- Write. HBase tables
JMS
- Read. JMS server
- Write. JMS server
Kafka
- Read. Kafka broker
- Write. Kafka broker
MapR Streams
- Read. MapR Stream
- Write. MapR Stream
Relational
- Read. Not applicable
- Write. Hive table. JDBC-compliant database