Table of Contents

Search

  1. Preface
  2. Introduction to Intelligent Streaming
  3. Intelligent Streaming Configuration
  4. Connections
  5. Sources and Targets in a Streaming Mapping
  6. Intelligent Streaming Mappings
  7. Window Transformation
  8. Data Type Reference

MapRStreams Data Objects

MapRStreams Data Objects

A MapRStreams data object is a physical data object that represents data in a MapR Stream. After you create a MapRStreams connection, create a MapRStreams data object to read from or write data to MapR Streams.
Before you create and use MapR Stream data objects in Streaming mappings, complete the required prerequisites.
For more information about the prerequisite tasks, see the
Informatica Big Data Management Cluster Integration Guide
.
When you configure the MapRStreams data object, specify the stream name that you read from in the
/pathname:topic name
format. You can specify the stream name or use a regular expression for the stream name pattern. The regular expression that you specify applies to the topic name and not the path name. When you write data to a MapR stream, specify the name of the stream that you publish to.
After you create a MapRStreams data object, create a read data object or a write data object operation. You can then add the data object read operation as a source and the data object write operation as a target in Streaming mappings.
When you configure the data operation properties, specify the format in which the MapR Streams data object reads or writes data. You can specify XML, JSON, or Avro as format. When you specify XML format, you must provide an XSD file. When you specify JSON or Avro format, you must provide a sample file.
You can pass any payload format directly from source to target in Streaming mappings. You can project columns in binary format pass a payload from source to target in its original form or to pass a payload format that is not supported.
Streaming mappings can read, process, and write hierarchical data. You can use array, struct, and map complex data types to process the hierarchical data. You assign complex data types to ports in a mapping to flow hierarchical data. Ports that flow hierarchical data are called complex ports.
For more information about processing hierarchical data, see the
Informatica Big Data Management User Guide
.