Table of Contents

Search

  1. Preface
  2. Introduction to Edge Data Streaming Custom Entity Types
  3. Managing Custom Entity Types
  4. Custom Entities from Maven Archetypes
  5. REST APIs
  6. HTTP Request and Response Parameters
  7. Sample JSON Requests and Responses
  8. Glossary

Developer Guide

Developer Guide

Edge Data Streaming Custom Entity Types Overview

Edge Data Streaming Custom Entity Types Overview

Informatica Edge Data Streaming (Edge Data Streaming) is a distributed, extensible, and scalable data aggregation solution. You can use Edge Data Streaming to move large volumes of high-velocity data from multiple points of generation to one or more data targets in real time.
To transfer data, you use source services and target services. Source services read and publish data in the form of messages on a particular topic. Target services that subscribe to the topic receive the messages and write the messages to the target. Source services and target services run on Edge Data Streaming nodes.
You create the nodes on the data source hosts and data target hosts. You can also create the nodes on hosts that have access to the data sources and data targets. In Informatica Administrator (the Administrator tool), you create a data flow design by adding source services and target services to a data flow. You connect the two services, and you add any transformations that you want to apply.
You map the source service and target service to the Edge Data Streaming nodes on which they must run. When you deploy the data flow design, the Edge Data Streaming node to which you mapped the source service starts the source service. The Edge Data Streaming node to which you mapped the target service starts the target service. The source service collects data from the data source in real time. The target service receives and writes the data to the data target.
You can add multiple source services, multiple target services, and multiple transformations to a data flow.
The following image shows the flow of messages from a source service to multiple target services through transformations:
In the Administrator tool, you add source services, target services, or transformations to a data flow by adding appropriate entity types. For example, to receive and write messages to a Hadoop Distributed File System (HDFS) cluster, you use the built-in HDFS target service type.
Use built-in entity types for common use cases. Use built-in source service types for aggregating data from flat files, TCP sources, UDP sources, Syslog sources, and MQTT brokers. Use built-in target service types for writing data to HDFS, Apache Cassandra, Informatica PowerCenter®, Informatica RulePoint, and Informatica Ultra Messaging. Use a built-in regular expression filter and JavaScript and string insert transformation types to transform data.
If an entity type that you want is not available in Edge Data Streaming, create a custom entity type by using the Edge Data Streaming application programming interfaces (APIs). Use the APIs to create custom source service types, custom target service types, and custom transformation types.

0 COMMENTS

We’d like to hear from you!