Preface
Introduction to Informatica Edge Data Streaming
- Informatica Edge Data Streaming Overview
- Edge Data Streaming Architecture
- Data Flow Model
- Edge Data Streaming High-Level Process
- Edge Data Streaming Data Flow Process
- Edge Data Streaming User Interface
- Example
Licenses
- Licenses Overview
- Viewing the License Details
- Updating a License
- Removing a License
Using Informatica Administrator
- Using Informatica Administrator Overview
- Manage Tab - Domain View
- Manage Tab - Services and Nodes View
  - Domain
  - Application Services
- Logs Tab
- Security Tab
- Managing Your Account
- Logging In
- Password Management
  - Changing Your Password
- Managing Users and Groups
  - Default Administrator
- Managing Users
  - Creating Users
  - Unlocking a User Account
- Managing Groups
  - Adding a Native Group
- Managing Privileges
  - Domain Privileges
- Roles
- Managing Roles
- Usage Collection Policy
  - Disabling Informatica Data Usage
Creating and Managing the Edge Data Streaming Service
- Creating and Managing the Edge Data Streaming Service Overview
- Creating the Edge Data Streaming Service
  - Creating the Edge Data Streaming Service in the Administrator Tool
  - Creating the Edge Data Streaming Service using Informatica Command Line Program
- Editing the Edge Data Streaming Service
Edge Data Streaming Entity Types
- Edge Data Streaming Entity Types Overview
- Aggregators
  - Aggregator Properties
- Built-in Source Service Types
- Built-in Target Service Types
- Built-in Transformation Types
- Using Parameters in Entity Properties
  - Setting Values for Parameters
  - Examples
- Custom Entity Types
- Advanced Configuration for Entities
- Configuring High Availability for Entities
Edge Data Streaming Nodes
- Edge Data Streaming Nodes Overview
- Node Groups
- Node Group Management Tab
- Working with Node Groups
Data Connections
- Data Connections Overview
- Ultra Messaging Data Connection
- WebSocket Data Connection
  - Configuring an External Load Balancer
  - WebSocket Data Connection Properties
Working With Data Flows
- Working With Data Flows Overview
- Types of Data Flows
- Creating a Data Flow
- Data Flow Design Tab
- Adding Entities to a Data Flow
- Edge Data Streaming Node Mapping
- Deploying a Data Flow
- Undeploying a Data Flow
- Undeploying and Deploying all Data Flows
  - Undeploying All Data Flows
  - Deploying All Data Flows
- Editing Data Flows and Entities
- Cloning a Data Flow
- Removing Data Flows and Entities
- Verifying Entity Properties
- Configuring Targets with Data Connection and Target Service Properties
  - Getting the Topic Name Assigned to a Connection
  - Getting the Receiver Type ID of a Target Service
- Getting Entity Alerts
Managing the Edge Data Streaming Components
- Managing the Edge Data Streaming Components Overview
- Administrator Daemon Management
- Edge Data Streaming Node Management
- Managing the Informatica Domain
Security
- Security Overview
- Authentication
- Component Security
- Secure Communication Within the Components
- Secure Data Storage
  - Update Encryption Keys
  - Updating Security Keys
- Secure Source Services and Target Services
- Privileges and Roles
High Availability
- High Availability Overview
- Restart and Failover
- Resilience
- Configuring High Availability in Edge Data Streaming
  - Design-Time High Availability
  - Run-Time High Availability
    - Entity High Availability
Disaster Recovery
- Disaster Recovery Overview
- Step 1: Replicate the EDS Installation
- Step 2: Back Up Data Flows
- Step 3: Back Up The Node Groups
- Step 4: Set Parameters
- Step 5: Replicate Source Files and Position Files
- Step 6: Restore EDS from the Disaster Recovery Site
Monitoring Edge Data Streaming Entities
- Monitoring Edge Data Streaming Entities Overview
- Viewing the Monitoring Tab
- Monitoring Tab Layout
  - System View
  - Grid View
- Edge Data Streaming Statistics
Appendix A: Troubleshooting
- Troubleshooting Licenses
- Troubleshooting Edge Data Streaming Node Issues
- Troubleshooting Administrator Daemon Issues
- Troubleshooting the Administrator Tool
- Troubleshooting Apache ZooKeeper
- Troubleshooting Component Connectivity Issues
- Troubleshooting Edge Data Streaming High Availability
- Troubleshooting Data Flows
- Troubleshooting Entities
- Troubleshooting Monitoring Tab Views
Appendix B: Frequently Asked Questions
- Frequently Asked Questions About Edge Data Streaming
Appendix C: Regular Expressions
Appendix D: Command Line Program
- Command Line Program Overview
- infacmd eds Plugin
- Running Commands
- infacmd Return Codes
- infacmd eds Command Reference
Appendix E: Configuring Edge Data Streaming to Work With a ZooKeeper Observer
Appendix F: Glossary
- Administrator Daemon
- data flow
- Informatica Administrator
- receiver type ID
- source service
- target service
- topic resolution domain
- unicast topic resolution daemon (LBMRD)
- EDS Node

User Guide

2.4.0
- 2.5.0

Back Next

HDFS Target Service Properties

You can configure the following properties for the HDFS target service type:

Entity Name

Name of the HDFS target service. Maximum length is 32 characters.

Description

Description of the target service. Maximum length is 256 characters.

Destination

URI of the target file to which the target service writes data.

The HDFS target service type supports the following URI formats:

HDFS URI format: hdfs://<namenode-name>[:<port>]/<path>/<file-name>
MapR URI format: mapr://IP:port/path/filename

Where:

namenode-name

is the host name or IP address of the HDFS NameNode.

port

is the port number on which the HDFS NameNode listens for connections. You can omit the port number if you have configured HDFS to listen for connections on the default port number, 8020.

path

and

file-name

compose the location of the target file in the target file system.

The URI format is suitable for a standalone HDFS target service. The URI is also suitable for an HDFS target service that runs on a node that is not part of a high availability setup. To use multiple target service instances for load balancing or high availability, use variables in the URI.

Security Mode

Indicates whether HDFS has Kerberos authentication enabled. Select one of the following options:

Secure

Non Secure

User Principal

User principal to log in to the HDFS super user account. Specify a user principal in the following format:

user@DOMAIN.COM

Keytab Path

Location of the keytab files that HDFS uses.

Date and Time Format

The time stamp that is appended to the name of the file written to the target.

Rollover Size

Target file size, in gigabytes (GB), at which to trigger rollover. Default is 1.

Rollover Time

Length of time, in hours, to keep a target file active. After the time period has elapsed, the target service rolls the file over. A value of zero (0) means that the HDFS target service does not roll the file over based on time. Default is 0.

Force Synchronization

Flush the client's buffer to the disk device for every EDS event. If you enable forceful synchronization, the data written by the target service is visible in the HDFS target file system immediately. Forceful synchronization degrades the performance of

EDS

. For more information, see the Hadoop documentation. You can forcefully synchronize the data after a specific time interval of when a buffer size is reached. The event that occurs first triggers the forceful synchronization.

Default is to not synchronize forcefully.

Time Interval

Time in seconds after which forceful synchronization occurs. A value of zero (0) means that forceful synchronization occurs immediately. Default is 0.

Buffer Size

The buffer size in bytes after which forceful synchronization occurs. A value of zero (0) means that forceful synchronization occurs immediately. Default is 0.

Receive Idle Events

Indicates whether the target service should flush the data to the target file if the size of the data is less than the rollover size.

Time Out (Seconds)

Time in seconds after which the target service flushes the data to the target file.

Statistics

You can choose to monitor the following statistics for the HDFS target service:

Bytes Received. The number of bytes the target service receives.

Events Received. The number of events the target service receives.

Receive Rate (Per Sec). The number of bytes the target service receives every second.

Events Reassigned. The number messages that the source service resends to the target service when it has not received acknowledgment from the target service.

Rename Saved Search

Table of Contents

User Guide

User Guide

HDFS Target Service Properties

HDFS Target Service Properties