Table of Contents

  1. Preface
  2. Introduction to Informatica Edge Data Streaming
  3. Licenses
  4. Using Informatica Administrator
  5. Creating and Managing the Edge Data Streaming Service
  6. Edge Data Streaming Entity Types
  7. Edge Data Streaming Nodes
  8. Data Connections
  9. Working With Data Flows
  10. Managing the Edge Data Streaming Components
  11. Security
  12. High Availability
  13. Disaster Recovery
  14. Monitoring Edge Data Streaming Entities
  15. Troubleshooting
  16. Frequently Asked Questions
  17. Regular Expressions
  18. Command Line Program
  19. Configuring Edge Data Streaming to Work With a ZooKeeper Observer
  20. Glossary

User Guide

User Guide

HDFS Target Service Properties

You can configure the following properties for the HDFS target service type:
Entity Name
Name of the HDFS target service. Maximum length is 32 characters.
Description
Description of the target service. Maximum length is 256 characters.
Destination
URI of the target file to which the target service writes data.
The HDFS target service type supports the following URI formats:
HDFS URI format
hdfs://<namenode-name>[:<port>]/<path>/<file-name>
MapR URI format
mapr://IP:port/path/filename
Where:
  • namenode-name is the host name or IP address of the HDFS NameNode.
  • port is the port number on which the HDFS NameNode listens for connections. You can omit the port number if you have configured HDFS to listen for connections on the default port number, 8020.
  • path and file-name compose the location of the target file in the target file system.
    The URI format is suitable for a standalone HDFS target service. The URI is also suitable for an HDFS target service that runs on a node that is not part of a high availability setup. To use multiple target service instances for load balancing or high availability, use variables in the URI.
Security Mode
Indicates whether HDFS has Kerberos authentication enabled. Select one of the following options:
  • Secure
  • Non Secure
User Principal
User principal to log in to the HDFS super user account. Specify a user principal in the following format:
user@DOMAIN.COM
Keytab Path
Location of the keytab files that HDFS uses.
Date and Time Format
The time stamp that is appended to the name of the file written to the target.
Rollover Size
Target file size, in gigabytes (GB), at which to trigger rollover. Default is 1.
Rollover Time
Length of time, in hours, to keep a target file active. After the time period has elapsed, the target service rolls the file over. A value of zero (0) means that the HDFS target service does not roll the file over based on time. Default is 0.
Force Synchronization
Flush the client's buffer to the disk device for every EDS event. If you enable forceful synchronization, the data written by the target service is visible in the HDFS target file system immediately. Forceful synchronization degrades the performance of EDS. For more information, see the Hadoop documentation. You can forcefully synchronize the data after a specific time interval of when a buffer size is reached. The event that occurs first triggers the forceful synchronization.
Default is to not synchronize forcefully.
Time Interval
Time in seconds after which forceful synchronization occurs. A value of zero (0) means that forceful synchronization occurs immediately. Default is 0.
Buffer Size
The buffer size in bytes after which forceful synchronization occurs. A value of zero (0) means that forceful synchronization occurs immediately. Default is 0.
Receive Idle Events
Indicates whether the target service should flush the data to the target file if the size of the data is less than the rollover size.
Time Out (Seconds)
Time in seconds after which the target service flushes the data to the target file.
Statistics
You can choose to monitor the following statistics for the HDFS target service:
  • Bytes Received. The number of bytes the target service receives.
  • Events Received. The number of events the target service receives.
  • Receive Rate (Per Sec). The number of bytes the target service receives every second.
  • Events Reassigned. The number messages that the source service resends to the target service when it has not received acknowledgment from the target service.
  • Files Rolled Over. Number of files rolled over.

Updated March 19, 2019


Explore Informatica Network