Table of Contents

  1. Preface
  2. Introduction to Informatica Edge Data Streaming
  3. Licenses
  4. Using Informatica Administrator
  5. Creating and Managing the Edge Data Streaming Service
  6. Edge Data Streaming Entity Types
  7. Edge Data Streaming Nodes
  8. Data Connections
  9. Working With Data Flows
  10. Managing the Edge Data Streaming Components
  11. Security
  12. High Availability
  13. Disaster Recovery
  14. Monitoring Edge Data Streaming Entities
  15. Troubleshooting
  16. Frequently Asked Questions
  17. Regular Expressions
  18. Command Line Program
  19. Configuring Edge Data Streaming to Work With a ZooKeeper Observer
  20. Glossary

User Guide

User Guide

Configuring the HDFS Target Service Type to Work With a High Availability HDFS

You can configure the HDFS target service to write to a highly available HDFS. Perform the following tasks before you add the HDFS target service to the data flow and deploy it:
  1. Ensure that the HDFS client and HDFS server in the Hadoop distribution are of the same version.
  2. Edit properties in the /hadoop-<version>/etc/hadoop/hdfs-site.xml file. The following code sample shows some of the properties that you can add:
    <property>
      <name>dfs.nameservices</name>
      <value>NAMESERVICE1</value>
    </property>
    <property>
      <name>dfs.ha.namenodes.NAMESERVICE1</name>
      <value>MACHINE1,MACHINE2</value>
    </property>
    <property>
      <name>dfs.namenode.rpc-address.NAMESERVICE1.MACHINE1/name>
      <value>MACHINE1:8020</value>
    </property>
    <property>
      <name>dfs.namenode.rpc-address.NAMESERVICE1.MACHINE2</name>
      <value>MACHINE2:8020</value>
    </property>
    <property>
      <name>dfs.client.failover.proxy.provider.NAMESERVICE1</name>
      <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
    
  3. When you configure the HDFS target service, specify the following URL in the Destination property:
    hdfs://NAMESERVICE1/user/http.txt

Updated March 19, 2019


Explore Informatica Network