Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Edge Data Streaming
  3. Licenses
  4. Using Informatica Administrator
  5. Creating and Managing the Edge Data Streaming Service
  6. Edge Data Streaming Entity Types
  7. Edge Data Streaming Nodes
  8. Data Connections
  9. Working With Data Flows
  10. Managing the Edge Data Streaming Components
  11. Security
  12. High Availability
  13. Disaster Recovery
  14. Monitoring Edge Data Streaming Entities
  15. Appendix A: Troubleshooting
  16. Appendix B: Frequently Asked Questions
  17. Appendix C: Regular Expressions
  18. Appendix D: Command Line Program
  19. Appendix E: Configuring Edge Data Streaming to Work With a ZooKeeper Observer
  20. Appendix F: Glossary

User Guide

User Guide

Time Stamp Variable

Time Stamp Variable

Use the time stamp variable
#infa.timestamp
when you want to configure high availability for an HDFS service.
The nodes in a high availability pair share the same node name. The HDFS target service instance that runs on one node becomes active and acquires a lease to write to the HDFS file. If an active service fails, a standby instance becomes active. The new active service has to wait several minutes for the HDFS NameNode node to close the file and recover the lease before it can write to the target file. This scenario results in data loss.
To enable the instance on the secondary node to write data to the HDFS file system, use the
#infa.timestamp
variable in the
<file-name>
component of the HDFS URI.
The following example shows a URI that uses the
#infa.timestamp
variable:
hdfs://MyNamenode/app_data/logs/#infa.timestamp.log
#infa.timestamp
is the variable that EDS replaces with the system time in milliseconds.
The name of the file varies with system time. The primary service creates a file whose name is the current system time in milliseconds, and starts writing to the file. If an active service fails, another service does not have to wait for the HDFS NameNode node to revoke the lease on the file. The newly active service creates its own file with the current system time as its name and starts writing to the file.

0 COMMENTS

We’d like to hear from you!