Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • Edge Data Streaming 2.4.0
  • All Products

Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Edge Data Streaming
  3. Licenses
  4. Using Informatica Administrator
  5. Creating and Managing the Edge Data Streaming Service
  6. Edge Data Streaming Entity Types
  7. Edge Data Streaming Nodes
  8. Data Connections
  9. Working With Data Flows
  10. Managing the Edge Data Streaming Components
  11. Security
  12. High Availability
  13. Disaster Recovery
  14. Monitoring Edge Data Streaming Entities
  15. Appendix A: Troubleshooting
  16. Appendix B: Frequently Asked Questions
  17. Appendix C: Regular Expressions
  18. Appendix D: Command Line Program
  19. Appendix E: Configuring Edge Data Streaming to Work With a ZooKeeper Observer
  20. Appendix F: Glossary

User Guide

User Guide

Back Next

Static File Source Properties

Static File Source Properties

To add a static file source service, use the Static File source service type.
You can configure the following properties for the Static File source service type:
Entity Name
Name of the source service. Maximum length is 32 characters.
Description
Description of the source service. Maximum length is 256 characters.
Directory
Full path to the directory that contains the source files.
Position File Directory
Full path to the directory where EDS writes the position file. Specify this directory if the source file directory does not have write permissions.
File Name
Name of the source file or the Java regular expression that applies to the convention used to name the active source file.
File Name is Regular Expression
Indicates that the file name is a Java regular expression.
Number of Threads
The number of threads used to process the files in parallel. Default is 5.
Specify at a minimum value of 1.
Waiting Time (seconds)
The time in seconds from file modification after which the file should be processed. Specify a waiting time that is greater than the time taken by the application to write to the active file.
Default is 5 seconds.
Processed File Directory
Optional. Location where the files should be moved after they are processed.
If you do not specify a location, the files will not be moved.
The source service moves all files except the file that it is reading from, to the processed file directory, irrespective of whether it has sent the files to the target service or not.
Delimiter
End-of-line character sequence that marks the end of a line in the source file.
You can choose one of the following delimiters:
  • LF. Line feed character sequence.
  • CRLF. Carriage return line feed character sequence.
  • EOF. End-of-file (EOF) marker.
  • Custom. Custom character sequence.
Default is LF.
Custom Delimiter
Custom character sequence that you want to specify as delimiter for the file source services.
Enter a custom character sequence if you select
Custom
 as the delimiter.
Maximum Event Size
Maximum length of data that the source service can read at a time, in bytes. Default value is 8192. Minimum value is 1.
If you select
EOF
 as the delimiter, specify an event size that is greater than the file size. If the source file naming convention uses a regular expression, specify an event size that is greater than the size of all files that match the regular expression.
Retry on Failure
Indicates that the target service should try to write to the target if the operation fails the first time.
Number of Retries
The number of times the target service should retry to write to the source.
Delay between Retries
Time in milliseconds between successive retries.
Persist Data
Persist the data from the source locally so that the source service can continue to process data even if the target is not available.
Persistence Options
You can configure the following persistence options:
  • batchSizeInBytes
    . The batch size of the events in bytes after which the data is written to the persistent store. Specify a value of 0 if you want to write the data to the persistent store immediately. Default is 262144 bytes.
  • flushInterval
    . The interval in milliseconds after which the source services writes data to the persistent store. Default is 5000 milliseconds.
  • maxDataFiles
    . The maximum number of data files that the source service can keep in the persistent store. After the number of files exceed this number, a rollover occurs. Default is 10.
  • maxDataFileSize
    . The maximum size of each data file in bytes. When the file reaches the specified size, a file rollover occurs. Default is 1073741824.
  • maxQueueSizeInBytes
    . The maximum size of data that is unsent or has not received acknowledgment in bytes that can be stored in the persistent store. If the specified size is exceeded, the source service does not read data from the source. If you specify a value of 0, the unsent data can be as much as the value of
    maxDataFiles
     times
    maxDataFileSize
    . Default is unlimited size.
Statistics
You can collect and monitor the following statistics for the Static File source service:
  • Bytes Sent. The number of bytes sent by the source service.
  • Events Sent. The number of events sent by the source service.
  • Events to be Sent. The number of events that the source service is yet to send.
  • Send Rate (Per Sec.). The number of bytes sent every second.
  • Events Dropped. The number of events dropped while processing the source data.
  • Events not Delivered. Number of events that the source service did not deliver.
  • Files Written to Target. The number of files sent to the target service by the source service.
  • Files to be Processed. The number of files to be processed. This number includes the file that is being processed and the pending files.
  • Error while Moving Files. The number of errors that occured while moving the files.
  • File Ignored. The number of files ignored by the source service. For example, if the delimiter is EOF and the value of the Maximum Event Size property is less than the size of the file, this statistic is updated.
0 COMMENTS
We’d like to hear from you!