Table of Contents

Search

  1. Preface
  2. Informatica Intelligent Cloud Services REST API
  3. Platform REST API version 2 resources
  4. Platform REST API version 3 resources
  5. Data Integration REST API
  6. Mass Ingestion Files REST API
  7. Mass Ingestion Streaming REST API
  8. RunAJob utility
  9. ParamSetCli utility
  10. REST API codes
  11. REST API resource quick references

REST API Reference

REST API Reference

Connection configuration for tasks with flat file as a source and as a target

Connection configuration for tasks with flat file as a source and as a target

When the source and target connection of the task is flat file, you can include the following fields and key-value pairs in the config array of the source and target connection:
Key
Type
Required
Description
File
String
Yes
Absolute path and name of the source file. Enter the base directory for multiple files mode.
initialPosition
String
Yes
Starting position to read data from the file to tail. Enter one of the following values:
  • Beginning of File. Read from the beginning of the file. Don't ingest any data that has already been rolled over.
  • Current Time. Read from the most recently updated part of the file. Don't ingest data that was rolled over or data in the file that was written.
rolloverPattern
String
-
File name pattern for the file that rolls over.
If the file to tail rolls over, the Secure Agent uses the file name pattern to identify files that have rolled over. If the Secure Agent stops during a file rollover, when it restarts, it picks up the file where it was left off.
You can use asterisk (*) and question mark (?) as wildcard characters to indicate that the files are rolled over in the same directory. For example,
${filename}.log.*
. Here, asterisk (*) represents the successive version numbers that would be appended to the file name.
tailingMode
String
Yes
Tail a file or multiple files based on the logging pattern. Enter one of the following values:
  • Single file. Tail one file.
  • Multiple files. Tail all the files indicated in the base directory. You can enter a regular expression to indicate the files to tail.
File Name
String
Yes
The name of the target file.
interimDirectory
String
Yes
Path to the staging directory on the Secure Agent.
rolloverSize
Integer
Yes
The file size, in KB, at which the task moves the file from the staging directory to the target.
You can enter a value between 1 and 2147483647.
rolloverEvents
Integer
Yes
Number of events or messages to accumulate before a file rollover.
You can enter a value between 1 and 2147483647.
rolloverTime
Integer
-
Length of time, in milliseconds, after which the target file rolls over.
You can enter a value between 1 and 2147483647.
edges
Array
-
Sequence of dataflow execution.

POST request example

To update a
streaming ingestion
task with a flat file source and target, you might send a request similar to the following example:
{ "name": "FileToFile", "description": "FileToFile_V2", "runtimeId": "01000025000000000003", "locationId": "5sJ0JDyJyWLlrosS5qJjsQ", "currentVersion": "2", "messageFormat": "binary", "nodes": [ { "name": "FileToFile_source", "type": "source", "connectionId": "0100000B000000000002", "transformationType": "", "config": [ { "key": "File", "value": "siagent.log" }, { "key": "initialPosition", "value": "Current Time" }, { "key": "rolloverPattern", "value": "" }, { "key": "tailingMode", "value": "Single file" } ] }, { "name": "FileToFile_target", "type": "target", "connectionId": "0100000B000000000002", "transformationType": "", "config": [ { "key": "File Name", "value": "testing.log" }, { "key": "interimDirectory", "value": "/home/agent/infa/test_file_target" }, { "key": "rolloverSize", "value": 100 }, { "key": "rolloverEvents", "value": 100 }, { "key": "rolloverTime", "value": 100 } ] } ], "edges": [ { "from": "FileToFile_source", "to": "FileToFile_target" } ], "runtimeOptions": { "maxLogSize": { "value": 10, "unit": "MB" }, "logLevel": "INFO" } }

0 COMMENTS

We’d like to hear from you!