Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Preface
  2. IDMC REST API
  3. REST API resource quick references
  4. Platform REST API version 2 resources
  5. Platform REST API version 3 resources
  6. Data Integration REST API
  7. Cloud Data Integration for PowerCenter (CDI-PC) REST API
  8. Data Ingestion and Replication REST API
  9. Customer Managed REST API
  10. RunAJob utility
  11. ParamSetCli utility
  12. REST API codes

REST API Reference

REST API Reference

Source attributes

Source attributes

The following table describes the source connection and task attributes in the request.
Field
Type
Description
Supported source types
connection
String
Name of the connection for the source system that the task uses to read data.
All sources
selectionRules
List
Optional object or table selection rules to select a subset of the source objects or tables for a task. By default, an Include rule that contains only the asterisk (*) wildcard character selects all objects on the source for an application ingestion and replication task or all tables in the source schema for a database ingestion and replication task. To narrow the source objects or tables that the task processes, you can add additional Include rules, Exclude rules, or a combination of both.
For example:
selectionRules: - include: TABLE_1 - exclude: TABLE_2
All sources
schema
String
The source schema that includes the source tables.
This option applies for
database ingestion and replication
sources except MongoDB.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
For MongoDB, to provide the database that stores collections, you must use this schema property instead of database.
journalName
String
For Db2 for i incremental loads in a
database ingestion and replication
task, the name of the journal that records the changes made to source tables.
Db2 for i
replicationSlotName
String
For PostgreSQL sources in a
database ingestion and replication
task, the unique name of a PostgreSQL replication slot.
PostgreSQL
replicationPlugin
String
For PostgreSQL sources in a
database ingestion and replication
task, the replication plugin. Options are:
  • pgoutput
  • wal2json
You can specify pgoutput only for PostgreSQL version 10 and later.
PostgreSQL
publication
String
For PostgreSQL sources in a
database ingestion and replication
task, the publication name used by the pgoutput plugin. Use this parameter only if you specified pgoutput as the replicationPlugin value.
PostgreSQL
restartPointForIncrementalLoad
String
For SAP ECC sources in an
application ingestion and replication
incremental load task and all sources in a
database ingestion and replication
incremental load or combined initial and incremental load task, the position in the source change stream or logs from which to start reading changes the first time the job runs.
Options are:
  • earliest
  • latest
For example: restartPointForIncrementalLoad: earliest, restartPointForIncrementalLoad: latest
For a
database ingestion and replication
task, default is latest.
For an
application ingestion and replication
task, the
earliest
option is not supported. The default is the restartPointForIncrementalLoadTimestamp value.
If you specify this parameter, do not also specify the restarPointForIncrementalLoadPosition or restartPointForIncrementalLoadTimestamp parameter.
SAP ECC, and
database ingestion and replication
sources
restartPointForIncrementalLoadPosition
Integer
For all sources in a
database ingestion and replication
incremental load or combined initial and incremental load task, the RBA position in source change stream or logs from which to start reading change records the first time the job runs.
For example:
restartPointForIncrementalLoadPosition: 0
If you specify this parameter, do not also specify the restarPointForIncrementalLoad or restartPointForIncrementalLoadTimestamp parameter.
Database ingestion and replication
sources
restartPointForIncrementalLoadTimestamp
Timestamp
For all sources in an
application ingestion and replication
incremental load task or
database ingestion and replication
combined initial and incremental load task, the date and time, including AM or PM, in the source change stream or logs from which to start reading change records the first time the job runs. For example: restartPointForIncrementalLoadTimestamp: 2021-08-18 02:50:00 PM
For an
application ingestion and replication
task, this parameter determines the default behavior.
If you specify this parameter, do not also specify the restarPointForIncrementalLoad or restartPointForIncrementalLoadPosition parameter.
All sources
cdcInterval
String (time interval)
For Salesforce sources in an
application ingestion and replication
incremental load or combined load task, the time interval for which the job runs to retrieve change records for CDC.
You can specify cdcIntervalDays, cdcIntervalHours, and cdcIntervalMins parameters.
Salesforce
fetchSize
Integer
For Salesforce sources in an
application ingestion and replication
initial load or incremental load task, the number of records that the job reads at a time from the source.
The default value for initial load jobs is 50000, and the default value for incremental load jobs is 2000.
If you specified Standard (REST) API for the salesforceAPI parameter, you must change the fetchSize value to 2000.
Salesforce
fetchSizeForInitialLoad
Integer
For Salesforce sources in an
application ingestion and replication
combined initial and incremental load task, the number of source records that the job reads at a time during the initial unload phase.
The default value is 50000.
If you specified Standard (REST) API for the salesforceAPI field, you must change the fetchSize value to 2000.
Salesforce
fetchSizeForIncrementalLoad
Integer
For Salesforce sources in an
application ingestion and replication
combined initial and incremental load task, the number of change records that the job can fetch at a time from the source during the incremental phase.
The default value is 2000.
For an initial load task or combined initial and incremental load task, if you specified Standard (REST) API for the salesforceAPI parameter, you must update the fetchSize value to 2000.
Salesforce
includeArchivedAndDeletedRows
Boolean
For Salesforce sources in an
application ingestion and replication
initial load or combined initial and incremental load task, controls whether the job can read archived and soft-deleted rows from the source during the initial load or unload phase of the combined load.
Default is false.
Salesforce
includeBase64Fields
Boolean
For an
application ingestion and replication
task with a Salesforce source, controls whether the job can replicate data from source fields that have the Base64 data type.
The default value is false.
Configure this parameter only if you set the salesforceAPI parameter to Standard (REST) API.
Salesforce
maximumBase64BodySize
Integer
For Salesforce sources in an
application ingestion and replication
task, the maximum body size of Base64 encoded data.
Default is 7 MB.
Pass this parameter only if you set the includeBase64Fields field to true.
Salesforce
includeViews
Boolean
Indicates whether to include views in the table counts and list of table names.
Options are:
  • true
    . Include views.
  • false
    . Do not include views.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
startDate
String
The start date for retrieving source records for the load operation.
Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing Cloud, Oracle Fusion Cloud
endDate
String
The end date for retrieving source records for the load operation.
Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing Cloud, Oracle Fusion Cloud,
batchSize
Integer
The batch size used when reading source records.
Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing Cloud, Oracle Fusion Cloud
mid
String
The unique Member Identification code assigned to your Salesforce Marketing Cloud account.
Salesforce Marketing Cloud
context
String
For SAP sources using the ODP extractor, the context name.
SAP ODP
cdcTechnique
String
For incremental load or combined initial and incremental load tasks, specifies the change data capture method.
Options are:
  • CDC Tables
    . Read data changes directly from the SQL Server CDC tables. This option applies only to SQL Server sources. Default is CDC Tables.
  • Log-based
    . Capture Inserts, Updates, and Deletes in near real time by reading the database transaction logs. Also capture DDL changes based on schema drift settings. For SQL Server sources, data changes are read from the SQL Server transaction log and the enabled SQL Server CDC tables. For Azure SQL Database sources, data changes are read from CDC tables only.
  • Query-based
    . Capture Inserts and Updates by using a SQL WHERE clause that points to a CDC query column. The query column is used to identify the rows that contain the changes made to the source tables since the beginning of the CDC interval.
Oracle, Microsoft SQL Server, MySQL, Db2 for LUW
cdcQueryBasedColumnType
String
For query-based CDC on source types, the column data type of the CDC query column in the source table. The column data type must combine the date and time. For Oracle, use Timestamp.
Oracle, Microsoft SQL Server, MySQL, Db2 for LUW
cdcQueryBasedColumnName
String
For query-based CDC on source types , the case-sensitive name of the CDC query column in the source table. The column must be present in the source table. Maximum length is 70 characters.
Oracle, Microsoft SQL Server, MySQL, Db2 for LUW
cdcQueryBasedDateColumn
String
For query-based CDC on source types that use a separate date column, the date column used to detect changed rows.
Oracle, Microsoft SQL Server, MySQL, Db2 for LUW
cdcQueryBasedTimeColumn
String
For query-based CDC on source types that use a separate time column, the time column used to detect changed rows.
Oracle, Microsoft SQL Server, MySQL, Db2 for LUW
CdcIntervalDays
Integer
The frequency of a query-based change data capture cycle, expressed in days. Sets the days portion of the CDC Interval property.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
CdcIntervalHours
Integer
The frequency of a query-based change data capture cycle, expressed in hours. Sets the hours portion of the CDC Interval field.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
CdcIntervalMins
Integer
The frequency of a query-based change data capture cycle, expressed in minutes. Sets the minutes portion of the CDC Interval field. Default value is 5 minutes.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
lobsEnabled
Boolean
Set this parameter to true if the source contains large object columns that you want to replicate to the target. Supported LOB data types are BLOB, CLOB, NCLOB, LONG, LONG RAW, and XML.
Oracle, PostgreSQL, Db2 for LUW, Db2 for i, SAP ABAP
workdayReplicationType
String
The type of web service to use for data retrieval.
Options are:
  • SOAP
    : Uses SOAP APIs to extract Workday data.
  • RaaS
    : Uses Workday Report-as-a-Service (RaaS) to extract source data from custom objects and fields through custom reports. You can use Workday RaaS only in initial load jobs.
Workday
workdayReportsNumber
Integer
If you use the RaaS API, specify the number of reports you want to extract from the source.
Workday
workdayReportName
String
If you use the RaaS API and you want to extract a single report, specify the name or URL of the custom report you want to read from the source.
Workday
workdayReportsConfigFile
String
If you use the RaaS API and you want to extract multiple reports, specify the path to the CSV file that you created for the list of custom reports that you want to read from the source.
Workday
outputType
String
The format in which you want the data to be stored on the target.
Workday
extractNonDefault
Boolean
Indicates whether extract source fields that do not contain any default value.
Workday
oracleFusionInterfaceType
String
The type of interface used to extract data from the source.
Options are:
  • REST. Extracts data from various applications of Oracle Fusion such as ERP, SCM, HCM, Sales, and Services, and transfer data to the target.
  • BICC. Extracts bulk data from the source to the target.
Oracle Fusion Cloud
oracleFusionModuleName
String
The name of the Oracle Fusion Cloud Applications Suite module to extract data.
Oracle Fusion Cloud
oracleFusionEnableChildObjects
Boolean
Indicates whether to include child objects from the Oracle Fusion Cloud source in the extraction.
Oracle Fusion Cloud
The following table describes the source advanced attributes in the request:
Field
Type
Description
Supported source types
salesforceAPI
String
For Salesforce sources in an
application ingestion and replication
task, the type of Salesforce API that you want to use to retrieve the source data. Options are:
  • Standard (REST) API
    . Replicates source fields of Base64 data type.
  • Bulk API 2.0
    . Excludes replication of source fields of Base64 data type. To use Bulk API 2.0, pass the value bulkv2 in the payload.
Default is Bulk API 2.0.
Salesforce
salesforceBulkV2Partitioning
Boolean
For initial and combined load jobs that use the Salesforce Bulk API 2.0, set this parameter to true to use partitioning when reading source data.
Salesforce
salesforceBulkV2RecordsPerPartition
Integer
For initial and combined load jobs that use the Salesforce Bulk API 2.0, the number of records to process per partition when reading data.
Salesforce
salesforceBulkV2MaxParallelPartitions
Integer
The maximum number of partition threads that can be used to query the source for data in parallel during initial load processing or the unload phase of combined jobs that use the Salesforce Bulk API 2.0. Use this property to control the number of source partition queries that can be executed against the source at the same time.
For example, if a table contains data in 100 partitions, all 100 partitions are queried at the same time by default. However, you can use this property to reduce the number of concurrent queries.
Default value is equal to the total number of partitions.
Salesforce
enableSourcePartitioning
Boolean
Controls whether to use multiple partitions to query for the source data to be unloaded in parallel during initial load processing or the unload phase of combined load jobs. Set this property to true to use multiple partitions with one query per partition.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
sourcePartitionCount
Integer
The number of partitions to use for querying for source data in parallel during initial load jobs and the unload phase of combined load jobs, when the Enable Partitioning custom property is selected.
Default is 5.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
jdbcPartitioningTechnique
String
The partitioning technique used to distribute source rows across partitions. Valid values are uniform, which distributes primary keys evenly across partitions, and heuristic, which estimates key ranges for faster calculation.
Default is uniform.
Oracle, Microsoft SQL Server, SAP HANA
sourceMaxParallelPartitions
Integer
The maximum number of partition reader threads that can be used to query the source in parallel during initial load processing or the unload phase of combined jobs.
Oracle, Microsoft SQL Server, SAP HANA
unloadHelperThreadCount
Integer
The number of unload helper threads allocated to an initial load job or the unload phase of a combined job to convert the unloaded data rows into a canonical format that can be passed to the writer.
Default value is 2.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
readEventBatchSize
Integer
The number of payload events written in batch to the internal event queue during CDC processing. When the event queue is implemented as an internal ring buffer, this value is the number of payload events that the reader writes to a single internal buffer slot.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
readerHelperThreadCount
Integer
The number of reader helper threads used during CDC processing to convert change data into a canonical format that can be passed to the target.
Default value is 3.
You can enter a larger value to allow more threads to be available for performing conversion processing in parallel.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
readerInputIsPersisted
Boolean
Set this parameter to true to persistently store transaction data in a disk buffer so that the data can be consumed continually, even when the writing of data to the target is slow or delayed.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA
unloadFlashbackDisabled
Boolean
Set this parameter to true to disable the use of Oracle Flashback when fetching data from the database.
Oracle, Microsoft SQL Server, MySQL
salesforceBulkV2Partitioning
Boolean
For initial and combined load jobs that use the Salesforce Bulk API 2.0, set this parameter to true to use partitioning when reading source data.
Salesforce
salesforceBulkV2RecordsPerPartition
Integer
For initial and combined load jobs that use the Salesforce Bulk API 2.0, the number of records to process per partition when reading data.
Salesforce
salesforceBulkV2MaxParallelPartitions
Integer
The maximum number of partition threads that can be used to query the source for data in parallel during initial load processing or the unload phase of combined jobs that use the Salesforce Bulk API 2.0. Use this property to control the number of source partition queries that can be executed against the source at the same time.
For example, if a table contains data in 100 partitions, all 100 partitions are queried at the same time by default. However, you can use this property to reduce the number of concurrent queries.
Default value is equal to the total number of partitions.
Salesforce
readerHistoricalCacheUseCloud
Boolean
Indicates whether the reader historical cache is stored in cloud storage.
Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for LUW, Db2 for i, Db2 for z/OS, SAP HANA

0 COMMENTS

We’d like to hear from you!