Administrator
- Administrator
- All Products
Field | Type | Description | Supported source types |
|---|---|---|---|
connection | String | Name of the connection for the source system that the task uses to read
data. | All sources |
selectionRules | List | Optional object or table selection rules to select a subset of the source
objects or tables for a task. By default, an Include rule that contains only the
asterisk (*) wildcard character selects all objects on the source for an application
ingestion and replication task or all tables in the source schema for a database
ingestion and replication task. To narrow the source objects or tables that the task
processes, you can add additional Include rules, Exclude rules, or a combination of
both. For example: selectionRules: - include: TABLE_1 - exclude:
TABLE_2 | All sources |
schema | String | The source schema that includes the source tables. This option applies for
database ingestion and replication sources except MongoDB. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA For MongoDB, to provide the
database that stores collections, you must use this schema property instead of
database. |
journalName | String | For Db2 for i incremental loads in a database ingestion and replication task, the name of the journal that records the changes made to
source tables. | Db2 for i |
replicationSlotName | String | For PostgreSQL sources in a database ingestion and replication task, the unique name of a PostgreSQL replication slot. | PostgreSQL |
replicationPlugin | String | For PostgreSQL sources in a database ingestion and replication task, the replication plugin. Options are:
You can specify pgoutput
only for PostgreSQL version 10 and later. | PostgreSQL |
publication | String | For PostgreSQL sources in a database ingestion and replication task, the publication name used by the pgoutput plugin. Use this
parameter only if you specified pgoutput as the replicationPlugin value. | PostgreSQL |
restartPointForIncrementalLoad | String | For SAP ECC sources in an application ingestion and replication incremental load task and all sources in a database ingestion and replication incremental load or combined initial and
incremental load task, the position in the source change stream or logs from which
to start reading changes the first time the job runs. Options are:
For example: restartPointForIncrementalLoad: earliest,
restartPointForIncrementalLoad: latest For a database ingestion and replication task, default is latest. For an
application ingestion and replication task, the earliest option is not supported. The default
is the restartPointForIncrementalLoadTimestamp value.If you specify this
parameter, do not also specify the restarPointForIncrementalLoadPosition or
restartPointForIncrementalLoadTimestamp parameter. | SAP ECC, and database ingestion and replication sources |
restartPointForIncrementalLoadPosition | Integer | For all sources in a database ingestion and replication incremental load or combined initial and incremental load task,
the RBA position in source change stream or logs from which to start reading change
records the first time the job runs. For
example: restartPointForIncrementalLoadPosition: 0 If you specify this
parameter, do not also specify the restarPointForIncrementalLoad or
restartPointForIncrementalLoadTimestamp parameter. | Database ingestion and replication sources |
restartPointForIncrementalLoadTimestamp | Timestamp | For all sources in an application ingestion and replication incremental load task or database ingestion and replication combined initial and incremental load task, the date and time,
including AM or PM, in the source change stream or logs from which to start reading
change records the first time the job runs. For example:
restartPointForIncrementalLoadTimestamp: 2021-08-18 02:50:00 PMFor an application ingestion and replication task, this parameter determines the
default behavior.If you
specify this parameter, do not also specify the restarPointForIncrementalLoad or
restartPointForIncrementalLoadPosition parameter. | All sources |
cdcInterval | String (time interval) | For Salesforce sources in an application ingestion and replication incremental load or combined load task, the time interval for
which the job runs to retrieve change records for CDC. You can specify
cdcIntervalDays, cdcIntervalHours, and cdcIntervalMins parameters. | Salesforce |
fetchSize | Integer | For Salesforce sources in an application ingestion and replication initial load or incremental load task, the number of records that
the job reads at a time from the source. The default value for initial load jobs
is 50000, and the default value for incremental load jobs is 2000. If you
specified Standard (REST) API for the salesforceAPI parameter, you must change the
fetchSize value to 2000. | Salesforce |
fetchSizeForInitialLoad | Integer | For Salesforce sources in an application ingestion and replication combined initial and incremental load task, the number of source
records that the job reads at a time during the initial unload phase. The default
value is 50000. If you specified Standard (REST) API for the salesforceAPI
field, you must change the fetchSize value to 2000. | Salesforce |
fetchSizeForIncrementalLoad | Integer | For Salesforce sources in an application ingestion and replication combined initial and incremental load task, the number of change
records that the job can fetch at a time from the source during the incremental
phase.The default value is 2000. For an initial load task or combined
initial and incremental load task, if you specified Standard (REST) API for the
salesforceAPI parameter, you must update the fetchSize value to 2000. | Salesforce |
includeArchivedAndDeletedRows | Boolean | For Salesforce sources in an application ingestion and replication initial load or combined initial and incremental load task,
controls whether the job can read archived and soft-deleted rows from the source
during the initial load or unload phase of the combined load.Default is
false. | Salesforce |
includeBase64Fields | Boolean | For an application ingestion and replication task with a Salesforce source, controls whether the job can
replicate data from source fields that have the Base64 data type. The default
value is false. Configure this parameter only if you set the salesforceAPI
parameter to Standard (REST) API. | Salesforce |
maximumBase64BodySize | Integer | For Salesforce sources in an application ingestion and replication task, the maximum body size of Base64 encoded data. Default is
7 MB. Pass this parameter only if you set the includeBase64Fields field to
true. | Salesforce |
includeViews | Boolean | Indicates whether to include views in the table counts and list of table names.
Options are:
| Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |
startDate | String | The start date for retrieving source records for the load operation. | Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP
Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing
Cloud, Oracle Fusion Cloud |
endDate | String | The end date for retrieving source records for the load operation. | Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP
Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing
Cloud, Oracle Fusion Cloud, |
batchSize | Integer | The batch size used when reading source records. | Salesforce, Marketo, Microsoft Dynamics 365, SAP (ODP Extractor, OData V2, SAP
Mass Ingestion), SAP (ABAP), NetSuite, Workday, ServiceNow, Salesforce Marketing
Cloud, Oracle Fusion Cloud |
mid | String | The unique Member Identification code assigned to your Salesforce Marketing
Cloud account. | Salesforce Marketing Cloud |
context | String | For SAP sources using the ODP extractor, the context name. | SAP ODP |
cdcTechnique | String | For incremental load or combined initial and incremental load tasks, specifies
the change data capture method. Options are:
| Oracle, Microsoft SQL Server, MySQL, Db2 for LUW |
cdcQueryBasedColumnType | String | For query-based CDC on source types, the column data type of the CDC query
column in the source table. The column data type must combine the date and time. For
Oracle, use Timestamp. | Oracle, Microsoft SQL Server, MySQL, Db2 for LUW |
cdcQueryBasedColumnName | String | For query-based CDC on source types , the case-sensitive name of the CDC query
column in the source table. The column must be present in the source table. Maximum
length is 70 characters. | Oracle, Microsoft SQL Server, MySQL, Db2 for LUW |
cdcQueryBasedDateColumn | String | For query-based CDC on source types that use a separate date column, the date
column used to detect changed rows. | Oracle, Microsoft SQL Server, MySQL, Db2 for LUW |
cdcQueryBasedTimeColumn | String | For query-based CDC on source types that use a separate time column, the time
column used to detect changed rows. | Oracle, Microsoft SQL Server, MySQL, Db2 for LUW |
CdcIntervalDays | Integer | The frequency of a query-based change data capture cycle, expressed in days.
Sets the days portion of the CDC Interval property. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2
for z/OS, SAP HANA |
CdcIntervalHours | Integer | The frequency of a query-based change data capture cycle, expressed in hours.
Sets the hours portion of the CDC Interval field. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2
for z/OS, SAP HANA |
CdcIntervalMins | Integer | The frequency of a query-based change data capture cycle, expressed in minutes.
Sets the minutes portion of the CDC Interval field. Default value is 5
minutes. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2
for z/OS, SAP HANA |
lobsEnabled | Boolean | Set this parameter to true if the source contains large object columns that you
want to replicate to the target. Supported LOB data types are BLOB, CLOB, NCLOB,
LONG, LONG RAW, and XML. | Oracle, PostgreSQL, Db2 for LUW, Db2 for i, SAP ABAP |
workdayReplicationType | String | The type of web service to use for data retrieval. Options are:
| Workday |
workdayReportsNumber | Integer | If you use the RaaS API, specify the number of reports you want to extract from
the source. | Workday |
workdayReportName | String | If you use the RaaS API and you want to extract a single report, specify the
name or URL of the custom report you want to read from the source. | Workday |
workdayReportsConfigFile | String | If you use the RaaS API and you want to extract multiple reports, specify the
path to the CSV file that you created for the list of custom reports that you want
to read from the source. | Workday |
outputType | String | The format in which you want the data to be stored on the target. | Workday |
extractNonDefault | Boolean | Indicates whether extract source fields that do not contain any default
value. | Workday |
oracleFusionInterfaceType | String | The type of interface used to extract data from the source. Options are:
| Oracle Fusion Cloud |
oracleFusionModuleName | String | The name of the Oracle Fusion Cloud Applications Suite module to extract
data. | Oracle Fusion Cloud |
oracleFusionEnableChildObjects | Boolean | Indicates whether to include child objects from the Oracle Fusion Cloud source
in the extraction. | Oracle Fusion Cloud |
Field | Type | Description | Supported source types |
|---|---|---|---|
salesforceAPI | String | For Salesforce sources in an application ingestion and replication task, the type of Salesforce API that you want to use to
retrieve the source data. Options are:
| Salesforce |
salesforceBulkV2Partitioning | Boolean | For initial and combined load jobs that use the Salesforce Bulk API 2.0, set
this parameter to true to use partitioning when reading source data. | Salesforce |
salesforceBulkV2RecordsPerPartition | Integer | For initial and combined load jobs that use the Salesforce Bulk API 2.0, the
number of records to process per partition when reading data. | Salesforce |
salesforceBulkV2MaxParallelPartitions | Integer | The maximum number of partition threads that can be used to query the source
for data in parallel during initial load processing or the unload phase of
combined jobs that use the Salesforce Bulk API 2.0. Use this property to control
the number of source partition queries that can be executed against the source at
the same time. For example, if a table contains data in 100 partitions, all 100
partitions are queried at the same time by default. However, you can use this
property to reduce the number of concurrent queries. Default value is
equal to the total number of partitions. | Salesforce |
enableSourcePartitioning | Boolean | Controls whether to use multiple partitions to query for the source data to
be unloaded in parallel during initial load processing or the unload phase of
combined load jobs. Set this property to true to use multiple partitions with one
query per partition. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |
sourcePartitionCount | Integer | The number of partitions to use for querying for source data in parallel
during initial load jobs and the unload phase of combined load jobs, when the
Enable Partitioning custom property is selected. Default is 5. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |
jdbcPartitioningTechnique | String | The partitioning technique used to distribute source rows across partitions.
Valid values are uniform, which distributes primary keys evenly across partitions,
and heuristic, which estimates key ranges for faster calculation. Default is
uniform. | Oracle, Microsoft SQL Server, SAP HANA |
sourceMaxParallelPartitions | Integer | The maximum number of partition reader threads that can be used to query the
source in parallel during initial load processing or the unload phase of combined
jobs. | Oracle, Microsoft SQL Server, SAP HANA |
unloadHelperThreadCount | Integer | The number of unload helper threads allocated to an initial load job or the
unload phase of a combined job to convert the unloaded data rows into a canonical
format that can be passed to the writer. Default value is 2. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |
readEventBatchSize | Integer | The number of payload events written in batch to the internal event queue
during CDC processing. When the event queue is implemented as an internal ring
buffer, this value is the number of payload events that the reader writes to a
single internal buffer slot. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2
for z/OS, SAP HANA |
readerHelperThreadCount | Integer | The number of reader helper threads used during CDC processing to convert
change data into a canonical format that can be passed to the target. Default
value is 3. You can enter a larger value to allow more threads to be
available for performing conversion processing in parallel. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Db2 for LUW, Db2 for i, Db2
for z/OS, SAP HANA |
readerInputIsPersisted | Boolean | Set this parameter to true to persistently store transaction data in a disk
buffer so that the data can be consumed continually, even when the writing of data
to the target is slow or delayed. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |
unloadFlashbackDisabled | Boolean | Set this parameter to true to disable the use of Oracle Flashback when
fetching data from the database. | Oracle, Microsoft SQL Server, MySQL |
salesforceBulkV2Partitioning | Boolean | For initial and combined load jobs that use the Salesforce Bulk API 2.0, set
this parameter to true to use partitioning when reading source data. | Salesforce |
salesforceBulkV2RecordsPerPartition | Integer | For initial and combined load jobs that use the Salesforce Bulk API 2.0, the
number of records to process per partition when reading data. | Salesforce |
salesforceBulkV2MaxParallelPartitions | Integer | The maximum number of partition threads that can be used to query the source
for data in parallel during initial load processing or the unload phase of
combined jobs that use the Salesforce Bulk API 2.0. Use this property to control
the number of source partition queries that can be executed against the source at
the same time. For example, if a table contains data in 100 partitions, all 100
partitions are queried at the same time by default. However, you can use this
property to reduce the number of concurrent queries. Default value is
equal to the total number of partitions. | Salesforce |
readerHistoricalCacheUseCloud | Boolean | Indicates whether the reader historical cache is stored in cloud storage. | Oracle, Microsoft SQL Server, MySQL, PostgreSQL, Teradata, Netezza, Db2 for
LUW, Db2 for i, Db2 for z/OS, SAP HANA |