By default, a Secure Agent can schedule two mapping tasks for execution. If there are more than two tasks, additional tasks are queued and then scheduled for execution when a slot becomes available. This can cause the Secure Agent machine's capacity to be underutilized.

To achieve better utilization of the CPU capacity of the Secure Agent machine and achieve a higher degree of concurrency, you can set the maxDTMProcesses custom property for the Data Integration Server to the number of parallel tasks. For example, setting this property to 16 allows 16 tasks to run simultaneously.

The following image shows configuration for the maxDTMProcesses custom property on the agent details page in Administrator:

The recommended maxDTMProcesses value varies based on the connection types of the jobs that the agent runs:

If the jobs use file-based or ODBC connections, use the following calculation:  0.75 times number of logical CPUs
For example, if there are 24 logical CPUs on the Secure Agent machine, set the value to 18.
If the jobs use cloud data lake or cloud data warehouse connectors, use the following calculation: 0.33 times number logical CPUs
For example, if there are 24 logical CPUs on the Secure Agent machine, set the value to 8.

When you set the maxDTMProcesses custom property, note the following guidelines:

Setting the property to a high value can help increase parallelism for task execution, but it can also cause performance bottlenecks in task execution time or increase job dispatch latency for mapping tasks.
Setting the property to a lower value can lead to mapping jobs waiting for previous jobs to complete to acquire a slot.
When you set the maxDTMProcesses property, don't exceed the terms of your license agreement. For example, if your organization is licensed for 3 Secure Agents and can run each agent on up to 4 CPUs, don't set this property to a value greater than 12.

Use these guidelines as a starting point and set the correct value iteratively.

For more information about setting agent properties, see

Runtime Environments

.