Allows the

Elastic Server

to queue Spark tasks. The default value is set to true.

If this property is set to false, the

Elastic Server

processes all submitted Spark tasks concurrently. If you set the maximum number of concurrent tasks on the

Elastic Server

to a value that is higher than the default concurrency, increase the Java heap size accordingly. Otherwise, the Secure Agent process might crash.

The

Elastic Server

processes data preview and SQL ELT optimization jobs immediately without queuing them.

Maximum number of concurrent Spark tasks that the

Elastic Server

can process. When the

Elastic Server

reaches the maximum number of concurrent Spark tasks, additional Spark tasks can be queued.

By default, the maximum number of concurrent Spark tasks depends on the cloud platform. The

Elastic Server

uses the following defaults for fully-managed and self-service clusters:

A local cluster can run 15 concurrent Spark tasks, and the concurrency can't be changed.

Note that the Java heap size is set to 2 GB by default. If you set the maximum number of concurrent tasks to a value that is higher than the default, increase the Java heap size using the default ratios.

For example, if you want to run 500 concurrent Spark tasks on Microsoft Azure, increase the Java heap size to 4 GB. If the Java heap size remains at 2 GB, the

Elastic Server

will process a maximum of 250 concurrent Spark tasks and the remaining 250 Spark tasks can be queued.

The

Elastic Server

uses an upper bound of 1,000 Spark tasks per 2 GB of Java heap space. If you set the maximum number of concurrent tasks to a value that is higher than 1,000 without increasing the Java heap size above 2 GB, the

Elastic Server

fails to start.

For more information about Secure Agent machine sizing, see

Data Integration Performance Tuning

.