Tasks

Back Next

Session properties in advanced mode

When you configure a task based on a mapping in advanced mode, you can configure session properties in the task or in the

advanced configuration

. Properties take effect in priority order based on where the property is configured.

The following table lists where you can configure session properties in different assets:

Asset	Property type
Mapping task	Advanced session properties Spark session properties
Dynamic mapping task	Advanced session properties
Advanced configuration	Runtime properties

For more information about

advanced configurations

, see the Administrator help.

You can specify some session properties at multiple levels. For example, you can specify the session property spark.executor.cores as a Spark session property in a

mapping

task and on a global level for all

mapping

tasks in an

advanced configuration

When you specify a session property at multiple levels, the property takes effect in the following priority order where system properties have the highest priority and properties with the prefix infacco.job have the lowest priority:

System properties
. System properties that override invalid user-defined properties. For example, a serverless runtime environment has a limit on Spark resources and a session property can’t request Spark resources above the predefined limit.

spark.custom.property
. Custom Spark session properties that you configure in a

mapping

task. These properties usually have a prefix such as spark or infaspark.

advanced.custom.property
. Custom advanced session properties that you configure in a

mapping

task or

dynamic mapping

task.

Non-custom Spark session properties or CLAIRE Tuning properties
. For example, spark.driver.memory or spark.memory.fraction.

infacco.job.<property>
. Runtime properties that you configure in an

advanced configuration

. These properties can affect Spark processing or mapping compilation.