The Logical Data Transformation Manager translates the mapping into a Scala program, packages it as an application, and sends it to the Databricks Engine Executor on the Data Integration Service machine.

The Databricks Engine Executor submits the application through REST API to the Databricks cluster, requests to run the application, and stages files for access during run time.

The Databricks cluster passes the request to the Databricks Spark driver on the driver node.

The Databricks Spark driver distributes the job to one or more Databricks Spark executors that reside on worker nodes.

The executors run the job and stage run-time data to the Databricks File System (DBFS) of the workspace.