Connectors and Connections

Back Next

Rules and guidelines for personal staging location

When you select the personal staging location as a staging environment, the data is first staged in a java temporary location and then copied to a personal staging location of the unity catalog. Both the staged files will be deleted after the

task

runs successfully.

However, to stage the data in a different directory, configure the DTM property

-Djava.io.tmpdir=/my/dir/path

in the JVM options in the system configuration settings of the Administrator service.

To enable data staging in a different directory, you should have read and write permission and enough disk space to stage the data in the directory.

When you specify a personal staging location in the Databricks Delta connection properties for staging, consider the following rules and guidelines:

You can only specify unity enabled catalog in the SQL warehouse JDBC URL.

All mappings that are configured run without

SQL ELT optimization

The data is staged in the folder

stage://tmp/<user_name>

where the

<user_name>

is picked from the Databricks token provided in the connection and requires read and write access to the personal staging location in root location of AWS and Azure.

Databricks Delta connection properties

Download Guide

Watch

Comments

Cloud Mass Ingestion Homepage