Rules and guidelines for personal staging location
Rules and guidelines for personal staging location
When you select the personal staging location as a staging environment, the data is first staged in a Java temporary location and then copied to a personal staging location of the unity catalog. Both the staged files will be deleted after the
task
runs successfully.
However, to stage the data in a different directory, configure the DTM property
-Djava.io.tmpdir=/my/dir/path
in the JVM options in the system configuration settings of the Administrator service.
To enable data staging in a different directory, you should have read and write permissions and enough disk space to stage the data in the directory.
When you specify a personal staging location in the Databricks connection properties for staging, consider the following rules and guidelines:
You can only specify unity enabled catalog in the SQL warehouse JDBC URL.
You cannot use personal staging location as the staging environment with OAuth machine-to-machine authentication.
Mappings run without
SQL ELT optimization
.
The data is staged in the folder
stage://tmp/<user_name>
where the
<user_name>
is picked from the Databricks token provided in the connection and requires read and write access to the personal staging location in the root location of AWS and Azure.
Effective in the October 2024 release, personal staging location entered deprecation. While you can use the functionality in the current release, Informatica intends to drop support for the functionality in a future release. Informatica recommends that you use a Volume to stage the data.