Verify cluster security settings depending on whether or not the cluster is configured with the Enterprise Security Package (ESP).
Clusters with ESP Enabled
When the HDInsight cluster is configured with the Enterprise Security Package (ESP), verify security settings to access Azure Data Lake Storage (ADLS) Gen2 resources.
The Enterprise Security Package uses Kerberos authentication and Apache Ranger authorization to enable Active Directory (AD) based authentication, multi-user support, and role-based access control.
To verify that you have the required environment, see the Azure documentation.
In addition, set the following properties on the cluster when you integrate an HDInsight 4.x cluster with WASBS storage:
fs.permissions.umask-mode
The umask used to set default permissions on created files and directories.
Set to
000
.
Run as end user instead of Hive user (doAs)
Enables the cluster to run jobs as the impersonation user and not the Hive user.
Set to FALSE.
Clusters without ESP Enabled
When the HDInsight 4.x cluster does not have the Enterprise Security Package enabled, verify the following properties on the cluster when it uses WASBS storage:
fs.permissions.umask-mode
The umask used to set default permissions on created files and directories.
Set to
000
.
Run as end user instead of Hive user (doAs)
Enables the cluster to run jobs as the impersonation user and not the Hive user.
Set to TRUE.
hive.metastore.execute.setugi
Enables the cluster to run jobs as the Data Integration Service user.