When the HDInsight cluster is configured with the Enterprise Security Package (ESP), verify security settings to access Azure Data Lake Storage (ADLS) Gen2 resources.

The Enterprise Security Package uses Kerberos authentication and Apache Ranger authorization to enable Active Directory (AD) based authentication, multi-user support, and role-based access control.

To verify that you have the required environment, see the Azure documentation.

Configure a HDInsight cluster with Enterprise Security Package by using Azure Active Directory Domain Services
Use Azure Data Lake Storage Gen2 with Azure HDInsight Clusters

In addition, set the following properties on the cluster when you integrate an HDInsight 4.x cluster with WASBS storage:

fs.permissions.umask-mode

The umask used to set default permissions on created files and directories.

Set to

000

.

Run as end user instead of Hive user (doAs)

Enables the cluster to run jobs as the impersonation user and not the Hive user.

Set to FALSE.

When the HDInsight 4.x cluster does not have the Enterprise Security Package enabled, verify the following properties on the cluster when it uses WASBS storage:

Hive Authorization Manager

Authorization provider for the cluster.

Set to

org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly

.

For more information, see the Azure HDInsight documentation.

Run as end user instead of Hive user (doAs)

Enables the cluster to run jobs as the impersonation user and not the Hive user.

Set to TRUE.