fs.azure.account.key.<youraccount>.blob.core.windows.net
Required for Azure HDInsight cluster that uses WASB storage. The storage account access key required to access the storage.
You can contact the HDInsight cluster administrator to get the storage account key associated with the HDInsight cluster. If you are unable to contact the administrator, perform the following steps to decrypt the encrypted storage account key:
Copy the value of the
fs.azure.account.key.<youraccount>.blob.core.windows.net
property.
<property>
<name>fs.azure.account.key.<youraccount>.blob.core.windows.net5</name>
<value>STORAGE ACCOUNT KEY</value>
</property>
Decrypt the storage account key. Run the
decrypt.sh
specified in the
fs.azure.shellkeyprovider.script
property along with the encrypted value you copied in the previous step.
<property>
<name>fs.azure.shellkeyprovider.script</name>
<value>/usr/lib/hdinsight-common/scripts/decrypt.sh</value>
</property>
Copy the decrypted value and update the value of
fs.azure.account.key.youraccount.blob.core.windows.net
property in the cluster configuration core-site.xml.
dfs.adls.oauth2.client.id
Required for Azure HDInsight cluster that uses ADLS storage. The application ID associated with the Service Principal required to authorize the service principal and access the storage.
To find the application ID for a service principal, in the Azure Portal, click
.
dfs.adls.oauth2.refresh.url
Required for Azure HDInsight cluster that uses ADLS storage. The OAuth 2.0 token endpoint required to authorize the service principal and access the storage.
To find the refresh URL OAuth 2.0 endpoint, in the Azure portal, click
.
dfs.adls.oauth2.credential
Required for Azure HDInsight cluster that uses ADLS storage. The password required to authorize the service principal and access the storage.
To find the password for a service principal, in the Azure portal, click
.
hadoop.proxyuser.<proxy user>.groups
Defines the groups that the proxy user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.
Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
hadoop.proxyuser.<proxy user>.hosts
Defines the host machines that a user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.
Set to a single host name or IP address, or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any host.
hadoop.proxyuser.yarn.groups
Comma-separated list of groups that you want to allow the YARN user to impersonate on a non-secure cluster.
Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
hadoop.proxyuser.yarn.hosts
Comma-separated list of hosts that you want to allow the YARN user to impersonate on a non-secure cluster.
Set to a single host name or IP address, or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any host.
io.compression.codecs
Enables compression on temporary staging tables.
Set to a comma-separated list of compression codec classes on the cluster.
hadoop.security.auth_to_local
Translates the principal names from the Active Directory and MIT realm into local names within the Hadoop cluster. Based on the Hadoop cluster used, you can set multiple rules.
Set to: RULE:[1:$1@$0](^.*@YOUR.REALM)s/^(.*)@YOUR.REALM\.COM$/$1/g
Set to: RULE:[2:$1@$0](^.*@YOUR.REALM\.$)s/^(.*)@YOUR.REALM\.COM$/$1/g