fs.azure.account.key.<your account>.blob.core.windows.net
Required for Azure HDInsight cluster that uses WASB storage. The storage account access key required to access the storage.
You can contact the HDInsight cluster administrator to get the storage account key associated with the HDInsight cluster. If you are unable to contact the administrator, perform the following steps to decrypt the encrypted storage account key:
Copy the value of the
fs.azure.account.key.<your account>.blob.core.windows.net
property.
<property>
<name>fs.azure.account.key.<youraccount>.blob.core.windows.net</name>
<value>STORAGE ACCOUNT KEY</value>
</property>
Decrypt the storage account key. Run the
decrypt.sh
specified in the
fs.azure.shellkeyprovider.script
property along with the encrypted value you copied in the previous step.
<property>
<name>fs.azure.shellkeyprovider.script</name>
<value>/usr/lib/hdinsight-common/scripts/decrypt.sh</value>
</property>
Copy the decrypted value and update the value of
fs.azure.account.key.youraccount.blob.core.windows.net
property in the cluster configuration core-site.xml.
dfs.adls.oauth2.client.id
Required for Azure HDInsight cluster that uses ADLS Gen1 storage without Enterprise Security Package. The application ID associated with the Service Principal required to authorize the service principal and access the storage.
To find the application ID for a service principal, in the Azure Portal, click
.
dfs.adls.oauth2.refresh.url
Required for Azure HDInsight cluster that uses ADLS Gen1 storage without Enterprise Security Package. The OAuth 2.0 token endpoint required to authorize the service principal and access the storage.
To find the refresh URL OAuth 2.0 endpoint, in the Azure portal, click
.
dfs.adls.oauth2.credential
Required for Azure HDInsight cluster that uses ADLS Gen1 storage without Enterprise Security Package. The password required to authorize the service principal and access the storage.
To find the password for a service principal, in the Azure portal, click
.
fs.azure.account.key.<your account>.dfs.core.windows.net
Required for Azure HDInsight cluster that uses ADLS Gen2 storage without Enterprise Security Package. The decrypted account key for the storage account.
You can contact the HDInsight cluster administrator to get the storage account key associated with the HDInsight cluster.
hadoop.proxyuser.<proxy user>.groups
Defines the groups that the proxy user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.
Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
After you make changes to proxy user properties, restart the credential service and other cluster services that use core-site configuration values.
hadoop.proxyuser.<proxy user>.users
Required for all cluster types.
Defines the user account that the proxy user account can impersonate. On a secure cluster, the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.
Set to a single user account or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
After you make changes to proxy user properties, restart the credential service and other cluster services that use core-site configuration values.
hadoop.proxyuser.<proxy user>.hosts
Defines the host machines that a user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.
Set to a single host name or IP address, or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any host.
After you make changes to proxy user properties, restart the credential service and other cluster services that use core-site configuration values.
hadoop.proxyuser.hive.hosts
Comma-separated list of hosts that you want to allow the Hive user to impersonate on a non-secure cluster.
When
hive.server2.enable.doAs
is false, append a comma-separated list of Informatica server host names or IP address where the Data Integration Service is running. If less security is preferred, use the wildcard " * " to allow impersonation from any host.
After you make changes to this property, restart the cluster services that use core-site configuration values.
hadoop.proxyuser.yarn.groups
Comma-separated list of groups that you want to allow the YARN user to impersonate on a non-secure cluster.
Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
After you make changes to proxyuser properties, restart the credential service and other cluster services that use core-site configuration values.
hadoop.proxyuser.yarn.hosts
Comma-separated list of hosts that you want to allow the YARN user to impersonate on a non-secure cluster.
Set to a single host name or IP address, or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any host.
After you make changes to proxy user properties, restart the credential service and other cluster services that use core-site configuration values.
io.compression.codecs
Enables compression on temporary staging tables.
Set to a comma-separated list of compression codec classes on the cluster.
hadoop.security.auth_to_local
Translates the principal names from the Active Directory and MIT realm into local names within the Hadoop cluster. Based on the Hadoop cluster used, you can set multiple rules.
Set to: RULE:[1:$1@$0](^.*@YOUR.REALM)s/^(.*)@YOUR.REALM\.COM$/$1/g
Set to: RULE:[2:$1@$0](^.*@YOUR.REALM\.$)s/^(.*)@YOUR.REALM\.COM$/$1/g
