Common Content for Data Engineering
- Common Content for Data Engineering 10.2.2
- All Products
Property | Description |
---|---|
Cluster Type | External Cluster |
Hadoop Distribution | HDInsight |
Cluster URL | Fully qualified host name to access the cluster. |
Cluster URL username | User name to access the cluster. |
Cluster URL password | Password for the Cluster URL username. |
Custom Property | Description |
---|---|
LdmCustomOptions.deployment.azure.account.key | The key to authenticate the Catalog Service to connect to Azure storage account . The value of the Azure storage account key might be encrypted or non encrypted. You can retrieve the value from
fs.azure.account.key.<storage account name> property in
core-site.xml file present in the Azure HDInsight cluster. |
LdmCustomOptions.deployment.azure.key.decryption.script.path | If the key specified in the
LdmCustomOptions.deployment.azure.account.key property is in encrypted format, you can use the decrypt shell script to decrypt the key using the key certificate. You must verify that you copy the decrypt shell script and key certificate file to the (same path as cluster machine) domain machine before enabling Catalog Service. Make sure that you maintain the path in the Azure HDInsight cluster machine for the copied files in the domain machine. The value for the property is the location of the decrypt shell script. For example,
/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh . The key certificate file, key_decryption_cert.prv, is present in the /usr/lib/hdinsight-common/certs/key_decryption_cert.prv directory of Azure HDInsight cluster. |
LdmCustomOptions.deployment.hdfs.default.fs | Address of the WASB storage account to which the Catalog Service must connect. The address includes the WASB storage container name with the storage account name. The value for the property is the complete WASB address with the container and storage account names. You can retrieve the value for the property from the
fs.defaultFS property in the
core-site.xml file present in the Azure HDInsight cluster. |