Table of Contents

Search

  1. Preface
  2. Part 1: Hadoop Integration
  3. Part 2: Databricks Integration
  4. Appendix A: Connections Reference

Configure ADLS Storage Access

Configure ADLS Storage Access

If you use ADLS storage, configure some Hadoop credential configuration options as Databricks Spark options.
Configure the following properties in the advanced Spark properties on the Databricks cluster to Delta Lake tables on ADLS Gen1:
Property
Description
dfs.adls.oauth2.access.token.provider.type
Access token provider type. Use the value
ClientCredential
.
dfs.adls.oauth2.client.id
The application ID associated with the Service Principal required to authorize the service principal and access the storage.
dfs.adls.oauth2.credential
The password required to authorize the service principal and access the storage.
dfs.adls.oauth2.refresh.url
The OAUTH 2.0 token endpoint required to authorize the service principal and access the storage.
See the Databricks documentation for examples on how to specify these settings in Scala or Python.
The following excerpt shows how these properties appear in the Spark properties window:
spark.hadoop.dfs.adls.oauth2.access.token.provider.type ClientCredential spark.hadoop.dfs.adls.oauth2.client.id <your-service-client-id> spark.hadoop.dfs.adls.oauth2.credential <your-service-credentials> spark.hadoop.dfs.adls.oauth2.refresh.url "https://login.microsoftonline.com/<your-directory-id>/oauth2/token"

0 COMMENTS

We’d like to hear from you!