Table of Contents

Search

  1. Preface
  2. Introduction to Databricks Delta Connector
  3. Connections for Databricks Delta
  4. Mappings for Databricks Delta
  5. Migrating a mapping
  6. Databricks Delta SQL ELT optimization
  7. Data type reference

Databricks Delta Connector

Databricks Delta Connector

Connection details

Connection details

The following table describes the
basic
connection properties:
Property
Description
Connection Name
Name of the connection.
Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -,
Maximum length is 255 characters.
Description
Description of the connection. Maximum length is 4000 characters.
Type
Databricks Delta
Runtime Environment
The name of the runtime environment where you want to run tasks.
Select a Secure agent, Hosted Agent, or serverless runtime environment.
Hosted Agent is not applicable for mappings in advanced mode.
SQL Warehouse JDBC URL
Databricks SQL Warehouse JDBC connection URL.
Required to connect to a Databricks SQL warehouse. Doesn't apply to Databricks clusters.
To get the SQL Warehouse JDBC URL, go to the Databricks console and select the JDBC driver version from the JDBC URL menu.
For JDBC URL version 2.6.22 or earlier, use the following syntax:
jdbc:spark://<Databricks Host>:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/<SQL endpoint cluster ID>;
For JDBC URL version 2.6.25 or later, use the following syntax:
jdbc:databricks://<Databricks Host>:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/<SQL endpoint cluster ID>;
Ensure that you set the required environment variables in the Secure Agent. Also specify the correct
JDBC Driver Class Name
under advanced connection settings.
Specify the database name in the Database Name connection property. If you specify the database name in the JDBC URL, it is not considered. The Databricks Host, Organization ID, and Cluster ID properties are not considered if you configure the SQL warehouse JDBC URL property.
Databricks Token
Personal access token to access Databricks.
Required for SQL warehouse and Databricks cluster.
Ensure that you have permissions to attach to the cluster identified in the
Cluster ID
property.
For mappings, you must have additional permissions to create Databricks clusters.
Catalog Name
If you use Unity Catalog, the name of an existing catalog in the metastore.
Optional for SQL warehouse. Doesn't apply to Databricks cluster.
You can also specify the catalog name in the end of the SQL warehouse JDBC URL.
The catalog name cannot contain special characters.
For more information about Unity Catalog, see the Databricks Delta documentation.

0 COMMENTS

We’d like to hear from you!