Databricks Connector

Back Next

Sources for Databricks

Add a Source transformation to extract data from a source.

When you add a Source transformation to a mapping, you define the source connection, source objects, and source properties related to the

Databricks

connection type.

The following table lists the source properties that are supported by SQL warehouse, all-purpose cluster, and job cluster:

Property	SQL warehouse¹	All-purpose cluster²	Job cluster³
Source Type - Single Object	Yes	Yes	Yes
Source Type - Parameter	Yes	Yes	Yes
Source Type - Query	Yes	No	No
Source Type - Multiple Objects	Yes	No	No
Parameter	Yes	No	Yes
Filter	Yes	No	Yes
Sort	No	No	No
Select distinct rows only	No	No	No
Database Name	Yes	No	Yes
Table Name	Yes	No	Yes
Pre SQL	Yes	No	No
Post SQL	Yes	No	No
SQL Override	Yes	No	No
Staging Location	Yes	No	Yes
Job Timeout	No	No	Yes
Job Status Poll Interval	No	No	Yes
DB REST API Timeout	No	No	Yes
DB REST API Retry Interval	No	No	Yes
Tracing Level	Yes	No	Yes
Key Range Partitioning	Yes	No	No
¹The Secure Agent connects to the SQL warehouse at design time and runtime. ²The Secure Agent connects to the all-purpose cluster to import the metadata at design time. ³The Secure Agent connects to the job cluster to run the mappings.