Data Engineering Integration
- Data Engineering Integration 10.4.0
- All Products
Property
| Description
|
---|---|
Name
| The name of the connection. The name is not case sensitive and must be unique within the domain. You can change this property after you create the connection. The name cannot exceed 128 characters, contain spaces, or contain the following special characters:~ ` ! $ % ^ & * ( ) - + = { [ } ] | \ : ; " ' < , > . ? /
|
ID
| String that the Data Integration Service uses to identify the connection. The ID is not case sensitive. It must be 255 characters or less and must be unique in the domain. You cannot change this property after you create the connection. Default value is the connection name.
|
Description
| Optional. The description of the connection. The description cannot exceed 4,000 characters.
|
Connection Type
| Choose Databricks.
|
Cluster Configuration
| Name of the cluster configuration associated with the Databricks environment.
Required if you do not configure the cloud provisioning configuration.
|
Cloud Provisioning Configuration
| Name of the cloud provisioning configuration associated with a Databricks cloud platform.
Required if you do not configure the cluster configuration.
|
Staging Directory
| The directory where the Databricks Spark engine stages run-time files.
If you specify a directory that does not exist, the Data Integration Service creates it at run time.
If you do not provide a directory path, the run-time staging files are written to
/<cluster staging directory>/DATABRICKS .
|
Advanced Properties
| List of advanced properties that are unique to the Databricks environment.
You can configure run-time properties for the Databricks environment in the Data Integration Service and in the Databricks connection. You can override a property configured at a high level by setting the value at a lower level. For example, if you configure a property in the Data Integration Service custom properties, you can override it in the Databricks connection. The Data Integration Service processes property overrides based on the following priorities:
Informatica does not recommend changing these property values before you consult with third-party documentation, Informatica documentation, or Informatica Global Customer Support. If you change a value without knowledge of the property, you might experience performance degradation or other unexpected results.
|
infaspark.pythontx.exec=/databricks/python3/bin/python3
infaspark.pythontx.executorEnv.PYTHONHOME=/databricks/python3