Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Data Engineering Integration
  3. Mappings
  4. Mapping Optimization
  5. Sources
  6. Targets
  7. Transformations
  8. Python Transformation
  9. Data Preview
  10. Cluster Workflows
  11. Profiles
  12. Monitoring
  13. Hierarchical Data Processing
  14. Hierarchical Data Processing Configuration
  15. Hierarchical Data Processing with Schema Changes
  16. Intelligent Structure Models
  17. Blockchain
  18. Stateful Computing
  19. Appendix A: Connections Reference
  20. Appendix B: Data Type Reference
  21. Appendix C: Function Reference

Databricks Cloud Provisioning Configuration Properties

Databricks Cloud Provisioning Configuration Properties

The properties in the Databricks cloud provisioning configuration enable the Data Integration Service to contact and create resources on the Databricks cloud platform.
The following table describes the Databricks cloud provisioning configuration properties:
Property
Description
Name
Name of the cloud provisioning configuration.
Because the Administrator tool lists cloud provisioning configuration objects with other connections, use a naming convention such as "CPC" as part of the name of the object to help identify it.
ID
The cluster ID of the Databricks cluster.
Description
Optional description of the cloud provisioning configuration.
Databricks domain
Domain name of the Databricks deployment.
Databricks token ID
The token ID created within Databricks required for authentication.
If the token has an expiration date, verify that you get a new token from the Databricks administrator before it expires.
Advanced Properties
Advanced properties that are unique to the Databricks cloud provisioning configuration.

Advanced Properties

Configure the following properties in the
Advanced Properties
of the Databricks configuration section:
infaspark.pythontx.exec
Required to run a Python transformation on the Databricks Spark engine. Set to the location of the Python executable binary on the worker nodes in the Databricks cluster.
When you provision the cluster at run time, set this property in the Databricks cloud provisioning configuration. Otherwise, set on the Databricks connection.
For example, set to:
infaspark.pythontx.exec=/databricks/python3/bin/python3
infaspark.pythontx.executorEnv.PYTHONHOME
Required to run a Python transformation on the Databricks Spark engine. Set to the location of the Python installation directory on the worker nodes in the Databricks cluster.
When you provision the cluster at run time, set this property in the Databricks cloud provisioning configuration. Otherwise, set on the Databricks connection.
For example, set to:
infaspark.pythontx.executorEnv.PYTHONHOME=/databricks/python3

0 COMMENTS

We’d like to hear from you!