Read this section to learn about new features that support Databricks functionality.
Delta Lake Schema Evolution
Effective in version 10.5.2, you can configure dynamic mappings to apply Databricks source schema changes to Delta Lake targets. You can choose from among three different strategies to update Delta Lake target tables when a source schema changes. Mappings succeed with merge enabled in the mapping and schema enforcement disabled in the Databricks workspace.
For more information, see the following documentation:
Use Databricks and Custom Parameters to Configure Ephemeral Clusters
Effective in version 10.5.2, you can use custom parameters and Databricks parameters in a JSON file to configure ephemeral clusters when you use a cluster creation workflow. You can use the JSON file, which is easily edited, instead of specifying parameter values in the Create Cluster task properties.
For more information, see cluster workflow documentation in the
Data Engineering Integration User Guide
Unmanaged Databricks Table Targets
Effective in version 10.5.2, you can configure mappings to write to unmanaged Databricks table targets.