Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.5.2
  3. Part 2: Version 10.5.1 - 10.5.1.1
  4. Part 3: Versions 10.5 - 10.5.0.1
  5. Part 4: Versions 10.4.1 - 10.4.1.3
  6. Part 5: Versions 10.4 - 10.4.0.2
  7. Part 6: Versions 10.2.2 - 10.2.2 HotFix 1
  8. Part 7: Version 10.2.1
  9. Part 8: Version 10.2 - 10.2 HotFix 2

What's New and Changed (10.5.2)

What's New and Changed (10.5.2)

Databricks

Databricks

Read this section to learn about new features that support Databricks functionality.

Delta Lake Schema Evolution

Effective in version 10.5.2, you can configure dynamic mappings to apply Databricks source schema changes to Delta Lake targets. You can choose from among three different strategies to update Delta Lake target tables when a source schema changes. Mappings succeed with merge enabled in the mapping and schema enforcement disabled in the Databricks workspace.
For more information, see the following documentation:

Use Databricks and Custom Parameters to Configure Ephemeral Clusters

Effective in version 10.5.2, you can use custom parameters and Databricks parameters in a JSON file to configure ephemeral clusters when you use a cluster creation workflow. You can use the JSON file, which is easily edited, instead of specifying parameter values in the Create Cluster task properties.
For more information, see cluster workflow documentation in the
Data Engineering Integration User Guide
.

Unmanaged Databricks Table Targets

Effective in version 10.5.2, you can configure mappings to write to unmanaged Databricks table targets.
For more information, see the
Data Engineering Integration User Guide