Connector Release Notes

Connector Release Notes

Databricks Delta Connector

Databricks Delta Connector

Fixed issues

The following table describes fixed issues:
CR
Description
CCON-92957
When you use IIF() function with ISDATE() function in a Databricks Delta mapping enabled for SQL ELT optimization, the mappings fails with the following error:
java.lang.ClassCastException: org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation cannot be cast to org.apache.spark.sql.execution.datasources.LogicalRelation
(February 2024)
CCON-92943
When you run a Databricks Delta mapping enabled for SQL ELT optimization, the IIF() function returns incorrect results in the following cases:
  • IIF() function is used with LAST_DAY(), ISNULL(), or SUBSTR() functions.
  • IIF() function is used with IS_NUMBER() and the input value is NULL, empty string, or white spaces.
  • IIF() function is used with IS_SPACES() and the input value is an empty string.
(February 2024)
CCON-94189
When you read Timestamp values from a source and write to a Databricks Delta target, the Timestamp values are preserved only up to millisecond precision of three digits in the target. (February 2024)
CCON-57945
When you push down an IIF() function in a mapping that reads from and writes to Databricks Delta, and the Boolean values are returned as 0 and 1, the mapping fails. When the Boolean values are true and false, the mapping runs successfully. (December 2023)
CCON-70460
When you configure a Databricks Delta connection using the personal staging environment and run two mappings concurrently to write two different parquet files to a target folder, the mappings fail with the following error:
[Error In Running Sql Query: [Simba][SparkJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: org.apache.hive.service.cli.HiveSQLException: Error running query: [_LEGACY_ERROR_TEMP_2332] org.apache.spark.sql.AnalysisException: [UNABLE_TO_INFER_SCHEMA] Unable to infer schema for Parquet.
(November 2023)

Known issues

The following table describes known issues:
Issue
Description
CCON-92977
When you use an unconnected lookup and run a Databricks Delta mapping enabled for SQL ELT optimization, the mapping fails if you use Nested IIF() function or if the IIF() function returns a Float value in the lookup query. (December 2023)
CBDM-29892
A mapping that reads from or writes decimal data types in hierarchical data fails with the following error:
java.lang.RuntimeException
(July 2023)

0 COMMENTS

We’d like to hear from you!