Release Notes (10.5.0.1)

Release Notes (10.5.0.1)

Data Engineering Fixed Issues (10.5.0.1)

Data Engineering Fixed Issues (10.5.0.1)

The following table describes fixed issues:
Issue
Description
BDM-38139
When you explicitly set ZLIB compression for ORC tables in a mapping that runs on the Spark engine, the Spark workflow generates an additional execution step that causes the mapping to fail.
BDM-37934
When you run a mapping on a complex file source in Spark mode, the mapping does not write data to the target file if both the following conditions are true:
  • The Hadoop distribution of the cluster is Cloudera CDH version 6.3.4.
  • The complex file source has a binary or custom input file format.
BDM-37574
Mappings that run on a Cloudera CDH 6.3 cluster to read from or write to a Parquet file fail with the following error:
NoSuchFieldError: ZSTD inside ParquetOptions.scala:85
BDM-37438
Mappings that run on the Spark engine for more than 24 hours fail with delegation token error. This issue occurs in a cluster on Cloudera CDH version 6.3.
BDM-37351
When the Spark engine runs a mapping that includes an Update Strategy transformation that performs an INSET, UPDATE or DELETE operation and the source includes duplicate rows, the mapping fails with a "Cardinality Violation in Merge statement" error.
BDM-36357
When you run enterprise discovery profile jobs using a resource configured on Enterprise Data Catalog, the pmdtmsvc2 DTM process intermittently shuts down. The DTM shutdown results in inconsistent states for mappings that run on the DTM.