When you run a streaming mapping that contains a Confluent Kafka source with Avro payload and Complex File data object target, the mapping fails with

Malformed/Invalid Avro record

error. This error occurs because the mapping fails to consume the messages sent from KafkaAvroSerializer even though the schema and the source data are synchronized.

On Azure HDInsight version 4.1, when you run a streaming mapping with multiple JMS sources that has the custom checkpoint directory set, the mapping fails on restart with the following error:

ERROR MicroBatchExecution: Query Write_cl_task_update [id = 934e2c43-219a-4245-808a-44e66138d9aa, runId = ab818a5a-4a83-4ebb-8e02-13472def8182] terminated with error
java.lang.IllegalStateException: batch 2 doesn't exist
at org.apache.spark.sql.execution.streaming.HDFSMetadataLog$.verifyBatchIds(HDFSMetadataLog.scala:470)

When you run a streaming mapping with Kafka sources and targets in Avro data format, the data written to the target file incorrectly adds the data type.

In a streaming mapping with a Kafka or Confluent Kafka source and Kafka, Confluent Kafka, or HDFS target, the mapping fails when the following conditions are true:

Target schema format is of type Flat
Mapping Flow
option is enabled in the target

The mapping fails with the following error:

Error: Mapping is failed.
Lost task 0.1 in stage 0.0 (TID 1,inbdsrh74qa01.informatica.com, executor 1):
java.util.NoSuchElementException: next on empty iterator at
scala.collection.Iterator$$anon$2.next(Iterator.scala:39) 

When you run a streaming mapping with Kafka source and Teradata connector for JDBC target, the mapping fails with

"[Error 1382] [SQLState HY000] The batch is empty."

error.

When you run a streaming mapping with a Kafka source and HDFS target on Cloudera CDP version 7.1.1, the mapping fails with a compilation error when the following conditions are true:

Topic name is linked to file name port and data to data port
Dynamic strategy is enabled at source
Run-time properties of the mapping is configured as
infaspark.hive.impersonated.complexfile.write=true

Data loss happens when you run a streaming mapping with JMS as source and multiple targets and durable subscriptions configured.

On a Databricks cluster, logical data types are not applicable for Parquet format and shows the

Invalid Avro record

error when you write to the Microsoft Azure Data Lake Storage Gen2 target.

Mapping validation fails when you connect sources to Window transformations using dynamic ports.

When you run a streaming mapping with a Kafka source, the mapping fails with

InvalidTopicName

error when the following conditions are true:

A topic pattern is specified for the topic name
A time stamp is specified to read messages based on the custom timestamp offset

When you run a streaming mapping with a Kafka source, the mapping fails with

WARN KafkaOffsetReader: Error in attempt 1 getting Kafka offsets

error when:

a topic contains multiple partitions, and
a time stamp is specified to read messages based on the custom timestamp offset

On AWS Databricks, when you run a streaming mapping to write binary data to Amazon S3 target, the mapping validates but fails while trying to generate a Spark execution plan.