Abstract
Supported Versions
Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Back Next

Troubleshooting Kafka Errors

While reading from Kafka, you might encounter an error similar to the following error:

java.lang.AssertionError: assertion failed: Failed to get records for 
spark-executor-IIS-Kafka-DStream-Consumer-5c58a763-7fda-4dd1-bd84-f15dad9cd765 AggregateSrc1 6 9014479
 after polling for 512

This error might occur if the polling timeout limit is low. Increase the polling timeout limit.

Configure the following property for Spark in the mapping properties:

Property	Description
spark.streaming.kafka.consumer.poll.ms	Set a value between 5000 and 10000 milliseconds.

Recommendations for Tuning the Kafka Cluster

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal

0 COMMENTS

We’d like to hear from you! Log in to comment.

Rename Saved Search

Table of Contents

Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Troubleshooting Kafka Errors

Troubleshooting Kafka Errors