Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Sizing Guidelines and Performance Tuning for Big Data Streaming 10.2.1

Determine Your Hardware

Determine Your Hardware

To optimize the performance, acquire the right type of hardware for your Big Data Streaming environment.
Consider the following points while determining your hardware:
  • The criteria for determining a deployment is the number of messages processed and not the size of data in gigabytes.
  • Choose the hardware such that batches do not get queued. Batches are queued when the batch processing time is greater than the batch interval time.
  • Use the following approach to determine hardware capacity:
    • Implement a proof of concept (POC) to determine the throughput for each core.
    • Determine the peak load in terms of the anticipated number of the messages processed per second.
    • Divide the peak load by the throughput for each core to get the total number of cores required and the size of the disk, memory, and network appropriately.

0 COMMENTS

We’d like to hear from you!