Tune the Partition Merge Solution for Optimal Performance
Tune the Partition Merge Solution for Optimal Performance
Use the following optimization techniques to get the best performance results based on your requirements:
Do not require Hive transactions to be enabled, and ensure that ACID transactions are disabled. Inserts into transaction enabled Hive tables are approximately three to five times slower than inserts into transaction disabled Hive tables.
Configure partitions so that each partition has approximately 5-10 GB of data. The partition merge solution performs updates by identifying the impacted partitions. These updates are efficient when the data is well distributed across many partitions.