Strategies for Incremental Updates on Hive in Big Data Management 10.2

Strategies for Incremental Updates on Hive in Big Data Management 10.2

Tune the Partition Merge Solution for Optimal Performance

Tune the Partition Merge Solution for Optimal Performance

Use the following optimization techniques to get the best performance results based on your requirements:
  • Do not require Hive transactions to be enabled, and ensure that ACID transactions are disabled. Inserts into transaction enabled Hive tables are approximately three to five times slower than inserts into transaction disabled Hive tables.
  • Configure partitions so that each partition has approximately 5-10 GB of data. The partition merge solution performs updates by identifying the impacted partitions. These updates are efficient when the data is well distributed across many partitions.

0 COMMENTS

We’d like to hear from you!