Consider the following guidelines before you enable incremental loading.
Do not enable incremental loading in the following situations:
When there are many metadata changes and loads occur more than two weeks apart.
An incremental load runs a set of queries that trigger multiple extraction operations from Cloudera Navigator. Occasionally, the number of queries that Metadata Manager issues during an incremental load can exceed the number of queries that Metadata Manager issues during a full load. This might happen when there are many metadata changes, incremental loading is enabled, and resource loads occur more than two weeks apart. To reduce load time, do not enable incremental loading when the Hadoop cluster is heavily used and you schedule loads every two weeks or more.
When the JSON extraction time for the incremental load exceeds the JSON extraction time for a full load.
When the Hadoop cluster is heavily used, the JSON extraction time for each extraction operation increases. If the JSON extraction time for an incremental load exceeds the JSON extraction time for full metadata load, do not enable incremental loading. To find the JSON extraction time, check the Metadata Manager load log.