Table of Contents

Search

  1. Preface
  2. Using Metadata Manager
  3. Configuring Metadata Manager
  4. Application Resources
  5. Business Glossary Resources
  6. Business Intelligence Resources
  7. Custom Resources
  8. Data Integration Resources
  9. Data Modeling Resources
  10. Database Management Resources
  11. Universal Resources
  12. Managing Resources
  13. Loading and Monitoring Resources
  14. Managing Permissions
  15. Resource Migration
  16. Repository Migration
  17. Appendix A: Metadata Manager Login
  18. Appendix B: Metadata Manager Properties Files
  19. Appendix C: Resource Configuration Files
  20. Appendix D: Glossary

Metadata Manager Administrator Guide

Metadata Manager Administrator Guide

Incremental Loading Guidelines

Incremental Loading Guidelines

Consider the following guidelines before you enable incremental loading.
Do not enable incremental loading in the following situations:
When there are many metadata changes and loads occur more than two weeks apart.
An incremental load runs a set of queries that trigger multiple extraction operations from Cloudera Navigator. Occasionally, the number of queries that Metadata Manager issues during an incremental load can exceed the number of queries that Metadata Manager issues during a full load. This might happen when there are many metadata changes, incremental loading is enabled, and resource loads occur more than two weeks apart. To reduce load time, do not enable incremental loading when the Hadoop cluster is heavily used and you schedule loads every two weeks or more.
When the JSON extraction time for the incremental load exceeds the JSON extraction time for a full load.
When the Hadoop cluster is heavily used, the JSON extraction time for each extraction operation increases. If the JSON extraction time for an incremental load exceeds the JSON extraction time for full metadata load, do not enable incremental loading. To find the JSON extraction time, check the Metadata Manager load log.

0 COMMENTS

We’d like to hear from you!