Table of Contents

Search

  1. Abstract for Profiling Sizing Guidelines
  2. Supported Versions
  3. Profiling and Discovery Sizing Guidelines

Profiling and Discovery Sizing Guidelines

Profiling and Discovery Sizing Guidelines

Memory Worksheet

Memory Worksheet

When you estimate the amount of memory required for a combination of profile jobs, the recommended approach is to start with the minimal amount of recommended memory. The base temporary disk recommendation is 8 GB. You can then add more memory to the machine based on the types of profile jobs and the data that the profile jobs use.
Flat file sources perform better if the entire data source can fit into the buffer pools of the operating system. If the data sources are large, you can add additional memory.
Some of the profile operations require memory to cache the source flat files or for the profiling algorithms of column profile, key discovery, functional dependency discovery, and data domain discovery. Enter the expected number of parallel jobs for each profile job type in the "Concurrency" column. Concurrency is the expected number of jobs of each profile type that run in parallel. Multiply the values in the columns A and B for each row in the worksheet and update the "A x B" column.
Use the following worksheet to record the values:
Profile Operation
Concurrency (A)
Memory (B)
A x B
Scorecard (flat file)
2 GB
Column Profile (flat file)
2 GB
Data Domain Discovery (flat file)
2 GB
Primary Key Discovery
1 GB
Functional Dependency Discovery
1 GB
The base memory recommendation of 8 GB meets the memory requirements of the following profile operations:
  • Run a scorecard on a relational source
  • Run a column profile on a relational source
  • Perform data domain discovery on a relational source
  • Perform foreign key discovery
  • Perform overlap discovery
Calculation
Add all the values in the "A x B" column. You can then add the recommended base memory of 8 GB with the total value of the "A x B" column to compute the required memory.

0 COMMENTS

We’d like to hear from you!