Data Quality Getting Started Guide

10.5
- 10.5.6
- 10.4.0

Back Next

Profiling Data Overview

A profile is a set of metadata that describes the content and structure of a data set.

Profiling and data discovery is often the first step in a project. You can run a profile to evaluate the structure of data and verify that data columns are populated with the types of information you expect. If a profile reveals problems in data, you can define steps in your project to fix those problems. For example, if a profile reveals that a column contains values of greater than expected length, you can design data quality processes to remove or fix the problem values.

A profile that analyzes the data quality of selected columns is called a column profile.

You can also use the Developer tool to discover primary key, foreign key, and functional dependency relationships, and to analyze join conditions on data columns.

A column profile provides the following facts about data:

The number of distinct and null values in each column, expressed as a number and a percentage.

The patterns of data in each column, and the frequencies with which these values occur.

Statistics about the column values, such as the maximum and minimum lengths of values and the first and last values in each column.

For join analysis profiles, the degree of overlap between two data columns, displayed as a Venn diagram and as a percentage value. Use join analysis profiles to identify possible problems with column join conditions.

You can run a column profile at any stage in a project to measure data quality and to verify that changes to the data meet your project objectives. You can run a column profile on a transformation in a mapping to indicate the effect that the transformation will have on data.

Story

HypoStores wants to verify that customer data is free from errors, inconsistencies, and duplicate information. Before HypoStores designs the processes to deliver the data quality objectives, it needs to measure the quality of its source data files and confirm that the data is ready to process.

Objectives

In this lesson, you complete the following tasks:

Perform a join analysis on the

Boston_Customers

data source and the

LA_Customers

data source.

View the results of the join analysis to determine whether or not you can successfully merge data from the two offices.

Run a column profile on the

All_Customers

data source.

View the column profiling results to observe the values and patterns contained in the data.

Prerequisites

Before you start this lesson, verify the following prerequisite:

You have completed lessons 1 and 2 in this tutorial.

Time Required

Set aside 20 minutes to complete this lesson.

Lesson 3. Run a Profile on Source Data

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal

0 COMMENTS

We’d like to hear from you! Log in to comment.

Rename Saved Search

Table of Contents

Data Quality Getting Started Guide

Data Quality Getting Started Guide

Profiling Data Overview

Profiling Data Overview

Story

Objectives

Prerequisites

Time Required