Table of Contents

Search

  1. Preface
  2. Data Profiling
  3. Profiles
  4. Profile results
  5. Tuning data profiling task performance
  6. Troubleshooting

Data Profiling

Data Profiling

Review and act on insights

Review and act on insights

You can review the inferred insights generated by the CLAIRE engine in
Data Profiling
. Hover the mouse over the insight to approve or reject the insight. After you review the insights, you need to save and run the profile.
You can click on the or icons to approve or reject the insights. When you approve an insight, a rule specification is created in
Data Quality
and the rule is assigned to the profile automatically. After approval, the status of the insight changes to "Approved, Pending". When you save the profile, the status of the insight changes to "Approved". A rule specification is automatically created and appears on the
Rules
tab for the columns to which the insight is referred to. You must save the profile to persist the status and the rule association. Once an insight is approved, the insight cannot be removed from the profile unless you delete the corresponding rule specification from the
Rules
tab of the profile.
When you approve an insight, a rule specification is created that monitors the quality of the column corresponding to the insight. For example, an algorithm to detect outlier based on the value frequency length. CLAIRE recommends that any value frequency with length greater than 5 is invalid. A new rule specification is automatically created with the logic to verify for values greater than 5 and tags them as invalid. The rule specification is also automatically assigned to the column in the profile.
The following image shows the automatically created rule specifications assigned to the corresponding source columns in the profile on the
Rules
tab:
  • The name of the automatically created rules follow the rs_<source column name>_<insight type>_< sequential number> pattern.
  • The description of the automatically created rules are appended with the "recommended by CLAIRE" text.
When you reject an insight, the status of the insight changes to "Reject, Pending". You must save the profile to complete the rejection. When you save the profile, the status of the insight changes to "Disapproved".
To review multiple insight statements, select the insights and click on the or icons on the top of the Insights area.
You can reset a pending insight review. An insight with "Approved, Pending" or "Reject, Pending" status can be cleared. Hover the mouse over the insight and click on the icon to reset the review. The status of the insight disappears. You can click on the icon on the top of the Insights area to reset all the pending insights. You can select the
Hide Rejected Insights
checkbox to hide all the rejected insights.
If an insight is approved or rejected, the same algorithm is not used again for the same columns.
After review of the inferred insights, you can drill down and view the anomalous data values and complex inconsistencies on the
Results
tab. For more information about profile results, see Profile results. You can also create rule occurrences and view scorecards to measure data quality scores and monitor data quality progress for profiles. For more information about rule occurrences and scorecards, see Rule occurrences and scorecards.

0 COMMENTS

We’d like to hear from you!