Data Profiling

Back Next

Automatic rule association with source objects

Data Profiling

automatically associates

Data Quality

assets as rules with columns, based on the column and source object name match. By default,

Data Profiling

associates rules with columns of Oracle, Flat File, ODBC, and Amazon S3 V2 connections.

To enable automatic rule association, make sure that you have a valid DataQualityClairRule package license for your organization. The DataQualityClaireRule package contains the connection-specific JSON files and a default JSON file.

Data Profiling

uses the connection-specific JSON file for all the supported connections. To enable automatic rule association for the remaining connections, you can configure the

DefaultAutoAssignRulesConfig.json

file.

Data Profiling

automatically associates rules with columns after you configure the

<connection_type>AutoAssignRulesConfig.json

file for the connection. You can configure the JSON file in the following location:

<secureagentlocation>/apps/Data_Integration_Server/data/profiling/AutoRuleAssignmentConfig/

You need not restart the Secure Agent after you configure or customize the Config.json files.

When you configure the

AutoAssignRulesConfig.json

file for a specific connection, the Data Quality assets are assigned as rules to the matching column and source object names.

If the column names and source object names do not match the auto assign criteria in the connection

AutoAssignRulesConfig.json

file.

Data Profiling

assigns rules to matching results from the

DefaultAutoAssignRulesConfig.json

file.

If column and source names in the

AutoAssignRulesConfig.json

file do not match the automatic rule association criteria, you can edit the connection-specific JSON file to change the source object name and column names.

Example

You created a profile with a source object that contains columns named ID, First Name, and Last Name. You might want to assign the Employee_details rule to the columns automatically.

To achieve this goal, you must ensure that you have the DataQualityClaireRule package license in your organization, and then copy the ClouldDataQuality_Bundles from Administrator Service to the ClouldDataQuality_Bundles project. If the column names in the

AutoAssignRulesConfig.json

file match the source column names in the source object,

Data Profiling

automatically assigns the Employee_details rule to the columns.

Automatic rule association steps

Ensure that the organization has the DataQualityClaireRule
package license enabled.

In Data Profiling service, create a project named CloudDataQuality_Bundles
.

Copy the CloudDataQuality_Bundles
bundle from

Administrator service

Add-On Bundles

Available Bundles

to the CloudDataQuality_Bundles
project. After you copy the bundle to the project, the project displays all the

Data Quality

assets that you can use for automatic rule association.

In the

<secureagentlocation>/apps/Data_Integration_Server/data/profiling/AutoRuleAssignmentConfig/

location, configure the

<connection_type>AutoAssignRulesConfig.json.file

with the data source information based on your requirements as shown in the following sample image:

View the associated rules in

Data Profiling

The following image shows the associated rules with the source objects:

The

Rules

tab displays (Recommended by CLAIRE)
as a suffix in the rule description.

Customize an AutoAssignRulesConfig.json file

In this scenario,

Data Profiling

contains a profile with a source object named Employee
and column named First Name
. The column names and source names that are present in the source object do not match in

AutoAssignRulesConfig.json

file. You might want to customize the

AutoAssignRulesConfig.json

file to add rules to columns to match the source names and source objects.

Existing Field Value	Customized Field Value
Change the sourceName field value from French_Company_Names	Change to Employee
Change the columnName field value from longitude	Change to First Name
The following image shows a sample AutoAssignRulesConfig.json file with existing source and column names:	The following image shows the changes made to the AutoAssignRulesConfig.json file: