Getting Started with Cloud Data Profiling REST API

Getting Started with Cloud Data Profiling REST API

Enable CLAIRE insights

Enable CLAIRE insights

Insights is a method for discovering data quality issues in your data. The issues can range from anomalous data values to complex inconsistencies. Insights in Data Profiling automates the process to detect data quality issues. The CLAIRE™ artificial intelligence engine provides insights and generates recommendations for your data that you can approve or reject.

Enable CLAIRE

POST method
Use the POST method to create a profile with CLAIRE configuration. You can enable CLAIRE to generate insights recommendations for your data that you can approve or reject.
The following table lists the parameter that you need to specify:
Property
Value
URL
Enter a URL to fetch the right profile. Use the following format: https://usw3-dqprofile.dm-us.informaticacloud.com/profiling-service/api/v1/profile
IDS-SESSION-ID
REST API session ID for the current session. Use in most REST API request headers. For example,
klCIHW6BvWmeFFwlQg7q84
API response code
201 Created
Request body
{ "connectionId": "4sTCN6J0Ea3b3wge6etcm4", "description": "Creating profile using Automation.", "drillDownType": "ON", "name": "CreateProfileWithProfilingService_667523", "profileableFields": [ { "fieldName": "VARCHAR2_COL","isDeleted": false,"sourceName": "CP_ALL_SUPPORTED_DATATYPE","fieldType": "DATASOURCEFIELD"} ], "samplingOptions": { "samplingType": "ALL_ROWS"} , "runtimeOptions": { "profileAdvProps": { "enableClaireAnomalyDetection": true} , "defaultEmailNotification": true }, "source": { "dataSourceType": "UNSET", "fields": [ { "dataType": "varchar","isDeleted": false,"name": "VARCHAR2_COL","order": 0, "precision": 20,"scale": 0,"pcType": "STRING" } ], "name": "CP_ALL_SUPPORTED_DATATYPE", "sourceType": "DATASOURCE" }, "frsProjectId": "0xOo7gZMYfxbEhZbV92y4q" }
In the
createProfile
request body, you can set the
enableClaireAnomalyDetection
option to true in the
runtimeOptions
Json object.
The following is an example of the
runtimeOptions
Json object configuration:
"runtimeOptions": { "profileAdvProps": { "enableClaireAnomalyDetection": true }, "defaultEmailNotification": true}

Retrieve insights from a profile

GET method
Use the GET method to retrieve insights from a profile.
The following table lists the parameter that you need to specify:
Property
Value
URL
Enter a URL to fetch insights for a profile. Use the following format: https://<ProfileBaseURI>/metric-store/api/v1/odata/Profiles('ProfileId)/Insights
IDS-SESSION-ID
REST API session ID for the current session. Use in most REST API request headers. For example,
klCIHW6BvWmeFFwlQg7q84
Response body
{ "@odata.context": "$metadata#Insights", "value": [ { "
columnKey
": 10001, "confirmationStatus": "Approved", "appliedColumn": "VARCHAR2_COL", "
insightId
": "1000", "insightType": "Completeness Check", "profileKey": 6085, "insightScore": "High", "insightStatement": "Data appears incomplete. The column includes one or more null, blank, or empty values or values that contain only zeros." } , { "columnKey": 10001, "confirmationStatus": null, "appliedColumn": "VARCHAR2_COL", "insightId": "1001", "insightType": "Column Length Deviation", "profileKey": 6085, "insightScore": "High", "insightStatement": "The length of the data values in the column has a high standard deviation." } , { "columnKey": 10001, "confirmationStatus": null, "appliedColumn": "VARCHAR2_COL", "insightId": "1015", "insightType": "Column Token Deviation", "profileKey": 6085, "insightScore": "Medium", "insightStatement": "The number of tokens in the column has a high standard deviation." } ] }
API response code
200 OK

Approve or reject insights

PATCH method
Use the PATCH method to approve or reject the insights.
The following table lists the parameter that you need to specify:
Property
Value
URL
Enter a URL to fetch insights for a profile. Use the following format: https://dqprofile-qa-pod1.mrel.infaqa.com/metric-store/api/v1/Profiles('157166de-dcd5-4839-bca9-967049dc3a29')/Insights
IDS-SESSION-ID
REST API session ID for the current session. Use in most REST API request headers. For example,
klCIHW6BvWmeFFwlQg7q84
Request body
{ "curatedInsights": [ { "confirmationStatus": "Approved", "insightId": 1001, "columnKey": 10001 } ] }
You can fetch the insightId and columnKey values from the response body of the
Retrieve insights from a profile
API call.
Response body
[ { "10001": { "profileId": "157166de-dcd5-4839-bca9-967049dc3a29", "columnId": "10001", "message": "Success", "status": "200 OK", "insightId": "1001" } } ]
API response code
200 OK

0 COMMENTS

We’d like to hear from you!