Getting Started with Cloud Data Profiling REST API

Getting Started with Cloud Data Profiling REST API

Create a profile

Create a profile

Create a data profiling task to determine the characteristics of columns in a source object, such as value frequency, patterns, and data types. Data profiling tasks are also called profiles.

Prerequisite

Ensure that you have created a connection to the data source before you create a profile. To create a connection using REST APIs, see the Create a connection section.
In this tutorial, create a profile named
Profile_Employee_2
using an Oracle connection.
POST request
Use the POST method to create a profile.
The following table lists the parameters that you need to specify to create a profile:
Property
Value
URL
Enter a URL to create a profile. Use the following format: https://usw3-dqprofile.dm-ap.informaticacloud.com/profiling-service/api/v1/profile
IDS-SESSION-ID
REST API session ID for the current session. Use in most REST API request headers. For example,
klCIHW6BvWmeFFwlQg7q84
API parameters
The following table lists the important query parameters exposed for this API:
Parameter
Description
Example value
Name
Name of the Data Profiling asset.
Profile_Employee_2
orgId
A system-generated field that specifies the organization ID of the user.
8bK8rrYo6bLdbsS29jujik
Description
Description of the Data Profiling asset.
Employee Details.
frsProjectId
Specifies the ID of the project where the profile task is stored.
To retrieve the frsProjectId, see the Prerequisites section.
2Nw2tVk5dJQeSFNDrgglob
ConnectionID
Specifies the connection ID to connect to the data source.
To retrieve the ConnectionID, see the Prerequisites section.
h40VFSvMcCpj1CEHDMHLgY
SourceType
Type of source.
DATASOURCE
isDeleted
This indicates that the columns are no longer profileable.
False
fieldType
The source field type whether it can be a rule output port (MAPPLETFIELD) or source column (DATASOURCEFIELD).
DATASOURCEFIELD
SourceName
The source object to create the profile on. To view source objects in a connection, see the Retrieving objects in a connection section.
EMPLOYEES
Field Name
Profileable columns. To view the list of profileable columns, see the Retrieving fields in a source object section.
JOB_ID, LAST_NAME, FIRST_NAME
API response code
201 created.
API example in JSON format
{ "examples": { "oracleCreateRequestExample": { "value": { "name": "Profile_Employee_2", "description": Employee Details, "frsProjectId": "2Nw2tVk5dJQeSFNDrgglob", "connectionId": "h40VFSvMcCpj1CEHDMHLgY", "isFilterEnabled": true, "runtimeOptions": { "scheduleId": null, "runtimeEnvironmentId": null, "defaultEmailNotification": true, "profileAdvProps": { "maxTopN": null, "maxPatterns": null, "maxPatternThresholdPercent": null, "maxRanks": null, "inferDateTime": true, "detectOutliers": true, "maxColumnsPerMapping": null, "minNoOfRowsForSplitMapping": null, "maxMemory": null, "maxPercentMemory": null, "defaultBufferBlockSize": null, "dtmBufferSize": null, "lineSequentialBufferLength": null, "stopOnErrors": null } }, "source": { "name": "EMPLOYEES", "fields": [ { "name": "JOB_ID", "dataType": "varchar", "precision": 10, "scale": 0, "pcType": "STRING", "order": 6 }, { "name": "LAST_NAME", "dataType": "varchar", "precision": 25, "scale": 0, "pcType": "STRING", "order": 2 }, { "name": "FIRST_NAME", "dataType": "varchar", "precision": 20, "scale": 0, "pcType": "STRING", "order": 1 } ], "rules": [ { "name": "is_null", "description": null, "frsId": "40MVIkdn5GSc4TthVjPYCH", "inFields": [ { "name": "input_1", "label": null, "dataType": "string", "precision": 50, "scale": 0 } ], "outFields": [ { "name": "is_null", "label": null, "dataType": "string", "precision": 100, "scale": 0 } ], "ruleType": "RULE_SPECIFICATION" } ], "dataSourceType": "ORACLE", "properties": { "dataSourceType": "ORACLE" }, "sourceType": "DATASOURCE" }, "profileableFields": [ { "frsId": "40MVIkdn5GSc4TthVjPYCH", "inputFieldMappings": [ { "dataSourceFieldName": "SSN", "inFieldName": "input_1", "isDeleted": false } ], "outputFieldMappings": [ { "datatype": "string", "outFieldName": "is_null", "label": null, "isDeleted": false } ], "ruleType": "RULE_SPECIFICATION", "fieldType": "MAPPLETFIELD" }, { "sourceName": "EMPLOYEES", "fieldName": "FIRST_NAME", "fieldType": "DATASOURCEFIELD" } ], "filters": [ { "isEnabled": true, "name": "employee id greater than 100", "description": "employee id greater than 100", "fieldFilters": [ { "columnName": "EMPLOYEE_ID", "operator": "GREATER_THAN", "value": "100" } ], "filterType": "SIMPLE" } ], "samplingOptions": { "rows": -1, "samplingType": "ALL_ROWS" }, "drillDownType": "ON" } } } }

0 COMMENTS

We’d like to hear from you!