Getting Started with Cloud Data Profiling REST API

Getting Started with Cloud Data Profiling REST API

Prerequisites

Prerequisites

Before you create a profile, you need to perform the following steps:
Create a new connection
Use this REST API call to create a connection of any type, for example, Oracle or ODBC. You can create a data profiling task using the source connection. Alternatively, you can create connections in Administrator.
The following table lists the API parameters to create a new connection:
Property
Value
URL
Enter a URL to create a connection. Use the following format: https://usw3.dm-us.informaticacloud.com/saas/api/v2/connection
Method
POST
Body
"@type": "connection", "name": "Oracle_Connection", "runtimeEnvironmentId": "010YJH25000000000001", "host": "<your connection host>", "database":"<database of your connection>", "codepage": "UTF-8", "type": "Oracle", "port":1522, "username":"<username>", "password": "<Password>", "connParams": { "CryptoProtocolVersion": "TLSv1.2", "keyStore": "<Keystore file location with filename and extension>", "keyStorePassword": "<Keystore password>", "encryptionMethod":"SSL", "ValidateServerCertificate":"True", "trustStore": "<trustore file location with filename and extension>", "trustStorePassword":""yourPassword4", "HostNameInCertificate":"QA18C4", "Metadata Advanced Connection Properties": "ValidateServerCertificate=0;LoginTimeout=300; ConnectionRetryCount=5; ConnectionRetryDelay=5", "Run-time Advanced Connection Properties": "ValidateServerCertificate=0;LoginTimeout=300; ConnectionRetryCount=5; ConnectionRetryDelay=5" } }
Create a project or folder
Use this REST API call to create projects and folders. You can create projects and folders to organize the profiling data.
The following table lists the API parameters to create a new project:
Property
Value
URL
Enter a URL to create a project. Use the following format: https://usw3.dm-us.informaticacloud.com/frs/v1/Project
Method
POST
Body
{"name":"Project","description":""}
The following table lists the API parameters to create a new folder:
Property
Value
URL
Enter a URL to create a folder. Use the following format: https://usw3.dm-us.informaticacloud.com/frs/v1/Projects('#projectId#')/Folders
Method
POST
Body
{"name":"foldername","description":""}
Retrieve a project ID
Use this REST API call to retrieve a project ID from the available projects and folders. You need to specify the project ID during profile creation so that you can organize your data profiling tasks.
The following table lists the API parameters to retrieve available projects for a profile:
Property
Value
URL
Enter a URL to fetch the project ID. Use the following format: https://usw3.dm-us.informaticacloud.com/frs/v1/Projects?$filter=(name eq <'projectName'>)
For example, https://usw3.dm-us.informaticacloud.com/frs/v1/Projects?$filter=(name eq 'project_Orderdetails')
Method
GET
Sample response
After you make the API call, you need to search for the project name in the response, and then copy the project ID.
The project ID is highlighted in the following response example:
{ "@odata.context": "$metadata#Projects", "value": [ { "id": "
2Nw2tVk5djQeSFNDrgglob
", "name": "Employee_Details", "description": "", "owner": "8x2xReMdLMdbbFg2kaRyyA", "createdBy": "8x2xReMdLMdbbFg2kaRyyA", "lastUpdatedBy": "8x2xReMdLMdbbFg2kaRyyA", "lastAccessedBy": "8x2xReMdLMdbbFg2kaRyyA", "createdTime": "2020-02-13T18:13:34Z", "lastUpdatedTime": "2020-02-13T18:13:34Z", "lastAccessedTime": "2020-02-13T18:13:34Z", "expiresBy": "2038-01-18T00:00:00Z", "parentInfo": [ { "parentId": "7cCn5thwWFLhiZoSosphKL", "parentName": "REG", "parentType": "Space" } ], "documentType": "Project", "contentType": "Binary", "documentState": "COMPLETE", "aclRule": "org", "subcontainerCount": 0, "customAttributes": null, "repoInfo": null, "isSourceControlled": false, "checkedOutById": null } ] }
Retrieve a connection ID
Use this REST API call to request the available connections for the profile creation. You need a connection to the source object so that you can create a data profiling task.
The following table lists the API parameters to retrieve available connections for a profile:
Property
Value
URL
Enter a URL to fetch the connection Id. Use the following format: https://usw3.dm-us.informaticacloud.com/frs/v1/Spaces('SYS')/Documents?filter=(documentType eq 'SAAS_CONNECTION')
Method
GET
Sample response
After you make the API call, you need to search for the connection name in the response, and then copy the connection ID.
The connection ID is highlighted in the following response example:
{ "@odata.context": "$metadata#Documents", "value": [{ "id": "
h40VFSvMcCpj1CEHDMHLgY
", "name": "ORA_SRC", "description": null, "owner": "4W7k7eaQOKub1SWbP32qCv", "createdBy": "4W7k7eaQOKub1SWbP32qCv", "lastUpdatedBy": "4W7k7eaQOKub1SWbP32qCv", "lastAccessedBy": "4W7k7eaQOKub1SWbP32qCv", "createdTime": "2020-10-29T12:35:00Z", "lastUpdatedTime": "2020-10-29T12:35:00Z", "lastAccessedTime": "2020-10-29T12:35:00Z", "expiresBy": "2038-01-18T00:00:00Z", "parentInfo": [{ "parentId": "bynSJ9htKIAlQPHWHUrnW1", "parentName": "SYS", "parentType": "Space" }, { "parentId": "1eTiMw87SE3f5LsNfQ6Z8D", "parentName": "_SYSTEM_PROJECT", "parentType": "Project" }, { "parentId": "anEbYk7U5aPgluVbKR8qwj", "parentName": "_SYSTEM_FOLDER", "parentType": "Folder" } ], "documentType": "SAAS_CONNECTION", "contentType": "JSON", "documentState": "COMPLETE", "aclRule": "org", "subcontainerCount": 0, "customAttributes": { "stringAttrs": [{ "name": "CONNECTION_TYPE", "value": "ORACLE" }, { "name": "CONNECTION_SUB_TYPE", "value": "" }, { "name": "CONNECTION_INSTANCE_NAME", "value": "" } ], "numberAttrs": [], "dateAttrs": [] }, "repoInfo": { "repoParams": [], "repoPath": null, "repoHandle": "01111O0B00000000000J", "frsPath": null }, "isSourceControlled": false, "checkedOutById": null }] }
Retrieve objects in a connection
Use this REST API call to request the available objects in a connection. You run a profile on the source objects.
The following table lists the API parameters to retrieve objects from a flat file connection:
Property
Value
URL
Enter a URL to fetch objects in a connection. For example, https://usw3.dm-us.informaticacloud.com/saas/api/v2/connection/source/0100000B000000000004?searchPattern=
Method
GET
Sample response
[ { "@type": "connListItem", "id": "0", "name": "ALL_DQ_Source1.csv" }, { "@type": "connListItem", "id": "1", "name": "Azure_DW_CP_All_Datatype.csv" }, { "@type": "connListItem", "id": "2", "name": "CP_ALL_SUPPORTED_DATATYPE.csv" }, { "@type": "connListItem", "id": "3", "name": "CP_CLEANSE_ALL_STEPS.csv" }, { "@type": "connListItem", "id": "4", "name": "CUSTOMERS_0.csv" }, { "@type": "connListItem", "id": "5", "name": "CUSTOMERS_1000000.csv" }, { "@type": "connListItem", "id": "6", "name": "CUSTOMERS_1250000.csv" }, { "@type": "connListItem", "id": "7", "name": "CUSTOMERS_1500000.csv" }, ]
Retrieve fields in an object
Use this REST API call to request the available fields in an object. The profile runs on the selected fields to extract column statistics.
The following table lists the API parameters to retrieve fields in an object from a flat file connection:
Property
Value
URL
Enter a URL to fetch an object from a flat file connection. Use the following format: https://usw3.dm-us.informaticacloud.com/saas/api/v2/connection/source/0100000B000000000004/field/ CP_ALL_SUPPORTED_DATATYPE.csv
Method
POST
Body
{"@type":"flatFileAttrs","delimiter":",","textQualifier":"\"","escapeChar":"", "firstDataRow":2,"headerLineNo":1}
The following table lists the API parameters to retrieve fields in an object from an Oracle connection:
Property
Value
URL
Enter a URL to fetch an object from a Oracle connection. Use the following format: https://usw3.dm-us.informaticacloud.com/saas/api/v2/connection/source/0100000B00000000000C/field/CP_ALL_SUPPORTED_DATATYPE?validateFields=false
Method
POST
Query String Parameters
validateFields=false
Sample response
[ { "@type": "field", "id": -1, "name": "VARCHAR2_COL", "type": "nstring", "uniqueName": "VARCHAR2_COL", "label": "VARCHAR2_COL", "parentObject": "CP_ALL_SUPPORTED_DATATYPE.csv", "pcType": "UNISTRING", "precision": 255, "scale": 0, "columnIndex": 0, "isKey": false, "isExternalId": false, "isSfIdLookup": false, "isNullable": true, "isUnique": false, "isCreateable": false, "isUpdateable": true, "isFilterable": true, "isCalculated": false, "showLabel": true, "naturalOrder": 0, "linkedFields": [], "relatedInfos": [], "references": [] }, ]
Retrieve a profile ID
Use this REST API call to request the available profiles in the organization.
The following table lists the API parameters to retrieve available profile jobs.
Property
Value
URL
Enter a URL to fetch available profile jobs. Use the following format: https://usw3.dm-us.informaticacloud.com/saas /profiling-service/api/v1/profile
Method
GET
Sample response
After you make the API call, you need to search for the profile name in the response, and then copy the profile ID.
The profile ID is highlighted in the following response example:
[ { "id": "
033f72cb-fd70-4bdd-8ad2-376a41670e27
", "profileKey": 126, "version": 0, "name": "FFConnectionsList_Check4697169", "orgId": "7zmpu0DNPKAklXvmGgxSEI", "description": "Creatin", "frsId": "4ZIE81BXmzslJdflhLFB0N", "frsProjectId": "7Rs1Dt7gmK1loZmnheL7kA", "connectionId": "cV3icFm4uMScXbp2k44fOH", "createdBy": "8qPaKBgSQV2jN2V9O08DO4", "createdByName": "DQQA_R35POD3_USER1", "createTime": 1601883225651, "updatedBy": "8qPaKBgSQV2jN2V9O08DO4", "updatedByName": "DQQA_R35POD3_USER1", "updateTime": 1601883225653, "isFilterEnabled": null, "samplingOptions": { "id": null, "rows": -1, "samplingType": "ALL_ROWS" }, "drillDownType": "ON" }, { "id": "06b3a271-6e99-47e0-a733-64210bd86632", "profileKey": 511, "version": 1, "name": "CreateDictionaryUsingAllValuesOfVarcharColumn1837479", "orgId": "7zmpu0DNPKAklXvmGgxSEI", "description": "", "frsId": "euzYO70jUUSj8HdJsLshEh", "connectionId": "52fFSjJOp61koaaUyZtisR", "createdBy": "8qPaKBgSQV2jN2V9O08DO4", "createdByName": "DQQA_R35POD3_USER1", "createTime": 1603554782224, "updatedBy": "8qPaKBgSQV2jN2V9O08DO4", "updatedByName": "DQQA_R35POD3_USER1", "updateTime": 1603554830660, "isFilterEnabled": null, "samplingOptions": { "id": null, "rows": -1, "samplingType": "ALL_ROWS" }, "drillDownType": "ON" } ]

0 COMMENTS

We’d like to hear from you!