REST API Reference

Back Next

Updating fields in objects with non-flat file formats

Use the fields resource to update fields in a source or target object in a non-flat file format such as Avro, Parquet, ORC, or JSON.

POST request

For Avro, Parquet, ORC, and JSON formats, you can update fields in the source or target object by including the format type and optionally, the schema. If you don't include the schema, the schema format is inferred from the data file.

Use one of the following URIs:

For a source object, use

/api/v2/connection/source/<id>/fields?objectName=<object name>

For a target object, use

/api/v2/connection/target/<id>/fields?objectName=<object name>

The following table describes the attributes to include in the dataFormat object:

Field	Type	Required	Description
formatId	String	Yes	Format type, for example, Avro.
schema	String	--	Schema format.

POST request example

To update fields in a Parquet source object, you might use the following request:

POST <serverUrl>/api/v2/connection/source/0100000B00000000000F/fields?objectName=infa.qa.bucket%2Fcustomer.parquet
1.0
Content-Type: application/xml
Accept: application/xml
icSessionId: <icSessionId>
{
    "@type": "dataFormat",
    "dformatId": "Parquet",
    "schema": "message AllData_root { optional int32 c_custkey; optional binary c_name (UTF8); optional binary c_address (UTF8); optional int64 c_nationkey; optional binary c_phone (UTF8); optional double c_acctbal; optional binary c_mktsegment (UTF8); required binary c_comment (UTF8);}"
}

A successful response might look like the following example:

[
    {
        "@type": "field",
        "id": -1,
        "name": "c_custkey",
        "type": "parquet_int32",
        "uniqueName": "c_custkey",
        "label": "c_custkey",
        "parentObject": "customer_tgt.parquet",
        "pcType": "INTEGER",
        "precision": 10,
        "scale": 0,
        "columnIndex": -1,
        "isKey": false,
        "isExternalId": false,
        "isSfIdLookup": false,
        "isNullable": true,
        "isUnique": false,
        "isCreateable": false,
        "isUpdateable": true,
        "isFilterable": true,
        "isCalculated": false,
        "javaType": "java.lang.Integer",
        "showLabel": true,
        "naturalOrder": 0,
        "linkedFields": [],
        "relatedInfos": [],
        "references": []
    },
    {
        "@type": "field",
        "id": -1,
        "name": "c_address",
        "type": "parquet_string",
        "uniqueName": "c_address",
        "label": "c_address",
        "parentObject": "customer_tgt.parquet",
        "pcType": "NSTRING",
        "precision": 4000,
        "scale": 0,
        "columnIndex": -1,
        "isKey": false,
        "isExternalId": false,
        "isSfIdLookup": false,
        "isNullable": true,
        "isUnique": false,
        "isCreateable": false,
        "isUpdateable": true,
        "isFilterable": true,
        "isCalculated": false,
        "javaType": "java.lang.String",
        "showLabel": true,
        "naturalOrder": 2,
        "linkedFields": [],
        "relatedInfos": [],
        "references": []
    },
    {
        "@type": "field",
        "id": -1,
        "name": "c_nationkey",
        "type": "parquet_int64",
        "uniqueName": "c_nationkey",
        "label": "c_nationkey",
        "parentObject": "customer_tgt.parquet",
        "pcType": "BIGINT",
        "precision": 19,
        "scale": 0,
        "columnIndex": -1,
        "isKey": false,
        "isExternalId": false,
        "isSfIdLookup": false,
        "isNullable": true,
        "isUnique": false,
        "isCreateable": false,
        "isUpdateable": true,
        "isFilterable": true,
        "isCalculated": false,
        "javaType": "java.math.BigInteger",
        "showLabel": true,
        "naturalOrder": 3,
        "linkedFields": [],
        "relatedInfos": [],
        "references": []
    },
    {
        "@type": "field",
        "id": -1,
        "name": "FileName",
        "type": "string",
        "uniqueName": "FileName",
        "label": "FileName",
        "parentObject": "customer_tgt.parquet",
        "pcType": "NSTRING",
        "precision": 1024,
        "scale": 0,
        "columnIndex": -1,
        "isKey": false,
        "isExternalId": false,
        "isSfIdLookup": false,
        "isNullable": false,
        "isUnique": false,
        "isCreateable": false,
        "isUpdateable": true,
        "isFilterable": true,
        "isCalculated": false,
        "javaType": "java.lang.String",
        "showLabel": true,
        "naturalOrder": 8,
        "linkedFields": [],
        "relatedInfos": [],
        "references": []
    }
]

Fields

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal