JupyterLab Extension for INFACore

Back Next

Convert to INFACore DataFrame and write data

You can configure the Write operation to write data to a target endpoint. If the data is in the Pandas DataFrame, you can convert it to the INFACore DataFrame before you write to the target.

You can configure the write and the pandas operations on the

Apply Functions

tab.

You can choose to convert the data that you read from the Pandas DataFrame to the INFACore DataFrame, and then configure a write operation.

In this example, we use a Snowflake connection to write customer data to a Snowflake target.

To first convert the data from the Pandas DataFrame to the INFACore DataFrame, select

From Pandas

Enter the variable name as p_df for the Pandas DataFrame, and click

The following code appears in the JupyterLab workspace. You can run the code to convert the data from the Pandas DataFrame to the INFACore DataFrame.

df_writer = ic.DataFrameWriter()
idf_1 = df_writer.from_pandas(p_df)

To write the data to the target, select

Write

on the

General

tab.

Select the data source and the connection to write to the data source.

If you have already select the data source and connection, and then configure the write operation, the data source and connection are prepopulated in the Write operation dialog box.

Enter the path to the target data object to write the data.

Click

Submit

You enter the data source and connection name for the target object, along with the object name to generate the write function code in the workspace.

The following code appears in the JupyterLab workspace. You can run the code to write the data to the target.

write_data_object = ic.get_data_source("Snowflake").get_connection("Snow_Conn").get_data_object("CONDLT/DLT_SCHEMA_01/CUSTOMERS_tgt")
write_option = {
"create_target": True
}
write_data_object.write(idf_1,write_option)

When you run the code, the SDK writes the data to the target. The following code snippet shows sample target entries:

[{'@type': 'activityLogEntry',
  'id': '0154JUC100000000058X',
  'type': 'MTT',
  'objectId': '0154JU0Z0000000000BC',
  'objectName': 'mct_1676633702326',
  'runId': 1,
  'startTime': '2023-02-17T06:40:24.000Z',
  'endTime': '2023-02-17T06:41:58.000Z',
  'startTimeUtc': '2023-02-17T11:40:24.000Z',
  'endTimeUtc': '2023-02-17T11:41:58.000Z',
  'state': 1,
  'failedSourceRows': 0,
  'successSourceRows': 3,
  'failedTargetRows': 0,
  'successTargetRows': 3,
  'startedBy': 'infacore_cai01',
  'runContextType': 'REST_API_V2',
  'entries': [{'@type': 'activityLogEntry',
    'id': '200105967',
    'type': 'MTT',
    'objectName': '',
    'runId': 1,
    'agentId': '0154JU0800000000000K',
    'runtimeEnvironmentId': '0154JU2500000000000K',

To use the INFACore SDK for python directly to write data to a data source, invoke the from_pandas() method to convert the Pandas DataFrame to the INFACore DataFrame,.and then call the write() method on the target DataObject instance.

Rename Saved Search

Table of Contents

JupyterLab Extension for INFACore

JupyterLab Extension for INFACore

Convert to INFACore DataFrame and write data

Convert to INFACore DataFrame and write data