INFACore SDK for Python

Back Next

Step 2. Specify the remote agent engine details and the intermediate staging type

In your development environment, set the remote agent runtime details. Also, set Databricks as an intermediate storage with Databricks Delta as the connection. INFACore uses the connection to stage the data in the development environment when you read from or write data to an endpoint.

Perform the following tasks in the INFACore SDK Python code in your development environment to set the remote agent and use Databricks as an intermediate storage. :

Enter the compute engine type as Remote and the runtime environment name.

Enter the ecosystem type and the configured Databricks Delta connection name.

Here is an example Python code snippet that demonstrates how to use INFACore with the remote agent details in a Databricks environment:

user_config = {

    "login": {

        "username": "infacore",

        "password": "<password>",

        "env_type": "test",

        "url": "https://dmr-us.informaticacloud.com"

    },

    "compute_engine": {

        "type": "remote",

        "runtime_env_name": "<abc01.infa.com>",

        "install": False

    },

    "ecosystem" : {

      "type": "Databricks Delta",

      "staging" :{

          "connection_name" : "Databricks_AWS"

      }

    }

}

When you run the code, INFACore establishes a connection with Databricks Delta using the specified runtime environment. It uses the connection to stage the data in Databricks when you read from or write to the configured endpoint.

Procedure

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal