Hi, I'm Ask INFA!
What would you like to know?
ASK INFAPreview
Please to access Ask INFA.

Table of Contents

Search

  1. Preface
  2. Introduction to Databricks Connector
  3. Connections for Databricks
  4. Mappings for Databricks
  5. Migrating a mapping
  6. SQL ELT with Databricks Connector
  7. Data type reference
  8. Troubleshooting

Databricks Connector

Databricks Connector

Configuring SQL ELT optimization

Configuring
SQL ELT optimization

To optimize a mapping, add the mapping to a task, and then configure
SQL ELT optimization
in the
mapping
task.
  1. Create a
    mapping
    task.
  2. In the
    SQL ELT optimization
    section on the
    Runtime Options
    tab, set the
    SQL ELT optimization
    value to
    Full
    or
    To Source
    .
  3. If full
    SQL ELT optimization
    is not available, select how Data Integration handles
    SQL ELT optimization
    in the
    Fallback Option
    menu:
    • Partial PDO. Default. Data Integration pushes as much transformation logic as possible to the source and target database. The task processes any transformation logic that it can't push to a database. You can use Partial PDO only when you read from and write to Databrick.
    • Non PDO. The task runs without
      SQL ELT optimization
      .
    • Fail Task. Data Integration fails the task.
    The fallback options are not applicable to mappings in advanced mode.
When you run the mapping task, the transformation logic is pushed to the Databricks database.

0 COMMENTS

We’d like to hear from you!