Table of Contents

Search

  1. Preface
  2. Mappings
  3. Mapping tutorial
  4. Parameters
  5. CLAIRE recommendations
  6. Data catalog discovery

Mappings

Mappings

Mappings in SQL ELT mode

Mappings in SQL ELT mode

Create a mapping in SQL ELT mode when your source and target are in the same cloud ecosystem and you want all the mapping logic to be processed by your cloud ecosystem. For example, you need to move data in an Azure Data Lake Storage data lake to your Snowflake cloud data warehouse, and you want all the data processing to occur within Snowflake.
When you run a mapping in SQL ELT mode, the transformation logic is translated into ecosystem-specific commands and SQL statements that run in the underlying cloud infrastructure. Because data isn't moved out of the cloud infrastructure to transform it, the overall processing speed increases.
Mappings in SQL ELT mode can read data from a cloud data warehouse and write it to the same cloud data warehouse. They can also read data from a data lake in your cloud ecosystem and write it to a cloud data warehouse in the same ecosystem.
Mappings in SQL ELT mode can load data to the following targets and extract data from the following sources based on the target type:
Target
Supported sources
Amazon Redshift
  • Amazon Redshift
  • Amazon S3
Databricks
  • Databricks
  • Amazon S3
  • Microsoft Azure Data Lake Storage Gen2
Google BigQuery
  • Google BigQuery
  • Amazon S3
  • Google Cloud Storage
Microsoft Fabric Data Warehouse
  • Microsoft Fabric Data Warehouse
  • Microsoft Fabric Lakehouse
Snowflake Data Cloud
  • Snowflake Data Cloud
  • Amazon S3
  • Microsoft Azure Data Lake Storage Gen2
To create a mapping in SQL ELT mode, you create a mapping and select
Mapping - SQL ELT
as the mapping type. You're then prompted to choose a target connection. If your organization doesn't have any connections, you're prompted to create one.
After you choose the target connection, the Mapping Designer opens.
The following image shows the Mapping Designer for a mapping in SQL ELT mode:
The Mapping Designer shows the "SQL ELT Mode" label at the top, followed by an icon that represents the cloud ecosystem. The transformation palette appears on the left, and the Parameters and Validation panels are open on the right. The mapping canvas shows a mapping with a source, Filter transformation, and target. The mapping properties show the mapping name, location and description. The mapping properties also show the cloud provider and target connection at the bottom.
The mapping is automatically configured with the target connection you chose. You can add additional sources, targets, and transformations to the mapping. The transformations available in the transformation palette are transformations that the target cloud data warehouse can process. For example, Snowflake can't process the logic in an SQL transformation, so the transformation palette for mappings in SQL ELT mode doesn't include the SQL transformation.
To run a mapping in SQL ELT mode, you create a mapping task.

0 COMMENTS

We’d like to hear from you!