Mappings

Back Next

Mappings

A mapping defines reusable data flow logic that you can use in

mapping

tasks.

Use a mapping to define data flow logic that is not available in

synchronization

tasks, such as specific ordering of logic or joining sources from different systems.

You can create the following types of mappings:

Mapping: Create a mapping when you need flexibility in your sources, targets, and transformation options. A mapping can read and write to a wide variety of heterogeneous data sources. It also offers a large variety of data transformation options.

When you run a mapping,
Data Integration
processes some or all of the transformation logic. You can choose to push some or all the transformation logic to the source, to the target, or both.
Data Integration
processes any transformation logic that isn't pushed to the sources and targets.
Mapping in advanced mode: Create a mapping in advanced mode when you want to process multilevel hierarchical data, embedded code snippets, and workloads at any scale.

A mapping in advanced mode requires an advanced cluster to run the mapping logic. When you start running mappings in advanced mode,
Data Integration
can automatically create a local
advanced cluster
for you to use.

When you run a mapping in advanced mode, you can choose to push some or all the transformation logic to the source, to the target, or both.
Data Integration
processes any transformation logic that isn't pushed to the sources and targets.
Mapping in SQL ELT mode: Create a mapping in SQL ELT mode when your source and target are in the same cloud ecosystem and you want to perform the data transformation entirely within the cloud ecosystem. For example, you want to read data from your Snowflake cloud data warehouse or data lake, load it to your Snowflake cloud data warehouse, and perform all of the data transformation within Snowflake.

When you run a mapping in SQL ELT mode,
Data Integration
translates the transformation logic into ecosystem-specific SQL statements and commands that run in the underlying cloud data warehouse. This increases the data processing speed because the data isn't moved out of the cloud infrastructure for processing. It also increases the efficiency of the data integration pipelines.

Use the Mapping Designer to configure mappings. When you configure a mapping, you describe the flow of data from source to target. You can add transformations to transform data, such as an Expression transformation for row-level calculations or a Filter transformation to remove data from the data flow. A transformation includes field rules to define incoming fields. Links visually represent how data moves through the data flow.

You can configure parameters to enable additional flexibility in how you can use the mapping. Parameters act as placeholders for information that you define in the

mapping

task. For example, you can use a parameter for a source connection in a mapping, and then define the source connection when you configure the task.

You can use components such as mapplets, business services, and hierarchical schema definitions in mappings. Components are assets that support mappings. Some components are required for certain transformations while others are optional. For example, a business services asset is required for a mapping that includes a Web Service transformation. Conversely, a saved query component is useful when you want to reuse a custom query in multiple mappings, but a saved query is not required.

Mapping Designer

Mapping templates

Mapping configuration

Mappings in advanced mode

Mappings in SQL ELT mode

Automapping

Mapping validation

Data preview in mappings

Data preview in advanced mode

Detailed data preview in mapplets

SQL ELT query preview

Field lineage