You must configure a Databricks connection with simple or hybrid mode when you enable
SQL ELT optimization
in a mapping task.
Source properties
When you configure
SQL ELT optimization
, the mappings support the following advance properties for a Databricks source:
Source Object Type
Single
Multiple
Query
Parameter
When you use the query source type to read from Databricks, you can choose to retain the field metadata and save the mapping. Even if you edit the query and run the mapping, the field metadata specified at design time is retained.
Query Options
Filter. You can use both simple and advanced filter conditions.
Database Name
Table Name
SQL Override
Contains, Ends With, and Starts With filter operators are not applicable when you use source filter to filter records.
Target properties
When you configure
SQL ELT optimization
, the mappings support the following properties for an Databricks target:
Target Object Type
Single
Parameter
Create New at Runtime
Operation
Insert
Update
Upsert
Delete
Data driven
Create Target
Target Database Name
Target Table Name
Update Mode
Write Disposition for Insert operation.
You cannot run pre-SQL or post-SQL commands in the source and target when you configure mappings for full
SQL ELT optimization
.
Lookup properties
When you configure
SQL ELT optimization
, the mappings support the following advance properties for a Databricks lookup:
Source Object Type
Single
Query
Parameter
Multiple Matches for cached lookup
Un-cached unconnected lookup is not supported for
SQL ELT optimization
. For cached lookup, only
Return all rows
is supported.
Database Name
Table Name
SQL Override
If you configure advanced properties that are not supported, the Secure Agent either ignores the properties or logs a
SQL ELT optimization
validation error in the session logs file. If the Secure Agent logs an error in the session log, the mappings run in the Informatica runtime environment without full
ASK INFAPreview
