Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Mappings
  4. Sources
  5. Targets
  6. Transformations
  7. Cluster Workflows
  8. Profiles
  9. Monitoring
  10. Hierarchical Data Processing
  11. Hierarchical Data Processing Configuration
  12. Hierarchical Data Processing with Schema Changes
  13. Intelligent Structure Models
  14. Stateful Computing
  15. Connections
  16. Data Type Reference
  17. Function Reference

User Guide

User Guide

Rules and Guidelines for Mappings on the Databricks Spark Engine

Rules and Guidelines for Mappings on the Databricks Spark Engine

Consider the following run-time differences on the Databricks Spark engine:
  • Set the optimizer level to none or minimal if a mapping validates but fails to run. If you set the optimizer level to use cost-based or semi-join optimization methods, the Data Integration Service ignores this at run-time and uses the default.
  • The run-time engine does not honor the early projection optimization method in all cases. If the Data Integration Service removes the links between unused ports, the run-time engine might reconnect the ports.

0 COMMENTS

We’d like to hear from you!