After you choose the Databricks Spark option, you can select a Databricks connection. The Data Integration Service pushes the profile logic to the Spark engine on the Databricks cluster to run profiles.

When you run a profile in the Databricks environment, the Developer tool submits the profile jobs to the Profiling Service Module. The Profiling Service Module then breaks down the profile jobs into a set of mappings. The Data Integration Service pushes the mappings to the Spark engine through the Hadoop connection. The Spark engine processes the mappings and the Data Integration Service writes the profile results to the profiling warehouse.