Preface
Understanding Domains
- Understanding Domains Overview
- Nodes
- Service Manager
- Application Services
- High Availability
- Informatica Data Usage Policy
  - Configuring Informatica DiscoveryIQ Proxy Details
  - Disabling Informatica Data Usage
Managing Your Account
- Managing Your Account Overview
- Password Management
  - Changing Your Password
- Preferences
- Informatica Network Credentials
  - Enter Informatica Network Credentials
  - Searching Informatica Knowledge Base
Using Informatica Administrator
- Using Informatica Administrator Overview
- Log In to Informatica Administrator
  - Informatica Administrator URL
  - Troubleshooting the Login to Informatica Administrator
- Manage Tab
- Manage Tab - Domain View
  - Details Panel
  - Resource Usage Indicators
- Manage Tab - Services and Nodes View
- Manage Tab - Connections View
- Manage Tab - Schedules View
- Monitor Tab
- Monitor Tab - Summary Statistics View
- Monitor Tab - Execution Statistics View
- Logs Tab
- Reports Tab
- Security Tab
- Service States
- Process States
- Job States
- Informatica Administrator Accessibility Overview
  - Keyboard Shortcuts
Using the Domain View
- About the Domain View
- Dependency Graph
  - Viewing Dependencies for Application Services, Nodes, and Grids
  - Recycling or Disabling Downstream Services
- Command History
- History View
  - Viewing History
  - Viewing Events
Domain Management
- Domain Management Overview
- Alert Management
- Folder Management
- Domain Security Management
- User Security Management
- Application Service Management
- Gateway Configuration
  - Configuring the Gateway and Worker Nodes
- Domain Configuration Management
- Rename the Domain
- Shutting Down a Domain
- Domain Properties
Nodes
- Nodes Overview
- Node Types
- Node Roles
- Define and Add Nodes
  - Adding Nodes to the Domain
- Configuring Node Properties
- Shutting Down and Restarting the Node
- Removing the Node Association
- Removing a Node
High Availability
- High Availability Overview
- Resilience
- Restart and Failover
  - Domain Failover
  - Application Service Restart and Failover
- Recovery
- Configuration for a Highly Available Domain
- Troubleshooting High Availability
Connections
- Connections Overview
- Connection Management
- Pass-through Security
  - Pass-Through Security with Data Object Caching
  - Adding Pass-Through Security
- Pooling Properties in Connection Objects
Connection Properties
- Connection Properties Overview
- Adabas Connection Properties
- Amazon Redshift Connection Properties
- Amazon S3 Connection Properties
- Blockchain Connection Properties
- Cassandra Connection Properties
- DataSift Connection Properties
- Databricks Connection Properties
- Facebook Connection Properties
- Greenplum Connection Properties
- Google Analytics Connection Properties
- Google BigQuery Connection Properties
  - Connection Modes
- Google Cloud Spanner Connection Properties
- Google Cloud Storage Connection Properties
- Hadoop Connection Properties
  - Hadoop Cluster Properties
  - Common Properties
  - Reject Directory Properties
  - Blaze Configuration
  - Spark Configuration
- HBase Connection Properties
- HDFS Connection Properties
- HBase Connection Properties for MapR-DB
- Hive Connection Properties
- HTTP Connection Properties
- IBM DB2 Connection Properties
- IBM DB2 for i5/OS Connection Properties
- IBM DB2 for z/OS Connection Properties
- IMS Connection Properties
- JDBC Connection Properties
- JDBC V2 Connection Properties
- JD Edwards EnterpriseOne Connection Properties
- LDAP Connection Properties
- LinkedIn Connection Properties
- Microsoft Azure Blob Storage Connection Properties
- Microsoft Azure Cosmos DB SQL API Connection Properties
- Microsoft Azure Data Lake Storage Gen1 Connection Properties
- Microsoft Azure Data Lake Storage Gen2 Connection Properties
- Microsoft Azure SQL Data Warehouse Connection Properties
- MS SQL Server Connection Properties
- Netezza Connection Properties
- OData Connection Properties
- ODBC Connection Properties
- Oracle Connection Properties
- Salesforce Connection Properties
- Salesforce Marketing Cloud Connection Properties
- SAP Connection Properties
- Sequential Connection Properties
- Snowflake Connection Properties
- Teradata Parallel Transporter Connection Properties
- Tableau Connection Properties
- Tableau V3 Connection Properties
- Twitter Connection Properties
- Twitter Streaming Connection Properties
- VSAM Connection Properties
- Web Content-Kapow Katalyst Connection Properties
- Web Services Connection Properties
- Identifier Properties in Database Connections
  - Regular Identifiers
  - Delimited Identifiers
  - Identifier Properties
Schedules
- Schedules Overview
- Create and Edit Schedules
  - Creating a Schedule
  - Editing a Schedule
- Pausing and Resuming a Schedule
- Removing Jobs from a Schedule
- Deleting a Schedule
Domain Object Export and Import
- Domain Object Export and Import Overview
- Export Process
  - Rules and Guidelines for Exporting Domain Objects
- View Domain Objects
  - Viewable Domain Object Names
- Import Process
  - Rules and Guidelines for Importing Domain Objects
  - Conflict Resolution
License Management
- License Management Overview
- Types of License Keys
  - Original Keys
  - Incremental Keys
- Creating a License Object
- Assigning a License to a Service
  - Rules and Guidelines for Assigning a License to a Service
- Unassigning a License from a Service
- Updating a License
- Removing a License
- License Properties
Monitoring
- Monitoring Overview
- Configuring Monitoring
  - Step 1. Configure Monitoring Settings
  - Step 2. Configure Reports and Statistics Views
- Optimizing Monitoring Performance
- Summary Statistics
  - Viewing Summary Statistics
- Monitor Data Integration Services
  - Properties View for a Data Integration Service
  - Reports View for a Data Integration Service
- Monitor Ad Hoc Jobs
- Monitor Applications
  - Properties View for an Application
  - Reports View for an Application
- Monitor Deployed Mapping Jobs
- Monitor Logical Data Objects
- Monitor SQL Data Services
- Monitor Web Services
- Monitor Workflows
- Job Status After Application Service Restart or Failover
- Monitoring a Folder of Objects
Log Management
- Log Management Overview
- Log Manager Architecture
- Log Location
- System Logs
- Log Management Configuration
- Using the Logs Tab
- Log Events
- Log Aggregator
  - Aggregating Application Service Logs
  - Processing Aggregated Application Service Logs
- Mapping Task Logs
Domain Reports
- Domain Reports Overview
- License Management Report
- Web Services Report
Node Diagnostics
- Node Diagnostics Overview
- Informatica Network Login
  - Logging In to the Informatica Network
- Generating Node Diagnostics
- Downloading Node Diagnostics
- Uploading Node Diagnostics
- Analyzing Node Diagnostics
  - Identify Bug Fixes
  - Identify Recommendations
Understanding Globalization
- Globalization Overview
  - Unicode
  - Working with a Unicode PowerCenter Repository
- Locales
- Data Movement Modes
  - Character Data Movement Modes
    - ASCII Data Movement Mode
    - Unicode Data Movement Mode
  - Changing Data Movement Modes
- Code Page Overview
- Code Page Compatibility
- Code Page Validation
- Relaxed Code Page Validation
- PowerCenter Code Page Conversion
  - Choosing Characters for PowerCenter Repository Metadata
- Case Study: Processing ISO 8859-1 Data
  - Configuring the ISO 8859-1 Environment
- Case Study: Processing Unicode UTF-8 Data
  - Configuring the UTF-8 Environment
Appendix A: Code Pages
- Supported Code Pages for Application Services
- Supported Code Pages for Sources and Targets
Appendix B: Custom Roles
- Analyst Service Custom Role
- Metadata Manager Service Custom Roles
- Operator Custom Role
- PowerCenter Repository Service Custom Roles
- Test Data Manager Custom Roles
Appendix C: Informatica Platform Connectivity
- Informatica Platform Connectivity Overview
- Domain Connectivity
  - Model Repository Connectivity
- PowerCenter Connectivity
- Native Connectivity
- ODBC Connectivity
- JDBC Connectivity
Appendix D: Configure the Web Browser
- Configure the Web Browser

Administrator Guide

10.4.0
- 10.5.6
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1

Back Next

Databricks Connection Properties

Use the Databricks connection to run mappings on a Databricks cluster.

A Databricks connection is a cluster type connection. You can create and manage a Databricks connection in the Administrator tool or the Developer tool. You can use infacmd to create a Databricks connection. Configure properties in the Databricks connection to enable communication between the Data Integration Service and the Databricks cluster.

The following table describes the general connection properties for the Databricks connection:

Property	Description
Name	The name of the connection. The name is not case sensitive and must be unique within the domain. You can change this property after you create the connection. The name cannot exceed 128 characters, contain spaces, or contain the following special characters:~ ` ! $ % ^ & * ( ) - + = { [ } ] \| \ : ; " ' < , > . ? /
ID	String that the Data Integration Service uses to identify the connection. The ID is not case sensitive. It must be 255 characters or less and must be unique in the domain. You cannot change this property after you create the connection. Default value is the connection name.
Description	Optional. The description of the connection. The description cannot exceed 4,000 characters.
Connection Type	Choose Databricks.
Cluster Configuration	Name of the cluster configuration associated with the Databricks environment. Required if you do not configure the cloud provisioning configuration.
Cloud Provisioning Configuration	Name of the cloud provisioning configuration associated with a Databricks cloud platform. Required if you do not configure the cluster configuration.
Staging Directory	The directory where the Databricks Spark engine stages run-time files. If you specify a directory that does not exist, the Data Integration Service creates it at run time. If you do not provide a directory path, the run-time staging files are written to /<cluster staging directory>/DATABRICKS .
Advanced Properties	List of advanced properties that are unique to the Databricks environment. You can configure run-time properties for the Databricks environment in the Data Integration Service and in the Databricks connection. You can override a property configured at a high level by setting the value at a lower level. For example, if you configure a property in the Data Integration Service custom properties, you can override it in the Databricks connection. The Data Integration Service processes property overrides based on the following priorities: Databricks connection advanced properties Data Integration Service custom properties Informatica does not recommend changing these property values before you consult with third-party documentation, Informatica documentation, or Informatica Global Customer Support. If you change a value without knowledge of the property, you might experience performance degradation or other unexpected results.

Advanced Properties

Configure the following properties in the

Advanced Properties

of the Databricks configuration section:

infaspark.json.parser.mode: Specifies the parser how to handle corrupt JSON records. You can set the value to one of the following modes:

DROPMALFORMED. The parser ignores all corrupted records. Default mode.
PERMISSIVE. The parser accepts non-standard fields as nulls in corrupted records.
FAILFAST. The parser generates an exception when it encounters a corrupted record and the Spark application goes down.

infaspark.json.parser.multiLine: Specifies whether the parser can read a multiline record in a JSON file. You can set the value to true or false. Default is false. Applies only to non-native distributions that use Spark version 2.2.x and above.

infaspark.flatfile.writer.nullValue: When the Databricks Spark engine writes to a target, it converts null values to empty strings (" "). For example, 12, AB,"",23p09udj.; The Databricks Spark engine can write the empty strings to string columns, but when it tries to write an empty string to a non-string column, the mapping fails with a type mismatch.
To allow the Databricks Spark engine to convert the empty strings back to null values and write to the target, configure the following advanced property in the Databricks Spark connection:

infaspark.flatfile.writer.nullValue=true

Rename Saved Search

Table of Contents

Administrator Guide

Administrator Guide

Databricks Connection Properties

Databricks Connection Properties

Advanced Properties