Table of Contents

Search

  1. Preface
  2. Part 1: Version 10.4.1
  3. Part 2: Version 10.4.0
  4. Part 3: Version 10.2.2
  5. Part 4: Version 10.2.1
  6. Part 5: Version 10.2
  7. Part 6: Version 10.1.1
  8. Part 7: Version 10.1

Data Preview

Data Preview

Effective in version 10.4.0, the Data Integration Service uses Spark Jobserver to preview data on the Spark engine. Spark Jobserver allows for faster data preview jobs because it maintains a running Spark context instead of refreshing the context for each job. Mappings configured to run with Amazon EMR, Cloudera CDH, and Hortonworks HDP use Spark Jobserver to preview data.
Previously, the Data Integration Service used spark-submit scripts for all data preview jobs on the Spark engine. Mappings configured to run with Azure HDInsight and MapR use spark-submit scripts to preview data on the Spark engine. Previewing data on mappings configured to run with Azure HDInsight and MapR is available for technical preview.
For more information, see the "Data Preview" chapter in the
Data Engineering Integration 10.4.0 User Guide
.

Union Transformation

Effective in version 10.4.0, you can choose a Union transformation as the preview point when you preview data. Previously, the Union transformation was not supported as a preview point.

infacmd dp Commands

You can use the infacmd dp plugin to perform data preview operations. Use infacmd dp commands to manually start and stop the Spark Jobserver.
The following table describes infacmd dp commands:
Command
Description
startSparkJobServer
Starts Spark Jobserver on the Integration Service machine.
By default, the Spark Jobserver starts when you preview hierarchical data.
stopSparkJobServer
Stops the Spark Jobserver running on specified Integration Service.
By default, the Spark Jobserver stops if it is idle for 60 minutes or when the Data Integration Service is stopped or recycled.
For more information, see the "infacmd dp Command Reference" chapter in the
Informatica 10.4.0 Command Reference
.

0 COMMENTS

We’d like to hear from you!