Table of Contents

Search

  1. Abstract
  2. Supported Versions
  3. Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Tuning and Sizing Guidelines for Data Engineering Integration (10.4.x)

Router Transformation

Router Transformation

Using more than one output group in a Router transformation might cause poor performance. To avoid performance impact, configure the following properties:
Property and Value
Description
infaspark.sql.forcePersist=true
Persists data in the memory and disk to avoid repeated reads for each output group.
spark.rdd.compress=true
Enables compression to store data more efficiently.
You can set these properties either in the Spark advanced properties for the Hadoop connection or at the mapping level. If you set the properties in the Spark configuration, the values apply to all mappings that you run on the Spark engine.

0 COMMENTS

We’d like to hear from you!